ZeroMQ - Performance Considerations

---

---

In general, performance considerations are as important as the success of applications and cloud-based solutions.

There are several points that we should keep in mind to optimize the performance of ZeroMQ (ZMQ). ZeroMQ is considered a high-performance, asynchronous messaging library, but we must consider both hardware and software factors to make the most use of it.

The following are the number of factors that affect the ZeroMQPerformance −

• Memory Usage
• Latency
• Throughput
• Stack

Let's discuss the above factors in detail −

Memory Usage

ZeroMQ is considered to be used as thin as possible, so that it can run even in a platforms having limited memory. The low memory footprint is also important to optimize the use of the L1i cache. When the code size is small enough the processor can keep the entire messaging code in the L1i cache and avoid slow accesses to physical memory to get new parts of the code.

The part of the library holding the actual code is 80kB long and the shared library is about 350kB on Linux platforms. Still, a lot of the code is inline, so it is placed as inline functions in header files rather than in libraries.

The following tables shows the memory usage as reported by top utility −

Latency

Latency indicates the time delay between a cause and its effect, usually in a communication system or process. In the context of computing and networking, latency measures the time it takes for data to travel from one point to another or for a system to respond to an action.

We have latency measured of 40 microseconds for 1-byte messages, with the networking stack taking 25 microseconds and ZeroMQ taking 15 microseconds. Using 10GbE at a TCP latency of 15 microseconds, we would expect latency to be around 30 microseconds. We also have a solution in which we bypass the OS kernel to talk directly to the hardware which aims to reduce the latency below 20 microseconds.

Low latency: If low latency is very important, the ZMQ_IMMEDIATE socket option should be used to prevent any buffering. It is also advisable to use in-proc (in memory messaging between thread) or inter process communication sockets instead of TCP since they are less costly.

Throughput

Throughput is nothing but the rate at which at which ZeroMQ sends and receives messages over a network. This is measured in message per second (mps) or even data volume per second (mb/s).

Throughput is affected by the following factors, including message size, socket types, network bandwidth, and system performance.

• Message Size: The larger message reduce the rate of message sent per second, while smaller message increase it.
• Socket type: The socket types push-pull or pub-sub can impact how message are distributed and how fast they are processed.
• Batching: Sending message in batches reduces overhead and increase the throughput.
• Network Bandwidth: The higher bandwidth allows more data to flow through, improving the throughput.
• System Resources: CPU, memory, and thread utilization affect how efficiently messages are processed.

Stack

In ZeroMQ, the stack refers to the layers of software and components required to send and receive messages. The entire stack, including network devices, NICs, processors, and operating systems, can affect results.It is generally composed of −

Application Layer

It is a layer where our application code interacts with ZeroMQ using its API. We define how messages are sent and received between different sockets (PUB-SUB, PUSH-PULL, etc.).

Messaging Layer

This layer manages message queuing, dispatch and routing based on the socket pattern we choose such as REQ-REP, PUB-SUB. It handles message buffering, batching, and flow control.

Transport Layer

This layer decides how to deliver a message depending on the chosen transport protocols. The following are the transport protocols:

• inproc: It is an in-process communication (threads within the same process).
• ipc: It is an inter-process communication (process on the same machine).
• tcp: It is used to communicate over a TCP network.
• pgm/epgm: It is a multicast transport protocol for broadcast.

Network Stack

ZeroMQ uses a system's networking layer like TCP/IP, which handles lower-level tasks such as packet transmission, retransmission in the case of TCP, and congestion control.

Hardware

The physical hardware, including the network interface, drives the actual transmission and receipt of the message.