Apache MXNet - System Architecture

This chapter will help you in understanding about the MXNet system architecture. Let us begin by learning about the MXNet Modules.

MXNet Modules

The diagram below is the MXNet system architecture and it shows the major modules and components of MXNet modules and their interaction.

In the above diagram −

The modules in blue color boxes are User Facing Modules.
The modules in green color boxes are System Modules.
Solid arrow represents high dependency, i.e. heavily rely on the interface.
Dotted arrow represents light dependency, i.e. Used data structure for convenience and interface consistency. In fact, it can be replaced by the alternatives.

Let us discuss more about user facing and system modules.

The user-facing modules are as follows −

NDArray − It provides flexible imperative programs for Apache MXNet. They are dynamic and asynchronous n-dimensional arrays.
KVStore − It acts as interface for efficient parameter synchronization. In KVStore, KV stands for Key-Value. So, it a key-value store interface.
Data Loading (IO) − This user facing module is used for efficient distributed data loading and augmentation.
Symbol Execution − It is a static symbolic graph executor. It provides efficient symbolic graph execution and optimization.
Symbol Construction − This user facing module provides user a way to construct a computation graph i.e. net configuration.

The system modules are as follows −

Storage Allocator − This system module, as name suggests, allocates and recycle memory blocks efficiently on host i.e. CPU and different devices i.e. GPUs.
Runtime Dependency Engine − Runtime dependency engine module schedules as well as executes the operations as per their read/write dependency.
Resource Manager − Resource Manager (RM) system module manages global resources like the random number generator and temporal space.
Operator − Operator system module consists of all the operators that define static forward and gradient calculation i.e. backpropagation.