- Apache MXNet Tutorial
- Apache MXNet - Home
- Apache MXNet - Introduction
- Apache MXNet - Installing MXNet
- Apache MXNet - Toolkits and Ecosystem
- Apache MXNet - System Architecture
- Apache MXNet - System Components
- Apache MXNet - Unified Operator API
- Apache MXNet - Distributed Training
- Apache MXNet - Python Packages
- Apache MXNet - NDArray
- Apache MXNet - Gluon
- Apache MXNet - KVStore and Visualization
- Apache MXNet - Python API ndarray
- Apache MXNet - Python API gluon
- Apache MXNet - Python API autograd and initializer
- Apache MXNet - Python API Symbol
- Apache MXNet - Python API Module
- Apache MXNet Useful Resources
- Apache MXNet - Quick Guide
- Apache MXNet - Useful Resources
- Apache MXNet - Discussion

# Apache MXNet - NDArray

In this chapter, we will be discussing about MXNet’s multi-dimensional array format called **ndarray**.

## Handling data with NDArray

First, we are going see how we can handle data with NDArray. Following are the prerequisites for the same −

### Prerequisites

To understand how we can handle data with this multi-dimensional array format, we need to fulfil the following prerequisites:

MXNet installed in a Python environment

Python 2.7.x or Python 3.x

### Implementation Example

Let us understand the basic functionality with the help of an example given below −

First, we need to import MXNet and ndarray from MXNet as follows −

import mxnet as mx from mxnet import nd

Once we import the necessary libraries, we will go with the following basic functionalities:

### A simple 1-D array with a python list

**Example**

x = nd.array([1,2,3,4,5,6,7,8,9,10]) print(x)

**Output**

The output is as mentioned below −

[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] <NDArray 10 @cpu(0)>

### A 2-D array with a python list

**Example**

y = nd.array([[1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10]]) print(y)

**Output**

The output is as stated below −

[[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] [ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] [ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.]] <NDArray 3x10 @cpu(0)>

### Creating an NDArray without any initialisation

Here, we will create a matrix with 3 rows and 4 columns by using **.empty** function. We will also use **.full** function, which will take an additional operator for what value you want to fill in the array.

**Example**

x = nd.empty((3, 4)) print(x) x = nd.full((3,4), 8) print(x)

**Output**

The output is given below −

[[0.000e+00 0.000e+00 0.000e+00 0.000e+00] [0.000e+00 0.000e+00 2.887e-42 0.000e+00] [0.000e+00 0.000e+00 0.000e+00 0.000e+00]] <NDArray 3x4 @cpu(0)> [[8. 8. 8. 8.] [8. 8. 8. 8.] [8. 8. 8. 8.]] <NDArray 3x4 @cpu(0)>

### Matrix of all zeros with the .zeros function

**Example**

x = nd.zeros((3, 8)) print(x)

**Output**

The output is as follows −

[[0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 0. 0. 0. 0.]] <NDArray 3x8 @cpu(0)>

### Matrix of all ones with the .ones function

**Example**

x = nd.ones((3, 8)) print(x)

**Output**

The output is mentioned below −

[[1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.]] <NDArray 3x8 @cpu(0)>

### Creating array whose values are sampled randomly

**Example**

y = nd.random_normal(0, 1, shape=(3, 4)) print(y)

**Output**

The output is given below −

[[ 1.2673576 -2.0345826 -0.32537818 -1.4583491 ] [-0.11176403 1.3606371 -0.7889914 -0.17639421] [-0.2532185 -0.42614475 -0.12548696 1.4022992 ]] <NDArray 3x4 @cpu(0)>

### Finding dimension of each NDArray

**Example**

y.shape

**Output**

The output is as follows −

(3, 4)

### Finding the size of each NDArray

**Example**

y.size

**Output**

12

### Finding the datatype of each NDArray

**Example**

y.dtype

**Output**

numpy.float32

## NDArray Operations

In this section, we will introduce you to MXNet’s array operations. NDArray support large number of standard mathematical as well as In-place operations.

## Standard Mathematical Operations

Following are standard mathematical operations supported by NDArray −

### Element-wise addition

First, we need to import MXNet and ndarray from MXNet as follows:

import mxnet as mx from mxnet import nd x = nd.ones((3, 5)) y = nd.random_normal(0, 1, shape=(3, 5)) print('x=', x) print('y=', y) x = x + y print('x = x + y, x=', x)

**Output**

The output is given herewith −

x= [[1. 1. 1. 1. 1.] [1. 1. 1. 1. 1.] [1. 1. 1. 1. 1.]] <NDArray 3x5 @cpu(0)> y= [[-1.0554522 -1.3118273 -0.14674698 0.641493 -0.73820823] [ 2.031364 0.5932667 0.10228804 1.179526 -0.5444829 ] [-0.34249446 1.1086396 1.2756858 -1.8332436 -0.5289873 ]] <NDArray 3x5 @cpu(0)> x = x + y, x= [[-0.05545223 -0.3118273 0.853253 1.6414931 0.26179177] [ 3.031364 1.5932667 1.102288 2.1795259 0.4555171 ] [ 0.6575055 2.1086397 2.2756858 -0.8332436 0.4710127 ]] <NDArray 3x5 @cpu(0)>

### Element-wise multiplication

**Example**

x = nd.array([1, 2, 3, 4]) y = nd.array([2, 2, 2, 1]) x * y

**Output**

You will see the following output−

[2. 4. 6. 4.] <NDArray 4 @cpu(0)>

### Exponentiation

**Example**

nd.exp(x)

**Output**

When you run the code, you will see the following output:

[ 2.7182817 7.389056 20.085537 54.59815 ] <NDArray 4 @cpu(0)>

### Matrix transpose to compute matrix-matrix product

**Example**

nd.dot(x, y.T)

**Output**

Given below is the output of the code −

[16.] <NDArray 1 @cpu(0)>

## In-place Operations

Every time, in the above example, we ran an operation, we allocated a new memory to host its result.

For example, if we write A = A+B, we will dereference the matrix that A used to point to and instead point it at the newly allocated memory. Let us understand it with the example given below, using Python’s id() function −

print('y=', y) print('id(y):', id(y)) y = y + x print('after y=y+x, y=', y) print('id(y):', id(y))

**Output**

Upon execution, you will receive the following output −

y= [2. 2. 2. 1.] <NDArray 4 @cpu(0)> id(y): 2438905634376 after y=y+x, y= [3. 4. 5. 5.] <NDArray 4 @cpu(0)> id(y): 2438905685664

In fact, we can also assign the result to a previously allocated array as follows −

print('x=', x) z = nd.zeros_like(x) print('z is zeros_like x, z=', z) print('id(z):', id(z)) print('y=', y) z[:] = x + y print('z[:] = x + y, z=', z) print('id(z) is the same as before:', id(z))

**Output**

The output is shown below −

x= [1. 2. 3. 4.] <NDArray 4 @cpu(0)> z is zeros_like x, z= [0. 0. 0. 0.] <NDArray 4 @cpu(0)> id(z): 2438905790760 y= [3. 4. 5. 5.] <NDArray 4 @cpu(0)> z[:] = x + y, z= [4. 6. 8. 9.] <NDArray 4 @cpu(0)> id(z) is the same as before: 2438905790760

From the above output, we can see that x+y will still allocate a temporary buffer to store the result before copying it to z. So now, we can perform operations in-place to make better use of memory and to avoid temporary buffer. To do this, we will specify the out keyword argument every operator support as follows −

print('x=', x, 'is in id(x):', id(x)) print('y=', y, 'is in id(y):', id(y)) print('z=', z, 'is in id(z):', id(z)) nd.elemwise_add(x, y, out=z) print('after nd.elemwise_add(x, y, out=z), x=', x, 'is in id(x):', id(x)) print('after nd.elemwise_add(x, y, out=z), y=', y, 'is in id(y):', id(y)) print('after nd.elemwise_add(x, y, out=z), z=', z, 'is in id(z):', id(z))

**Output**

On executing the above program, you will get the following result −

x= [1. 2. 3. 4.] <NDArray 4 @cpu(0)> is in id(x): 2438905791152 y= [3. 4. 5. 5.] <NDArray 4 @cpu(0)> is in id(y): 2438905685664 z= [4. 6. 8. 9.] <NDArray 4 @cpu(0)> is in id(z): 2438905790760 after nd.elemwise_add(x, y, out=z), x= [1. 2. 3. 4.] <NDArray 4 @cpu(0)> is in id(x): 2438905791152 after nd.elemwise_add(x, y, out=z), y= [3. 4. 5. 5.] <NDArray 4 @cpu(0)> is in id(y): 2438905685664 after nd.elemwise_add(x, y, out=z), z= [4. 6. 8. 9.] <NDArray 4 @cpu(0)> is in id(z): 2438905790760

## NDArray Contexts

In Apache MXNet, each array has a context and one context could be the CPU, whereas other contexts might be several GPUs. The things can get even worst, when we deploy the work across multiple servers. That’s why, we need to assign arrays to contexts intelligently. It will minimise the time spent transferring data between devices.

For example, try initialising an array as follows −

from mxnet import nd z = nd.ones(shape=(3,3), ctx=mx.cpu(0)) print(z)

**Output**

When you execute the above code, you should see the following output −

[[1. 1. 1.] [1. 1. 1.] [1. 1. 1.]] <NDArray 3x3 @cpu(0)>

We can copy the given NDArray from one context to another context by using the copyto() method as follows −

x_gpu = x.copyto(gpu(0)) print(x_gpu)

## NumPy array vs. NDArray

We all the familiar with NumPy arrays but Apache MXNet offers its own array implementation named NDArray. Actually, it was initially designed to be similar to NumPy but there is a key difference −

The key difference is in the way calculations are executed in NumPy and NDArray. Every NDArray manipulation in MXNet is done in asynchronous and non-blocking way, which means that, when we write code like c = a * b, the function is pushed to the **Execution Engine**, which will start the calculation.

Here, a and b both are NDArrays. The benefit of using it is that, the function immediately returns back, and the user thread can continue execution despite the fact that the previous calculation may not have been completed yet.

### Working of Execution Engine

If we talk about the working of execution engine, it builds the computation graph. The computation graph may reorder or combine some calculations, but it always honors dependency order.

For example, if there are other manipulation with ‘X’ done later in the programming code, the Execution Engine will start doing them once the result of ‘X’ is available. Execution engine will handle some important works for the users, such as writing of callbacks to start execution of subsequent code.

In Apache MXNet, with the help of NDArray, to get the result of computation we only need to access the resulting variable. The flow of the code will be blocked until the computation results are assigned to the resulting variable. In this way, it increases code performance while still supporting imperative programming mode.

## Converting NDArray to NumPy Array

Let us learn how can we convert NDArray to NumPy Array in MXNet.

**Combining higher-level operator with the help of few lower-level operators**

Sometimes, we can assemble a higher-level operator by using the existing operators. One of the best examples of this is, the **np.full_like()** operator, which is not there in NDArray API. It can easily be replaced with a combination of existing operators as follows:

from mxnet import nd import numpy as np np_x = np.full_like(a=np.arange(7, dtype=int), fill_value=15) nd_x = nd.ones(shape=(7,)) * 15 np.array_equal(np_x, nd_x.asnumpy())

**Output**

We will get the output similar as follows −

True

**Finding similar operator with different name and/or signature**

Among all the operators, some of them have slightly different name, but they are similar in the terms of functionality. An example of this is **nd.ravel_index()** with **np.ravel()** functions. In the same way, some operators may have similar names, but they have different signatures. An example of this is **np.split()** and **nd.split()** are similar.

Let’s understand it with the following programming example:

def pad_array123(data, max_length): data_expanded = data.reshape(1, 1, 1, data.shape[0]) data_padded = nd.pad(data_expanded, mode='constant', pad_width=[0, 0, 0, 0, 0, 0, 0, max_length - data.shape[0]], constant_value=0) data_reshaped_back = data_padded.reshape(max_length) return data_reshaped_back pad_array123(nd.array([1, 2, 3]), max_length=10)

**Output**

The output is stated below −

[1. 2. 3. 0. 0. 0. 0. 0. 0. 0.] <NDArray 10 @cpu(0)>

## Minimising impact of blocking calls

In some of the cases, we have to use either **.asnumpy()** or **.asscalar()** methods, but this will force MXNet to block the execution, until the result can be retrieved. We can minimise the impact of a blocking call by calling **.asnumpy()** or **.asscalar()** methods in the moment, when we think the calculation of this value is already done.

### Implementation Example

**Example**

from __future__ import print_function import mxnet as mx from mxnet import gluon, nd, autograd from mxnet.ndarray import NDArray from mxnet.gluon import HybridBlock import numpy as np class LossBuffer(object): """ Simple buffer for storing loss value """ def __init__(self): self._loss = None def new_loss(self, loss): ret = self._loss self._loss = loss return ret @property def loss(self): return self._loss net = gluon.nn.Dense(10) ce = gluon.loss.SoftmaxCELoss() net.initialize() data = nd.random.uniform(shape=(1024, 100)) label = nd.array(np.random.randint(0, 10, (1024,)), dtype='int32') train_dataset = gluon.data.ArrayDataset(data, label) train_data = gluon.data.DataLoader(train_dataset, batch_size=128, shuffle=True, num_workers=2) trainer = gluon.Trainer(net.collect_params(), optimizer='sgd') loss_buffer = LossBuffer() for data, label in train_data: with autograd.record(): out = net(data) # This call saves new loss and returns previous loss prev_loss = loss_buffer.new_loss(ce(out, label)) loss_buffer.loss.backward() trainer.step(data.shape[0]) if prev_loss is not None: print("Loss: {}".format(np.mean(prev_loss.asnumpy())))

**Output**

The output is cited below:

Loss: 2.3373236656188965 Loss: 2.3656985759735107 Loss: 2.3613128662109375 Loss: 2.3197104930877686 Loss: 2.3054862022399902 Loss: 2.329197406768799 Loss: 2.318927526473999