Apache MXNet - Python API Module


Advertisements

Apache MXNet’s module API is like a FeedForward model and it is easier to compose similar to Torch module. It consists of following classes −

BaseModule([logger])

It represents the base class of a module. A module can be thought of as computation component or computation machine. The job of a module is to execute forward and backward passes. It also updates parameters in a model.

Methods

Following table shows the methods consisted in BaseModule class

This method will get states from all devices
Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
fit(train_data[, eval_data, eval_metric, …]) This method trains the module parameters.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
forward_backward(data_batch) It is a convenient function, as name implies, that calls both forward and backward.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context])
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
iter_predict(eval_data[, num_batch, reset, …]) This method will iterate over predictions.
load_params(fname) It will, as name specifies, load model parameters from file.
predict(eval_data[, num_batch, …]) It will run the prediction and collects the outputs as well.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
save_params(fname) As name specifies, this function will save the model parameters to file.
score(eval_data, eval_metric[, num_batch, …]) It runs the prediction on eval_data and also evaluates the performance according to the given eval_metric.
set_params(arg_params, aux_params[, …]) This method will assign the parameter and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It set up the buckets and binds the executor for the default bucket key. This method represents the binding for a BucketingModule.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will get the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will get outputs from the previous forward computation.
get_params() It gets the current parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context]) This method will get states from all devices.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
load(prefix, epoch[, sym_gen, …]) This method will create a model from the previously saved checkpoint.
load_dict([sym_dict, sym_gen, …]) This method will create a model from a dictionary (dict) mapping bucket_key to symbols. It also shares arg_params and aux_params.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
save_checkpoint(prefix, epoch[, remove_amp_cast]) This method, as name implies, saves the current progress to the checkpoint for all buckets in BucketingModule. It is recommended to use mx.callback.module_checkpoint as epoch_end_callback to save during training.
set_params(arg_params, aux_params[,…]) As name specifies, this function will assign parameters and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
switch_bucket(bucket_key, data_shapes[, …]) It will switche to a different bucket.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

Attributes

Following table shows the attributes consisted in the methods of BaseModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
symbol As name specified, this attribute gets the symbol associated with this module.

data_shapes: You can refer the link available at https://mxnet.apache.org for details. output_shapes: More

output_shapes: More information is available at https://mxnet.apache.org/api/python

BucketingModule(sym_gen[…])

It represents the Bucketingmodule class of a Module which helps to deal efficiently with varying length inputs.

Methods

Following table shows the methods consisted in BucketingModule class

Attributes

Following table shows the attributes consisted in the methods of BaseModule class

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
Symbol As name specified, this attribute gets the symbol associated with this module.

data_shapes − You can refer the link at https://mxnet.apache.org/api/python/docs for more information.

output_shapes− You can refer the link at https://mxnet.apache.org/api/python/docs for more information.

Module(symbol[,data_names, label_names,…])

It represents a basic module that wrap a symbol.

Methods

Following table shows the methods consisted in Module class

Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
borrow_optimizer(shared_module) As name implies, this method will borrow the optimizer from a shared module.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context]) This method will get states from all devices
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
load(prefix, epoch[, sym_gen, …]) This method will create a model from the previously saved checkpoint.
load_optimizer_states(fname) This method will load an optimizer i.e. the updater state from a file.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
reshape(data_shapes[, label_shapes]) This method, as name implies, reshape the module for new input shapes.
save_checkpoint(prefix, epoch[, …]) It saves the current progress to checkpoint.
save_optimizer_states(fname) This method saves the optimizer or the updater state to a file.
set_params(arg_params, aux_params[,…]) As name specifies, this function will assign parameters and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

Attributes

Following table shows the attributes consisted in the methods of Module class

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
label_names It consists of the list of names for labels required by this module.

data_shapes: Visit the link https://mxnet.apache.org/api/python/docs/api/module for further details.

output_shapes: The link given herewith https://mxnet.apache.org/api/python/docs/api/module/index.html will offer other important information.

PythonLossModule([name,data_names,…])

The base of this class is mxnet.module.python_module.PythonModule. PythonLossModule class is a convenient module class which implements all or many of the module APIs as empty functions.

Methods

Following table shows the methods consisted in PythonLossModule class:

Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
install_monitor(mon) This method will install monitor on all executors.

PythonModule([data_names,label_names…])

The base of this class is mxnet.module.base_module.BaseModule. PythonModule class also is a convenient module class which implements all or many of the module APIs as empty functions.

Methods

Following table shows the methods consisted in PythonModule class −

Methods Definition
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

Attributes

Following table shows the attributes consisted in the methods of PythonModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.

data_shapes − Follow the link https://mxnet.apache.org for details.

output_shapes − For more details, visit the link available at https://mxnet.apache.org

SequentialModule([logger])

The base of this class is mxnet.module.base_module.BaseModule. SequentialModule class also is a container module that can chain more than two (multiple) modules together.

Methods

Following table shows the methods consisted in SequentialModule class

Methods Definition
add(module, **kwargs) This is most important function of this class. It adds a module to the chain.
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

Attributes

Following table shows the attributes consisted in the methods of BaseModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.

data_shapes − The link given herewith https://mxnet.apache.org will help you in understanding the attribute in much detail.

output_shapes − Follow the link available at https://mxnet.apache.org/api for details.

Implementation Examples

In the example below, we are going create a mxnet module.

import mxnet as mx
input_data = mx.symbol.Variable('input_data')
f_connected1 = mx.symbol.FullyConnected(data, name='f_connected1', num_hidden=128)
activation_1 = mx.symbol.Activation(f_connected1, name='relu1', act_type="relu")
f_connected2 = mx.symbol.FullyConnected(activation_1, name = 'f_connected2', num_hidden = 64)
activation_2 = mx.symbol.Activation(f_connected2, name='relu2',
act_type="relu")
f_connected3 = mx.symbol.FullyConnected(activation_2, name='fc3', num_hidden=10)
out = mx.symbol.SoftmaxOutput(f_connected3, name = 'softmax')
mod = mx.mod.Module(out)
print(out)

Output

The output is mentioned below −

<Symbol softmax>

Example

print(mod)

Output

The output is shown below −

<mxnet.module.module.Module object at 0x00000123A9892F28>

In this example below, we will be implementing forward computation

import mxnet as mx
from collections import namedtuple
Batch = namedtuple('Batch', ['data'])
data = mx.sym.Variable('data')
out = data * 2
mod = mx.mod.Module(symbol=out, label_names=None)
mod.bind(data_shapes=[('data', (1, 10))])
mod.init_params()
data1 = [mx.nd.ones((1, 10))]
mod.forward(Batch(data1))
print (mod.get_outputs()[0].asnumpy())

Output

When you execute the above code, you should see the following output −

[[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]]

Example

data2 = [mx.nd.ones((3, 5))]

mod.forward(Batch(data2))
print (mod.get_outputs()[0].asnumpy())

Output

Given below is the output of the code −

[[2. 2. 2. 2. 2.]
[2. 2. 2. 2. 2.]
[2. 2. 2. 2. 2.]]
Advertisements