What is Distributed-Memory Multicomputer in Computer Architecture?

A distributed-memory multicomputer system is modeled in the figure. The system includes multiple computers known as nodes, related by a message-passing network. Each node is an independent computer including a processor, local memory, and sometimes connected disks or I/O peripherals.

The message-passing network supports point-to-point static connections among the nodes. All local memories are private and are applicable only by local processors. For this method, traditional multicomputer has been referred to as no-remote-memory-access (NORMA) machines.

This condition will constantly be deleted in future multicomputer with distributed shared memories. Internode communication is carried out by passing messages through the static connection network.

Multicomputer Generations − Modern multicomputer use hardware routers to pass messages. A computer node is attached to each router. The boundary router may be connected to I/O and peripheral devices.

Message passing between any two nodes involves a sequence of routers and channels. Mixed types of nodes are allowed in a heterogeneous multicomputer. The internode communications in a heterogeneous multicomputer are achieved through compatible data representations and message-passing protocols.

Message-passing multicomputer has gone through two generations of development, and a new generation is emerging.

The first-generation (1983-1987) was based on processor board technology using hypercube architecture and software-controlled message switching. The Caltech Cosmic and Intel iPSC/1 represented the first-generation development.

The second-generation (1988-1992) was implemented with mesh-connected architecture, hardware message routing, and a software environment for medium-grain distributed computing, as represented by the Intel Paragon and the Parsys SuperNode 1000.

The emerging third generation (1993-1997) is expected to be a fine-grain multicomputer, like the MIT J-Machine and Caltech Mosaic, implemented with both processor and communication gears on the same VLSI chip.

There are popular topologies include the ring, tree, mesh, torus, hypercube, cube-connected cycle, etc. There are various communication patterns are demanded among the nodes, such as one-to-one, broadcasting, permutations, and multicast patterns.

There are important issues for multicomputer include message-routing schemes, network flow control strategies, deadlock avoidance, virtual channels, message-passing primitives, and program decomposition techniques.