- Concurrency in Python Tutorial
- Home
- Introduction
- Concurrency vs Parallelism
- System & Memory Architecture
- Threads
- Implementation of Threads
- Synchronizing Threads
- Threads Intercommunication
- Testing Thread Applications
- Debugging Thread Applications
- Benchmarking & Profiling
- Pool of Threads
- Pool of Processes
- Multiprocessing
- Processes Intercommunication
- Event-Driven Programming
- Reactive Programming
- Concurrency in Python Resources
- Quick Guide
- Useful Resources
- Discussion
System and Memory Architecture
There are different system and memory architecture styles that need to be considered while designing the program or concurrent system. It is very necessary because one system & memory style may be suitable for one task but may be error prone to other task.
Computer system architectures supporting concurrency
Michael Flynn in 1972 gave taxonomy for categorizing different styles of computer system architecture. This taxonomy defines four different styles as follows −
- Single instruction stream, single data stream (SISD)
- Single instruction stream, multiple data stream (SIMD)
- Multiple instruction stream, single data stream (MISD)
- Multiple instruction stream, multiple data stream (MIMD).
Single instruction stream, single data stream (SISD)
As the name suggests, such kind of systems would have one sequential incoming data stream and one single processing unit to execute the data stream. They are just like uniprocessor systems having parallel computing architecture. Following is the architecture of SISD −
Advantages of SISD
The advantages of SISD architecture are as follows −
- It requires less power.
- There is no issue of complex communication protocol between multiple cores.
Disadvantages of SISD
The disadvantages of SISD architecture are as follows −
- The speed of SISD architecture is limited just like single-core processors.
- It is not suitable for larger applications.
Single instruction stream, multiple data stream (SIMD)
As the name suggests, such kind of systems would have multiple incoming data streams and number of processing units that can act on a single instruction at any given time. They are just like multiprocessor systems having parallel computing architecture. Following is the architecture of SIMD −
The best example for SIMD is the graphics cards. These cards have hundreds of individual processing units. If we talk about computational difference between SISD and SIMD then for the adding arrays [5, 15, 20] and [15, 25, 10], SISD architecture would have to perform three different add operations. On the other hand, with the SIMD architecture, we can add then in a single add operation.
Advantages of SIMD
The advantages of SIMD architecture are as follows −
Same operation on multiple elements can be performed using one instruction only.
Throughput of the system can be increased by increasing the number of cores of the processor.
Processing speed is higher than SISD architecture.
Disadvantages of SIMD
The disadvantages of SIMD architecture are as follows −
- There is complex communication between numbers of cores of processor.
- The cost is higher than SISD architecture.
Multiple Instruction Single Data (MISD) stream
Systems with MISD stream have number of processing units performing different operations by executing different instructions on the same data set. Following is the architecture of MISD −
The representatives of MISD architecture do not yet exist commercially.
Multiple Instruction Multiple Data (MIMD) stream
In the system using MIMD architecture, each processor in a multiprocessor system can execute different sets of instructions independently on the different set of data set in parallel. It is opposite to SIMD architecture in which single operation is executed on multiple data sets. Following is the architecture of MIMD −
A normal multiprocessor uses the MIMD architecture. These architectures are basically used in a number of application areas such as computer-aided design/computer-aided manufacturing, simulation, modeling, communication switches, etc.
Memory architectures supporting concurrency
While working with the concepts like concurrency and parallelism, there is always a need to speed up the programs. One solution found by computer designers is to create shared-memory multi-computers, i.e., computers having single physical address space, which is accessed by all the cores that a processor is having. In this scenario, there can be a number of different styles of architecture but following are the three important architecture styles −
UMA (Uniform Memory Access)
In this model, all the processors share the physical memory uniformly. All the processors have equal access time to all the memory words. Each processor may have a private cache memory. The peripheral devices follow a set of rules.
When all the processors have equal access to all the peripheral devices, the system is called a symmetric multiprocessor. When only one or a few processors can access the peripheral devices, the system is called an asymmetric multiprocessor.
Non-uniform Memory Access (NUMA)
In the NUMA multiprocessor model, the access time varies with the location of the memory word. Here, the shared memory is physically distributed among all the processors, called local memories. The collection of all local memories forms a global address space which can be accessed by all the processors.
Cache Only Memory Architecture (COMA)
The COMA model is a specialized version of the NUMA model. Here, all the distributed main memories are converted to cache memories.
