MapReduce Framework - Problem

Build a simple MapReduce framework that distributes map and reduce tasks across multiple threads.

Your framework should:

Take an input array of integers and a number of threads
Split the input into chunks for parallel processing
Apply a map function (square each number) using multiple threads
Apply a reduce function (sum all results) to combine the outputs
Return the final reduced result

The framework should handle thread synchronization and data partitioning automatically.

Note: For simplicity, the map function is fixed to square numbers, and the reduce function sums the results.

Input & Output

Example 1 — Basic Case

$ Input: data = [1,2,3,4], numThreads = 2

› Output: 30

💡 Note: Split into [1,2] and [3,4]. Thread 1: 1² + 2² = 5, Thread 2: 3² + 4² = 25. Final: 5 + 25 = 30

Example 2 — Single Thread

$ Input: data = [2,3], numThreads = 1

› Output: 13

💡 Note: Only one thread processes all: 2² + 3² = 4 + 9 = 13

Example 3 — More Threads than Data

$ Input: data = [5], numThreads = 3

› Output: 25

💡 Note: Only one thread needed: 5² = 25. Extra threads remain idle

Constraints

1 ≤ data.length ≤ 10⁴
1 ≤ numThreads ≤ 16
-10³ ≤ data[i] ≤ 10³

Visualization

Tap to expand

Asked in

G Google 45 a Amazon 38 f Facebook 32 M Microsoft 28

The key insight is to partition data across threads where each thread performs both map (square) and local reduce (sum) operations, then combine partial results. The optimal approach uses thread pools with proper synchronization. Time: O(n/k) with k threads, Space: O(k) for partial results.

Common Approaches

✓ Multi-threaded MapReduce

⏱️ Time: O(n/k) Space: O(k)

Split the input array into chunks, assign each chunk to a separate thread for parallel processing. Each thread computes local sum of squares, then combine all partial results.

Sequential Processing

⏱️ Time: O(n) Space: O(1)

Process each element one by one in the main thread. Apply the map function (square) to each number, then sum all results. No parallelism or threading involved.

Multi-threaded MapReduce — Algorithm Steps

Partition input array into equal chunks based on number of threads
Create worker threads, each processes one chunk
Each thread maps (squares) its numbers and reduces (sums) locally
Main thread waits for all workers and sums their partial results

Visualization

Tap to expand

Step-by-Step Walkthrough

Partition Data

Split array into chunks for each thread

Parallel Map

Each thread squares its assigned numbers

Parallel Reduce

Each thread sums its local results

Final Reduce

Combine all partial sums

Code -

solution.c — C

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>

typedef struct {
    int* data;
    int start;
    int end;
    int result;
} WorkerArgs;

void* worker(void* arg) {
    WorkerArgs* args = (WorkerArgs*)arg;
    int localSum = 0;
    for (int i = args->start; i < args->end; i++) {
        localSum += args->data[i] * args->data[i];
    }
    args->result = localSum;
    return NULL;
}

int solution(int* data, int dataSize, int numThreads) {
    if (dataSize == 0 || numThreads <= 0) {
        return 0;
    }
    
    int chunkSize = (dataSize + numThreads - 1) / numThreads;
    pthread_t* threads = malloc(numThreads * sizeof(pthread_t));
    WorkerArgs* args = malloc(numThreads * sizeof(WorkerArgs));
    
    int activeThreads = 0;
    for (int i = 0; i < numThreads; i++) {
        int start = i * chunkSize;
        int end = (start + chunkSize < dataSize) ? start + chunkSize : dataSize;
        
        if (start < dataSize) {
            args[i].data = data;
            args[i].start = start;
            args[i].end = end;
            args[i].result = 0;
            pthread_create(&threads[i], NULL, worker, &args[i]);
            activeThreads++;
        }
    }
    
    int totalSum = 0;
    for (int i = 0; i < activeThreads; i++) {
        pthread_join(threads[i], NULL);
        totalSum += args[i].result;
    }
    
    free(threads);
    free(args);
    return totalSum;
}

int main() {
    char line[10000];
    fgets(line, sizeof(line), stdin);
    
    int data[1000];
    int dataSize = 0;
    char* ptr = line + 1;
    char* token = strtok(ptr, ",]");
    while (token != NULL) {
        data[dataSize++] = atoi(token);
        token = strtok(NULL, ",]");
    }
    
    int numThreads;
    scanf("%d", &numThreads);
    
    int result = solution(data, dataSize, numThreads);
    printf("%d\n", result);
    return 0;
}

Time & Space Complexity

Time Complexity

⏱️

O(n/k)

n elements processed across k threads in parallel, assuming sufficient CPU cores

✓ Linear Growth

Space Complexity

O(k)

Need to store k partial results from worker threads

✓ Linear Space

23.4K Views

Medium Frequency

~35 min Avg. Time

892 Likes

Ln 1, Col 1

Smart Actions

💡 Explanation

AI Ready

💡 Suggestion Tab to accept Esc to dismiss

// Output will appear here after running code

Code Editor Closed

Click the red button to reopen

MapReduce Framework - Problem

Input & Output

Constraints

Visualization

Related Problems

Common Approaches

Multi-threaded MapReduce — Algorithm Steps

Visualization

Code -

Time & Space Complexity

Select Compiler