Python - Chuncked summation every K value

Chunked summation, also known as partial sum or rolling sum, is a process of calculating the sum of elements in smaller chunks or subsets rather than processing the entire sequence at once. Each chunk represents a group of consecutive elements from the sequence, and the sum is calculated for each chunk individually.

For example, consider the sequence [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], and let's calculate the chunked sum with a chunk size of 3 ?

• Chunk 1: [1, 2, 3] ? Sum: 1 + 2 + 3 = 6

• Chunk 2: [4, 5, 6] ? Sum: 4 + 5 + 6 = 15

• Chunk 3: [7, 8, 9] ? Sum: 7 + 8 + 9 = 24

• Chunk 4: [10] ? Sum: 10

The result of the chunked sum with a chunk size of 3 would be [6, 15, 24, 10].

Chunked sum is useful for processing large datasets, time series data in segments, and avoiding memory-related issues while improving computational efficiency.

Using Loops

One straightforward approach is to use a loop to iterate over the list and calculate the sum of every K values ?

def chunked_sum_loop(values, K):
    result = []
    chunk_sum = 0
    for i, value in enumerate(values, 1):
        chunk_sum += value
        if i % K == 0:
            result.append(chunk_sum)
            chunk_sum = 0
    if chunk_sum != 0:
        result.append(chunk_sum)
    return result

values = [5, 78, 787, 99, 44, 9, 22]
K = 2
print("The chunked sum of the given values:", chunked_sum_loop(values, K))

The output of the above code is ?

The chunked sum of the given values: [83, 886, 53, 22]

We initialize an empty result list to store the chunked sums and chunk_sum to keep track of each chunk's sum. Using enumerate(values, 1), we get both index and value, starting from 1. When the index is divisible by K, we append the current sum to results and reset the counter.

Using List Comprehension

List comprehension provides a more concise approach for generating chunked sums ?

def chunked_sum_list_comprehension(values, K):
    return [sum(values[i:i+K]) for i in range(0, len(values), K)]

values = [5, 78, 787, 99, 44, 9, 22]
K = 2
print("The chunked sum of the given values:", chunked_sum_list_comprehension(values, K))

The output of the above code is ?

The chunked sum of the given values: [83, 886, 53, 22]

This approach uses range(0, len(values), K) to iterate over starting indices of each chunk, then slices values[i:i+K] to extract each chunk and calculates its sum.

Using itertools.islice()

The islice() function from the itertools module provides an efficient way to extract chunks from an iterable ?

from itertools import islice

def chunked_sum_islice(values, K):
    iterator = iter(values)
    result = []
    while True:
        chunk = list(islice(iterator, K))
        if not chunk:
            break
        result.append(sum(chunk))
    return result

values = [5, 78, 7, 8, 9, 9, 4, 4, 9, 22]
K = 2
print("The chunked sum of the given values:", chunked_sum_islice(values, K))

The output of the above code is ?

The chunked sum of the given values: [83, 15, 18, 8, 31]

This method creates an iterator from the values and continuously extracts chunks of size K using islice(). When an empty chunk is returned, we've reached the end of the data.

Comparison

Conclusion

Use list comprehension for simple chunked summation with readable code. For memory-efficient processing of large datasets, prefer islice(). Use loops when you need more complex logic or custom handling.