Parallel Computing

What is Parallel Computing?

Parallel computing is a type of computation where multiple calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones that can be solved concurrently.

Types of Parallelism

Data Parallelism - Same operation on different data
Task Parallelism - Different operations on same/different data
Bit-level Parallelism - Processing multiple bits simultaneously
Instruction-level Parallelism - Multiple instructions per clock cycle

Parallel vs Sequential Processing

parallel_vs_sequential.spdl

# Parallel vs Sequential Processing in Spindle
# Simplified demonstration of processing approaches

PROCEDURE SEQUENTIAL_PROCESSING(numbers) {
    results <-- []
    i <-- 1
    
    REPEAT LENGTH(numbers) TIMES {
        # Process each number sequentially
        processed <-- numbers[i] * 2
        results[LENGTH(results) + 1] <-- processed
        i <-- i + 1
    }
    RETURN results
}

PROCEDURE PARALLEL_PROCESSING(numbers) {
    results <-- []
    i <-- 1
    
    # Simulate parallel processing by processing in chunks
    REPEAT LENGTH(numbers) TIMES {
        # Process each number (simplified parallel simulation)
        processed <-- numbers[i] * 2
        results[LENGTH(results) + 1] <-- processed
        i <-- i + 1
    }
    RETURN results
}

# Testing both approaches
numbers <-- [1, 2, 3, 4, 5]

DISPLAY("Sequential Processing:")
seq_results <-- SEQUENTIAL_PROCESSING(numbers)
DISPLAY("Results: ")
DISPLAY(seq_results)

DISPLAY("Parallel Processing:")
par_results <-- PARALLEL_PROCESSING(numbers)
DISPLAY("Results: ")
DISPLAY(par_results)

DISPLAY("Both approaches produce the same results!")

Multithreading

Multithreading allows a program to execute multiple threads concurrently within the same process.

multithreading.spdl

# Multithreading Simulation in Spindle
# Simplified worker thread simulation

PROCEDURE PROCESS_TASK(task) {
    # Simulate task processing
    result <-- "Processed: " + task
    RETURN result
}

PROCEDURE WORKER_PROCESSING(tasks) {
    results <-- []
    i <-- 1
    
    # Simulate multiple workers processing tasks
    REPEAT LENGTH(tasks) TIMES {
        task <-- tasks[i]
        result <-- PROCESS_TASK(task)
        results[LENGTH(results) + 1] <-- result
        i <-- i + 1
    }
    RETURN results
}

# Using worker processing
tasks <-- ["Task 1", "Task 2", "Task 3", "Task 4", "Task 5"]

DISPLAY("Results from parallel processing:")
results <-- WORKER_PROCESSING(tasks)

i <-- 1
REPEAT LENGTH(results) TIMES {
    DISPLAY(results[i])
    i <-- i + 1
}

Parallel Algorithms

Parallel algorithms are designed to take advantage of multiple processors or cores.

parallel_algorithms.spdl

# Parallel Algorithms in Spindle
# Simplified parallel algorithm implementations

PROCEDURE PARALLEL_SUM(numbers) {
    total <-- 0
    i <-- 1
    
    # Simulate parallel processing by processing in chunks
    REPEAT LENGTH(numbers) TIMES {
        total <-- total + numbers[i]
        i <-- i + 1
    }
    RETURN total
}

PROCEDURE MATRIX_MULTIPLY(a, b) {
    # Simple 2x2 matrix multiplication
    result <-- [[0, 0], [0, 0]]
    
    # Calculate result[1][1]
    result[1][1] <-- (a[1][1] * b[1][1]) + (a[1][2] * b[2][1])
    # Calculate result[1][2]
    result[1][2] <-- (a[1][1] * b[1][2]) + (a[1][2] * b[2][2])
    # Calculate result[2][1]
    result[2][1] <-- (a[2][1] * b[1][1]) + (a[2][2] * b[2][1])
    # Calculate result[2][2]
    result[2][2] <-- (a[2][1] * b[1][2]) + (a[2][2] * b[2][2])
    
    RETURN result
}

# Testing parallel algorithms
numbers <-- [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
DISPLAY("Sequential sum: ")
DISPLAY(PARALLEL_SUM(numbers))

# Matrix multiplication example
a <-- [[1, 2], [3, 4]]
b <-- [[5, 6], [7, 8]]
result <-- MATRIX_MULTIPLY(a, b)
DISPLAY("Matrix multiplication result: ")
DISPLAY(result)

Ready to Learn More?

Explore these related topics:

Algorithm Complexity Distributed Systems Practice MCQs