Mpi Program to Calculate Sum of N Numbers

Introduction

The Message Passing Interface (MPI) is a standardized and portable message-passing system designed to function on a wide variety of parallel computers. It provides a library specification of routines, datatypes, and environment management calls that form a complete interface for programs written in C, C++, and Fortran.

In this guide, we'll focus on creating an MPI program that calculates the sum of N numbers distributed across multiple processes. This is a fundamental parallel computing task that demonstrates how to use MPI's communication capabilities.

MPI Basics

Key Concepts

• Processes: Independent programs that run in parallel
• Communicator: A group of processes that can communicate
• Rank: Unique identifier for each process within a communicator
• Message Passing: Sending data between processes

MPI Initialization

All MPI programs must start with initialization and end with finalization:

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);
    // Program logic
    MPI_Finalize();
    return 0;
}

Sum Calculation

The sum calculation involves distributing numbers among processes, calculating partial sums, and then combining the results. Here's the approach:

1. Process 0 reads all numbers and distributes them
2. Each process calculates a partial sum of its assigned numbers
3. All partial sums are collected and summed by process 0

Formula

Example Program

Here's a complete C program that calculates the sum of N numbers using MPI:

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);

    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    int N = 1000; // Total numbers to sum
    int local_N = N / size; // Numbers per process
    int* numbers = NULL;
    int local_sum = 0, total_sum = 0;

    if (rank == 0) {
        numbers = (int*)malloc(N * sizeof(int));
        // Initialize numbers (example: 1 to N)
        for (int i = 0; i < N; i++) {
            numbers[i] = i + 1;
        }
    }

    // Scatter numbers to all processes
    int* local_numbers = (int*)malloc(local_N * sizeof(int));
    MPI_Scatter(numbers, local_N, MPI_INT,
                local_numbers, local_N, MPI_INT,
                0, MPI_COMM_WORLD);

    // Calculate local sum
    for (int i = 0; i < local_N; i++) {
        local_sum += local_numbers[i];
    }

    // Reduce all local sums to get total sum
    MPI_Reduce(&local_sum, &total_sum, 1, MPI_INT,
               MPI_SUM, 0, MPI_COMM_WORLD);

    if (rank == 0) {
        printf("Total sum: %d\n", total_sum);
        free(numbers);
    }

    free(local_numbers);
    MPI_Finalize();
    return 0;
}

Compilation and Execution

To compile and run this program:

1. Save the code to a file named mpi_sum.c
2. Compile with MPI: mpicc mpi_sum.c -o mpi_sum
3. Run with multiple processes: mpirun -np 4 ./mpi_sum

Performance Considerations

When implementing parallel sum calculations, consider these factors:

• Load Balancing: Ensure numbers are evenly distributed
• Communication Overhead: Minimize data transfer between processes
• Memory Usage: Each process should only store its portion of data
• Scalability: The program should work efficiently with different numbers of processes