quinn caps

download quinn caps

of 84

  • date post

    02-Jun-2018
  • Category

    Documents

  • view

    222
  • download

    0

Embed Size (px)

Transcript of quinn caps

  • 8/11/2019 quinn caps

    1/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Parallel Programmingin C with MPI and OpenMP

    Michael J. Quinn

  • 8/11/2019 quinn caps

    2/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Chapter 4

    Message-Passing Programming

  • 8/11/2019 quinn caps

    3/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Learning Objectives

    Understanding how MPI programs execute

    Familiarity with fundamental MPI functions

  • 8/11/2019 quinn caps

    4/84

  • 8/11/2019 quinn caps

    5/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Message-passing Model

  • 8/11/2019 quinn caps

    6/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Processes

    Number is specified at start-up time

    Remains constant throughout execution of program

    All execute same program

    Each has unique ID number

    Alternately performs computations and communicates

  • 8/11/2019 quinn caps

    7/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Advantages of Message-passing Model

    Gives programmer ability to manage the memoryhierarchy

    Portability to many architectures

    Easier to create a deterministic program

    Simplifies debugging

  • 8/11/2019 quinn caps

    8/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    The Message Passing Interface

    Late 1980s: vendors had unique libraries

    1989: Parallel Virtual Machine (PVM) developed atOak Ridge National Lab

    1992: Work on MPI standard begun

    1994: Version 1.0 of MPI standard

    1997: Version 2.0 of MPI standard

    Today: MPI is dominant message passing librarystandard

  • 8/11/2019 quinn caps

    9/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Circuit Satisfiability

    11

    1111

    1111

    11

    111

    1

    not satisfied

    0

  • 8/11/2019 quinn caps

    10/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Solution Method

    Circuit satisfiability is NP-complete

    No known algorithms to solve in polynomial time

    We seek all solutions

    We find through exhaustive search

    16 inputs !65,536 combinations to test

  • 8/11/2019 quinn caps

    11/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Partitioning: Functional Decomposition

  • 8/11/2019 quinn caps

    12/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Agglomeration and Mapping

    Properties of parallel algorithm

    Fixed number of tasks

    No communications between tasks

    Time needed per task is variable

    Map tasks to processors in a cyclic fashion

  • 8/11/2019 quinn caps

    13/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Cyclic (interleaved) Allocation

    Assumepprocesses

    Each process gets everypthpiece of work

    Example: 5 processes and 12 pieces of work

    P0: 0, 5, 10 P1: 1, 6, 11

    P2: 2, 7

    P3: 3, 8

    P4: 4, 9

  • 8/11/2019 quinn caps

    14/84

  • 8/11/2019 quinn caps

    15/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Summary of Program Design

    Program will consider all 65,536 combinations of 16boolean inputs

    Combinations allocated in cyclic fashion to processes

    Each process examines each of its combinations

    If it finds a satisfiable combination, it will print it

  • 8/11/2019 quinn caps

    16/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Include Files

    MPI header file

    #include

    #include

  • 8/11/2019 quinn caps

    17/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Local Variables

    int main (int argc, char *argv[]) {int i;int id; /* Process rank */

    int p; /* Number of processes */void check_circuit (int, int);

  • 8/11/2019 quinn caps

    18/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Initialize MPI

    First MPI function called by each process

    Not necessarily first executable statement

    Allows system to do any necessary setup

    MPI_Init (&argc, &argv);

  • 8/11/2019 quinn caps

    19/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Communicators

    Communicator: opaque object that provides message-passing environment for processes

    MPI_COMM_WORLD

    Default communicator Includes all processes

    Possible to create new communicators

    Will do this in Chapters 8 and 9

  • 8/11/2019 quinn caps

    20/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Communicator

    MPI_COMM_WORLD

    Communicator

    0

    2

    1

    3

    4

    5

    Processes

    Ranks

    Communicator Name

  • 8/11/2019 quinn caps

    21/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Determine Number of Processes

    First argument is communicator

    Number of processes returned through secondargument

    MPI_Comm_size (MPI_COMM_WORLD, &p);

  • 8/11/2019 quinn caps

    22/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Determine Process Rank

    First argument is communicator

    Process rank (in range 0, 1, ,p-1) returned throughsecond argument

    MPI_Comm_rank (MPI_COMM_WORLD, &id);

  • 8/11/2019 quinn caps

    23/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Replication of Automatic Variables

    1id

    6p

    2id

    6p

    0id

    6p

    3id

    6p

    4id

    6p

    5id

    6p

  • 8/11/2019 quinn caps

    24/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    What about External Variables?

    int total;

    int main (int argc, char *argv[]) {

    int i;int id;int p;

  • 8/11/2019 quinn caps

    25/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Cyclic Allocation of Work

    for (i = id; i < 65536; i += p)check_circuit (id, i);

  • 8/11/2019 quinn caps

    26/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Shutting Down MPI

    Call after all other MPI library calls

    Allows system to free up MPI resources

    MPI_Finalize();

  • 8/11/2019 quinn caps

    27/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    #include #include

    int main (int argc, char *argv[]) {int i;int id;int p;void check_circuit (int, int);

    MPI_Init (&argc, &argv);MPI_Comm_rank (MPI_COMM_WORLD, &id);MPI_Comm_size (MPI_COMM_WORLD, &p);

    for (i = id; i < 65536; i += p)check_circuit (id, i);

    printf ("Process %d is done\n", id);

    fflush (stdout);MPI_Finalize();return 0;

    } Put fflush()after everyprintf()

  • 8/11/2019 quinn caps

    28/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    /* Return 1 if 'i'th bit of 'n' is 1; 0 otherwise */#define EXTRACT_BIT(n,i) ((n&(1

  • 8/11/2019 quinn caps

    29/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Compiling MPI Programs

    mpicc: script to compile and link C+MPI programs

    Flags: same meaning as C compiler -O"optimize

    -o "where to put executable

    mpicc -O -o foo foo.c

  • 8/11/2019 quinn caps

    30/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Running MPI Programs

    mpirun -np

    -np

    "number of processes

    "executable

    "command-line arguments

  • 8/11/2019 quinn caps

    31/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Specifying Host Processors

    File .mpi-machinesin home directory lists hostprocessors in order of their use

    Example .mpi_machinesfile contentsband01.cs.ppu.edu

    band02.cs.ppu.edu

    band03.cs.ppu.edu

    band04.cs.ppu.edu

  • 8/11/2019 quinn caps

    32/84

    Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

    Enabling Remote Logins

    MPI needs to be able to initiate processes on other processorswithout supplying a password

    Each processor in group must list all other processors inits .rhostsfile; e.g.,

    band01.cs.ppu.edu studentband02.cs.ppu.edu student

    band03.cs.ppu.edu student

    band04.cs.ppu.edu student

  • 8/11/2019 quinn caps

    33/84

    Copyright The McGraw-Hill Companies, Inc