LINUX & Parallel Processing
date post
08-Jan-2016Category
Documents
view
34download
0
Embed Size (px)
description
Transcript of LINUX & Parallel Processing
LINUX & Parallel ProcessingWouter BrissinckVrije Universiteit BrusselDienst [email protected]+32 2 629 2965
OverviewThe DreamThe FactsThe ToolsThe Real ThingThe Conclusion
History 60 1 computer N users80 N computers N users90 N computers 1 user
past problem = f(computer)future computer = f(problem)
The Sequential FrustrationFaster CPU : High development cost + time
Speed of light : faster smallerSize of Si atom is the ultimate limit
ParallelismOffers scalable performanceSolves as yet unsolvable problemsproblem sizecomputation timeHardware cost Software development
ScalabilityPROBLEM * p1p
Scalabilityproblem on 1 CPU takes T timep times larger problem on p CPUs takes T time
Same technology ( only more of it )Same code ( if ... )Economy of scale
Hardware cost ?Parallel machines :expensive to buyexpensive to maintainexpensive to upgradeunreliablelousy operating systems N.O.W :you own one !Its maintained for youYou just did, didnt you ?You rely on it every day !How did you call LINUX ?A network-of-workstations is a virtually free parallel machine !
OverviewThe DreamThe FactsThe ToolsThe Real ThingThe Conclusion
Problems100 painters 1 day ?
Synchronizationshared resourcesdependenciesCommunication1 painter 100 days
Amdahls Lawa = sequential fractionb = communication cost
Amdahls Law#processors=pSpeedup=Sa = sequential fractionb = communication cost
GranularityGranularity =N.O.WDedicated MIMDPipelining in CPUcoarsefine
Network latencyLT = L + len / BL=Latency(s)len=length(bits)B=Bandwidth(bits/s)proc1proc2
The N of N.O.W?
Software DesignSequential specification Parallel implementationParallel specification (less) parallel implementationPreferably : 1 process/processor
Software Design : classificationMaster - SlavePipeliningSystolic Array Functional parallelismData parallelism
OverviewThe DreamThe FactsThe ToolsThe Real ThingThe Conclusion
P.V.MPVM : overviewPVM installationUsing PVMconsolecompilingwriting codeMessage passingin Cin C++gdb and pvm
Parallel Virtual MachineMost popular Message Passing EngineWorks on MANY platforms ( LINUX, NT, SUN, CRAY, ...)Supports C, C++, fortranAllows heterogenious virtual machinesFREE software ( ftp.netlib.org )
Heterogenious NOWsPVM provides data format translation of messagesMust compile application for each kind of machineCan take advantage of specific resources in the LAN
PVM featuresUser-configured host poolApplication = set of (unix) processesautomatically mappedforced by programmerMessage Passing modelstrongly typed messagesasynchronous (blocking/non-blocking receive)Multiprocessor support
How PVM worksConsole for resource managementDeamon for each host in the virtual machinemessage passingfault tolerancePVM library provides primitives fortask spawningsend/receiveVM info, ...
How PVM worksTasks are identified by TIDs (pvmd supplied)Group server provides for groups of tasksTasks can join and leave group at willGroups can overlapUseful for multicasts
N.O.W SetupPCPCPCPCLINUXLINUXLINUXLINUXPVMMasterSlaveSlaveSlaveSlave
Example : master#include "pvm3.hmain(){int cc, tid;char buf[100];cc = pvm_spawn("hello_other", (char**)0, 0, "", 1, &tid);if (cc == 1) {cc = pvm_recv(-1, -1);pvm_bufinfo(cc, (int*)0, (int*)0, &tid);pvm_upkstr(buf);printf("from t%x: %s\n", tid, buf);} pvm_exit();}
Example : Slave#include "pvm3.h"main(){int ptid;char buf[100];ptid = pvm_parent();strcpy(buf, "hello, world from ");gethostname(buf + strlen(buf), 64);
pvm_initsend(PvmDataDefault);pvm_pkstr(buf);pvm_send(ptid, 1);
pvm_exit();}
PVM : installingSet environment variables (before build)PVM_ROOTPVM_ARCHMake defaultMake sure you can rsh without authentication
PVM and rshPVM starts tasks using rshFails if a password is required.rhosts on remote account must mention the machine where the console is runningTypical error : Cant start pvmd
Typical situationAll machines share the same user accountsAccount has these directories/files :/shared/pvm3/lib/LINUX/pvmd3/shared/pvm3/lib/SUN4/pvmd3~/pvm3/bin/LINUX/Eg.App~/pvm3/bin/SUN4/Eg.App.rhosts file looks like this : ( mymachine is the machine which runs the console )mymachinemymachine.my.domain
PVM Consoleadd hostname : add a host to VMconf : show list of hostsps -a : show all pvm processes runningManually started (i.e. not spawned) tasks are shown by a -reset : kill all pvm tasksquit : leave console, but keep deamon alivehalt : kill deamon
PVM Console : hostfilehostfile contains hosts to be added, with options
mymachine ep=$(SIM_ROOT)/bin/$(PVM_ARCH)faraway lo=woutie pw=whoknows* lo=wouterOn machine without network (just loopback):
* ip=127.0.0.1
Compiling a programApplication must be recompiled for each architecturebinary goes in ~/pvm3/bin/$(PVM_ARCH)aimk determines architecture, then calls makeMust be linked with -lpvm3 and -lgpvm3Libraries in $(PVM_ROOT)/lib/$(PVM_ARCH)
The PVM libraryEnrolling and TIDs
int tid=pvm_mytid()
int tid=pvm_parent()
int info=pvm_exit()
The PVM librarySpawning a task
flag = PvmTaskDefaultPvmTaskHost (where is host)PvmTaskDebug (use gdb to spawn)
typical error : tids[i] = -7 : executable not found
int numt=pvm_spawn(char *task, char **argv, int flag, char* where,int ntask, int *tids)
The PVM libraryMessage sending :initialize a buffer (initsend)pack the datastructuresend (send or mcast)Message receiving :blocking/non-blocking receiveunpack (the way you packed)
The PVM libraryUse pvm_initsend() to initiate sendint bufid = pvm_initsend(int encoding)Use one of the several pack functions to pack int info=pvm_pkint(int *np,int nitem, int stride)To send int info=pvm_send( int tid, int msgtag)int info=pvm_mcast(int *tids, int ntask, int msgtag)
The PVM librarypvm_recv waits till a message with a given source-tid and tag arrives (-1 is wildcard) :int bufid=pvm_recv (int tid, int msgtag)pvm_nrecv returns if no appropriate message has arrivedint bufid=pvm_nrecv (int tid, int msgtag)Use one of the several unpack functions to unpack (exactly the way you packed) int info=pvm_upkint(int *np,int nitem, int stride)
Message passing : an example
struct picture{int sizex,sizey;char* name;colourMap* cols;};
Message passing : an example
void pkPicture(picture* p){pvm_pkint(&p->sizex,1,1);pvm_pkint(&p->sizey,1,1);pvm_pkstr(p->name);pkColourMap(p->cols);};
Message passing : an examplepicture* upkPicture(){picture* p=new picture;pvm_upkint(&p->sizex,1,1);pvm_upkint(&p->sizey,1,1);p->name=new char[20];pvm_upkstr(p->name);p->cols=upkColourMap();return p;};
Message passing : an example//Im Apicture myPic;...pvm_initsend(PvmDataDefault);pkPicture(&myPic);pvm_send(B,PICTAG);
Message passing : an example//Im Bpicture* myPic;
pvm_recv(A,PICTAG);myPic=upkPicture();
In C++ ?
Messages contain data, not codeNo cute way to send and receive objectsProvide constructors that build from receive bufferReceiving end must have a list of possible objects
PVM and GDBSpawn tasks with PvmTaskDebugPvmd will call $PVM_ROOT/lib/debugger tasknameMake sure this scripts starts an xterm with a debugger running tasknameGREAT debugging tool !!
OverviewThe DreamThe FactsThe ToolsThe Real ThingThe Conclusion
Ray-TracingRay-Tracingmaster slaveData Parallel
Simulation : the Problem
Modeling : exampleHow many queues to achieve average queue length < 4 ?
Modeling : viewsModeling viewpoints :event viewprocess viewModel classificationLogic gatesQueueing netsPetri nets.
Simulation : Design Cycle
Simulation : Time InvestmentModeling & Validation = Human Thinking Simulation = Machine Thinking
Simulation : Time Investment (2)Simple Model :Short Simulation TimeLarge Modeling TimeLarge Validation TimeComplex Model :Large Simulation TimeShort Modeling TimeShort Validation Time
Parallel SimulatorsTwo Methods :Multiple-Run :same simulation, different parameters/input stimulieasy (stupid but effective)event-parallelism :distribute model over processorshard : fine granular
Modeling : causalityGlobal Event Ordering
Parallel Simulators : event parallelismIdea : simulate different events in parallelCondition : causal independence !Local Time Global Timelvt1lvt2e1e2e1e2
Parallel Simulators : event parallelismVery fine grainStrong synchronization (=limited parallelism)Strongly model dependent performanceHard to implementSoftware TOOL
Parallel Simulators : Event ParallelismFine grain specification :
Parallel Simulators : Event ParallelismCoarse grain implementation : (top view)
Parallel Simulators : Event ParallelismCoarse grain implementation : (side view)
Sequential
Sequential
Sequential
Parallel Simulator
Parallel Simulators : some results
OverviewThe DreamThe FactsThe ToolsThe Real ThingThe Conclusion
ConclusionNOW + LINUX + PVM = Scalability for free !Drawbacks :Software is hard to implementPerformance is sensitive to problemmachineimplementationSolution : software tools
ReferencesSee our web-page athttp://infoweb.vub.ac.be/~parallel