Gennaro Cordasco , Rosario De Chiara Ugo Erra and … › ~cordasco › papers › ICUMT09 ›...

Gennaro Cordasco1, Rosario De Chiara1,

Ugo Erra2 and Vittorio Scarano1

1ISISLab-Dipartimento di

Informatica ed Applicazioni

Università degli Studi di Salerno

Italy

2Dipartimento di

Matematica e Informatica

Università degli studi della Basilicata

Italy

Outline

� Massive Simulation Environment (MSE)

� Motivation and Challenge

� Distributed Massive Simulation Environment (DMSE)

Issues in designing DMSEs� Issues in designing DMSEs

� Workload Balancing: The Prediction Binary Tree (PBT)

� Our Proposal

� Conclusion

International Conference on Ultra Modern Telecommunication (ICUMT 2009) 2

Massive Simulation Environment (MSE)

� 3D environment simulating a group of characters (aka

actors) whose behaviors are related to social

interactions and rules among group members

� MSEs have been investigated since the 1980s. (Military � MSEs have been investigated since the 1980s. (Military

Simulations)

� Early works take inspiration from particles system [R83]

� Trends:

� Larger scale

� More realistic simulation



Flock Simulation:� [Reynolds87] Steering Behaviors

For Autonomous Characters

� [DEST04] Massive simulation

using GPU of a distributed

behavioral model of a flock with

Separation Alignment Cohesion

4

behavioral model of a flock with

obstacle avoidance

� [EFS09] A GPU-based Method for Massive Simulation of Distributed Behavioral Models with CUDA

Boid model� The boid model can be expanded by assembling basic

behaviors to obtain more complex behaviors:

Seek and Flee Pursuit and Evade Obstacle avoidance


Seek and Flee Pursuit and Evade Obstacle avoidance

Alignment, Separation,

Cohesion

Flee, Pursuit, Offset Pursuit,

Seek, Evade


Cohesion

Flee, Pursuit, Offset Pursuit,

Seek, Evade


Cohesion

L/R flank, L/R Face, March

forward, at ease

Enemy pursuit, shooting,

evasion

Boid model

behaviors

Basic

behaviors

Directional

commands

Combat

commands

[Reynolds 1987] [Reynolds 1999 ] [BDS09]


� [BDS09] Massive Battle: Coordinated Movement of

Autonomous Agents Battle

6

Platoons are able to march following a path;

7

MVE vs MSE

World of Warcraft

8

Massive Battle

International Conference on Ultra Modern Telecommunication (ICUMT 2009)

DMSE Challenges

� Scalability

A single machine is not able to manage

thousands of player at the same time

(Massive battle is able of simulating at

DMSE

(Distributed

Massive (Massive battle is able of simulating at

most 5000 units)

� Responsiveness

� Consistency

� Reliability


Massive

Simulation

Environment)

P2P-DMSE

� Each peer has two

subsystems:

� Simulation engine

� Visualization engine

Simulation

Engine

Visualization

Engine� Visualization engine

� Each engine defines its AOI (Area of interest)

� Simulation AOI (Region simulated and neighborhood)

� Visualization AOI (Field of view of the visualization

camera)


Engine

10

P2P-DMSE

Simulation

Engine

Visualization

EngineEngine

Issues in designing DMSEs

� World Partitioning

� Load Balancing

� World State Propagation


Similarity with Parallel Computing

� Minimize the completion time

� Decomposition: divide the whole job into tasks

� Tasks granularity

Load

Balancing

Communication

Costs

Locality of

Interaction

Spatial

Coherence

World Partitioning

� Mapping: allocate tasks to workers (peers) aiming to:

� Balance the overall load distribution

� Minimize tasks inter-processor dependency


Balancing Costs Interaction Coherence

Fine-grained� xxxx xxxx xxxx

Coarse-grained xxxx � � �

Load Balancing

World State Propagation

World PartitioningUsing a DHT we can easily partition the environment map:

The peer whose ID is the closest to the region ID become the region

master for the region. Region masters are responsible to:

� Simulate all the actors which belong to the region;

� Deliver the state of the region (that is the state of each actor which

belongs to the region) to the peers whose AOI overlaps with the

region; region;

� Handle handovers of actors between regions.

14

Underlying networkDHT ID space

Peer

Region

Peer

Region

World Partitioning

Issues:

� Partitioning

� Static

� Dynamic

� Granularity

� Fine

Load balancing

Intertask communication

Locality of interaction

� Coarse

15


Peer

Region

Peer

Region

Locality of interaction

World State Propagation

Publish/Subscribe design pattern:

� a multicast channel is assigned to each region

� users then simply subscribe to the channels associated

with the regions which overlap with their AOIwith the regions which overlap with their AOI

� the granularity of the World partitioning is again an

issue


Interaction between zone

� A graph G=(N,E) is used to model zone and their

dependencies:

� Each node v ∈ N represents a zone to be simulated

� Edge among nodes represents zone dependencies� Edge among nodes represents zone dependencies


17

Tiled Mapping Strategies

� We are interested in tiled mapping strategies where the

whole map is partitioned into Regions (contiguous 2-

dimensional blocks of zones).

� Tiled mapping allows to exploit the local dependencies � Tiled mapping allows to exploit the local dependencies

among zones:

� Locality of interactions

� Spatial coherence


Step-wise computations

� Simulations are performed in successive phases

� Temporal coherence: The amount of time required by a

Region r in phase f is comparable to the amount of time

required by r in phase f+1

Phase f Phase f+1


Phase f Phase f+1Region r

The Prediction Binary Tree [CCDES08]

� A semi-static coarse-grained decomposition/mapping

strategy for parallel mesh-like computation which exploits

temporal coherence

� semi-static: decisions are made before each computing � semi-static: decisions are made before each computing

phase

� coarse-grained: to reduce communication and exploit the

local dependencies among tasks

� mesh-like: the strategy is based on tiled mapping

� temporal coherence is exploited in order to balance task

sizes


The Prediction Binary Tree (PBT)

� The PBT is in charge directing the tiling-based mapping

strategy:

� Each computing phase is split into m tiles

� Each leaf of the PBT represents a tile



� Before each new phase tile sizes are adjusted according

to the previous phase


light tasks

heavy task


� Before each new phase tile sizes are adjusted according

to the previous phase


Improved balancing

Our Proposal

Hybrid approach

Peer Subsystems:

� Simulation engine

provides computing resource to a distributed simulation � provides computing resource to a distributed simulation

environment

� uses a DHT as a Data Storage and for World states propagation

� Visualization engine

� works on its own


Our Proposal: Load Balancing

� Two level approach:

� Coarse grained static decomposition of the whole Map

Environment into Regions. Regions are assigned to nodes

(region masters) using a DHT scheme(region masters) using a DHT scheme

25


Peer

Region

Peer

Region


� Two level approach:

� Coarse grained static decomposition of the whole Map

Environment into Regions. Regions are assigned to nodes

(region masters) using a DHT scheme(region masters) using a DHT scheme

� Each region is dynamically partitioned exploiting the PBT

idea

26International Conference on Ultra Modern Telecommunication (ICUMT 2009)


27

Underlying network

DHT ID space

Region dynamic partition

Peer

Static region

Dynamic sub-region

Peer

Static region

Dynamic sub-region

PBT

Our Proposal



Peer

Static region

Dynamic sub-region

Peer

Static region

Dynamic sub-region

PBT

� A forest of PBT maintained by Region masters: After each

computing phase the PBT is updated

� Region master is also responsible of keep interested

nodes aware about the state of the PBT

� The simulation is performed by the nodes which are

responsible of the PBT leaves



Conclusion

� MSEs due to their scalability requirement appear to be a

natural application for P2P architectures

� As any distributed computing platform, DMSEs based on

P2P approach present several issues (Load balancing /

Communication trade-off).Communication trade-off).

� We presented a P2P architecture which exploits the

feature of both DHT schemes and a novel data structure

(PBT) in order to provide a balanced workload without

introducing excessive communication.

� We are working on the implementation of this

architecture for a distributed version of Massive Battle


Salerno

ITALY

Gennaro Cordasco, Rosario De Chiara, Ugo Erra, Vittorio Scarano

http://www.isislab.it

References[R83] Reeves W. T., “Particle Systems-A Technique for Modeling a Class of Fuzzy Objects”. In ACM

Transactions on Graphics, V2–2, April 1983. and reprinted in Computer Graphics. V17–3, July

1983, (ACM SIGGRAPH ’83 Proceedings), pp. 359-376.

[R87] Reynolds C., “ Flocks, herds and schools: a distributed behavioral model”. In SIGGRAPH’87:

Proceedings of the 14th Annual Conference on Computer Graphics and Interactive

Techniques, ACM, New York, NY, USA, 1987.

[R99] Reynolds C., “Steering behaviors for autonomous characters”. In Game Developers

Conference, Miller Freeman Game Group, San Francisco, CA, USA, 1999.

[DEST04] De Chiara R. , Erra U., Tatafiore M., and Scarano V., “Massive simulation using GPU of a [DEST04] De Chiara R. , Erra U., Tatafiore M., and Scarano V., “Massive simulation using GPU of a

distributed behavioral model of a flock with obstacle avoidance ”. In Proceedings of Vision,

Modeling, and Visualization 2004 (VMV 2004) pp. 233-240.

[R06] Reynolds C., “ Big fast crowds on PS3”. In Sandbox’06: Proceedings of the 2006 ACM

SIGGRAPH Symposium on Videogames, ACM, New York, NY, USA, 2006.

[BDS09] Boccardo A., De Chiara R., and Scarano, V., “Massive Battle: Coordinated Movement of

Autonomous Agents”. In Proceedings of the Workshop on 3D Advanced Media In Gaming And

Simulation (3AMIGAS), 2009

[CCDES09] Cordasco G., Cosenza B., De Chiara R., Erra U., and Scarano, V., “Experiences with

Mesh-like computations using Prediction Binary Trees”. In Scalable Computing: Practice and

Experience, Scientific international journal for parallel and distributed computing (SCPE), Vol.

10, ISSN: 1895-1767, pages 173-187, June 2009.


Gennaro Cordasco , Rosario De Chiara Ugo Erra and … › ~cordasco › papers › ICUMT09 ›...

Documents

Transcript of Gennaro Cordasco , Rosario De Chiara Ugo Erra and … › ~cordasco › papers › ICUMT09 ›...