Download - AN OPEN-SOURCE BENCHMARK S MICROSERVICES AND THEIR … › ~delimitrou › slides › ... · AN OPEN-SOURCE BENCHMARK SUITE FOR MICROSERVICES AND THEIR HARDWARE-SOFTWARE IMPLICATIONS

AN OPEN-SOURCE BENCHMARK SUITE FOR MICROSERVICES ANDTHEIR HARDWARE-SOFTWARE IMPLICATIONS FOR CLOUD AND

EDGE SYSTEMS

Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayantara Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Yuan He, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon

Zaruvinsky, Mateo Espinosa, Rick Lin, Zhongling Liu, Jake Padilla and Christina Delimitrou

Cornell University

ASPLOS 2019

Session Cloud I

EXECUTIVE SUMMARY

Cloud applications migrating from monoliths to microservices• Monoliths: all functionality in a single service

• Microservices: many single-concerned, loosely-coupled services

• Modularity, specialization, faster development

• Datacenters designed for monoliths microservices have different requirements

An end-to-end benchmark suite for large-scale microservices

Architectural and system implications• Hardware design

• OS/networking overheads

• Cluster management

• Application & programming frameworks

• Tail at scale

1

FROM MONOLITHS TO MICROSERVICES

Monolithic applications• Single binary with entire business logic

Limitations• Too complex for continuous development

• Obstacle to adopting new frameworks

• Poor scalability & elasticity

2

Monolith Application

login

orders

payments

shipping


Microservices• Fine-grained, loosely-coupled, and single-

concerned

• Communicate with RPCs or RESTful APIs

Pros• Agile development

• Better modularity & elasticity

• Testing and debugging in isolation

Cons• Different hardware & software constraints

• Dependencies complicate cluster management

3

loginorders payments

shipping


4

MOTIVATION

Explore implications of microservices across the system stack

5

1. Hardware design

2. OS/Network overheads

3. Cluster management

4.

1. Hardware design



4. Application and frameworks

5. Tail at scale

MOTIVATION


6

1. Hardware design



4.

1. Hardware design




5. Tail at scale

Need representative, end-to-end applicationsbuilt with microservices

MOTIVATION

Previous work in cloud benchmarking• CloudSuite [ASPLOS’12]

• Sirius [ASPLOS’15]

• TailBench [IISWC’17]

• μSuite [IISWC’18]

DeathStarBench suite• Focus on large-scale microservices that stress typical datacenter design

7

Focus either on monolithic applications or applications with few tiers

DEATHSTARBENCH SUITE

Design principles• Representativeness

» Use of popular open-source applications and frameworks

» Service architecture following public documentation of real systems using microservices

8



• End-to-end operation» Full functionality using microservices

9



• End-to-end operation

• Heterogeneity» Wide range of programming languages and microservices frameworks

10




• Heterogeneity

• Modularity» Single-concerned and loosely-coupled services

11




• Heterogeneity

• Modularity

• Reconfigurability» Easy to update or change components with minimal effort

12


5 end-to-end applications, tens of unique microservices each• Social Network

• Media Service

• E-Commerce Service

• Banking System

• Drone Coordination System

13


Social network

14


Media service

15


E-commerce service

16


Banking system

17


Drone coordination system

18

ClientLoad

BalancerNGINX Controller

Construct

Route

TargetDB

ImageDB

VideoDB

LocationDB

SpeedDB

LuminosityDB

OrientationDB

Image

Video

Location

Speed

Luminosity

Orientation

Stocking

ImageDB

MotionCtrl

Image

Recognition

Obstacle

Avoidance

Controller

Log(node.js)

Edge Router

Edge Swarm

Frontend EdgeCloud

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Read

Post

User

Timeline

Write Home

Timeline

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

StorageRead Home

Timeline

Social

Graph

Load

BalancerNGINX

Video Store

Frontend

Image Store

Frontend

Client

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Frontend Logic Caching & Storage

Index1IndexnIndex0

CASE STUDY: SOCIAL NETWORK

User sign up/login

19


Write posts

20

ClientLoad

BalancerNGINX

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

StorageRead Home

Timeline

Social

Graph

Memcached MongoDB

Redis

Memcached MongoDB

Read

PostMemcached MongoDB

Memcached MongoDBUser

Timeline

RabbitMQ

Write Home

Timeline


Memcached MongoDB

Memcached MongoDB

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Video Store

Frontend

Image Store

Frontend Index1IndexnIndex0


Read home timeline

21

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Read

Post

User

Timeline

Write Home

Timeline

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

StorageRead Home

Timeline

Social

Graph

Load

BalancerNGINX

Video Store

Frontend

Image Store

Frontend

Client

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage


Index1IndexnIndex0


Search

22

Index1IndexnIndex0

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

StorageRead Home

Timeline

Social

Graph

Read

Post

User

Timeline

Write Home

Timeline

Load

BalancerNGINXClient


Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Video Store

Frontend

Image Store

Frontend


Recommendation

23

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

StorageRead Home

Timeline

Social

Graph

Read

Post

User

Timeline

Write Home

Timeline

Load

BalancerNGINXClient


Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Video Store

Frontend

Image Store

Frontend Index1IndexnIndex0

ARCHITECTURAL AND SYSTEM IMPLICATIONS


24

1. Hardware design



4.

1. Hardware design




5. Tail at scale

HARDWARE DESIGN

Brawny vs. wimpy cores• Microservices are more sensitive to performance

unpredictability than monoliths

25

Microservices Monoliths

1. Hardware design




5. Tail at scale

HARDWARE DESIGN


unpredictability than monoliths

26

Microservices

Monoliths

1. Hardware design




5. Tail at scale

HARDWARE DESIGN


unpredictability than monolithic apps

• Xeon vs Cavium servers

Cycle breakdown of each microservice• Smaller fraction of frontend stalls than monoliths

I-cache pressure• Lower I-cache pressure than monoliths

27

1. Hardware design




5. Tail at scale

OS/NETWORK OVERHEADS

RPC overheads• A large fraction of time spent in network stack

FPGA network acceleration• Offload TCP stack on FPGA

• 10 − 68x improvement on network processing latency

• 43% - 2.2x improvement on end-to-end latency

28

1. Hardware design




5. Tail at scale

MemcachedNGINX

CLUSTER MANAGEMENT

Latency back-pressure• Bottleneck services pressure upstreaming services

• Cause: Imperfect pipelining» HTTP/TCP HoL blocking

» Limited number of worker threads/connections

29

NGINX bottleneck NGINX Memcached

🔥

NGINX Memcached

🔥Memcached bottleneck

Example: HTTP 1.1 HoL blocking

1. Hardware design




5. Tail at scale

CLUSTER MANAGEMENT

Cascading QoS violations• Hotspots propagating along the dependency graph

• No obvious correlation to CPU utilization

• Difficulty in discovering the bottleneck and long time to recover from QoS violations

30

1. Hardware design




5. Tail at scale

APPLICATION AND FRAMEWORKS

Serverless frameworks• Compared long-running microservices on EC2 with short–

running microservices on AWS Lambda

• Agile resource adjustments with diurnal load pattern

• Higher performance variability due to

» No control of lambda placement

» Communication through S3

» Loading of dependencies

31

1. Hardware design




5. Tail at scale

TAIL AT SCALE

Impact of slow servers• Larger cluster larger impact of slow servers

• More severe tail latency increase compared to monoliths

32

1. Hardware design



1. Hardware design




5. Tail at Scale

CONCLUSIONS

• Cloud applications from monoliths to microservices

• Study implications of microservices across the system stack

• Open-source benchmark suite for cloud and IoT microservices

• Explored the implications of microservices• More sensitive to performance unpredictability

• Potential of hardware acceleration for networking

• Need for cluster managers that account for dependencies

• Tail at scale effects more prominent in microservices

33

QUESTIONS?• Cloud applications from monoliths to microservices

• Study implications of microservices across the system stack

• Open-source benchmark suite for cloud and IoT microservices

• Explored the implications of microservices• More sensitive to performance unpredictability

• Potential of hardware acceleration for networking

• Need for cluster managers that account for dependencies

• Tail at scale effects more prominent in microservices

34