AN OPEN-SOURCE BENCHMARK SUITE FOR MICROSERVICES AND THEIR HARDWARE-SOFTWARE IMPLICATIONS FOR CLOUD AND

EDGE SYSTEMS Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayantara Katarki,

Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Yuan He, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon

Zaruvinsky, Mateo Espinosa, Rick Lin, Zhongling Liu, Jake Padilla and Christina Delimitrou

Cornell University

ASPLOS 2019

Session Cloud I

EXECUTIVE SUMMARY  Cloud applications migrating from monoliths to microservices

• Monoliths: all functionality in a single service

• Microservices: many single-concerned, loosely-coupled services

• Modularity, specialization, faster development

• Datacenters designed for monoliths  microservices have different requirements

 An end-to-end benchmark suite for large-scale microservices

 Architectural and system implications • Hardware design

• OS/networking overheads

• Cluster management

• Application & programming frameworks

• Tail at scale

1

FROM MONOLITHS TO MICROSERVICES  Monolithic applications

• Single binary with entire business logic

 Limitations • Too complex for continuous development

• Obstacle to adopting new frameworks

• Poor scalability & elasticity

2

Monolith Application

login

orders

payments

shipping

FROM MONOLITHS TO MICROSERVICES  Microservices

• Fine-grained, loosely-coupled, and single- concerned

• Communicate with RPCs or RESTful APIs

 Pros • Agile development

• Better modularity & elasticity

• Testing and debugging in isolation

 Cons • Different hardware & software constraints

• Dependencies  complicate cluster management

3

login orders payments

shipping

FROM MONOLITHS TO MICROSERVICES

4

MOTIVATION  Explore implications of microservices across the system stack

5

1. Hardware design

2. OS/Network overheads

3. Cluster management

4.

1. Hardware design

2. OS/Network overheads

3. Cluster management

4. Application and frameworks

5. Tail at scale

MOTIVATION  Explore implications of microservices across the system stack

6

1. Hardware design

2. OS/Network overheads

3. Cluster management

4.

1. Hardware design

2. OS/Network overheads

3. Cluster management

4. Application and frameworks

5. Tail at scale

Need representative, end-to-end applications built with microservices

MOTIVATION  Previous work in cloud benchmarking

• CloudSuite [ASPLOS’12]

• Sirius [ASPLOS’15]

• TailBench [IISWC’17]

• μSuite [IISWC’18]

 DeathStarBench suite • Focus on large-scale microservices that stress typical datacenter design

7

Focus either on monolithic applications or applications with few tiers

DEATHSTARBENCH SUITE  Design principles

• Representativeness » Use of popular open-source applications and frameworks

» Service architecture following public documentation of real systems using microservices

8

DEATHSTARBENCH SUITE  Design principles

• Representativeness

• End-to-end operation » Full functionality using microservices

9

DEATHSTARBENCH SUITE  Design principles

• Representativeness

• End-to-end operation

• Heterogeneity » Wide range of programming languages and microservices frameworks

10

DEATHSTARBENCH SUITE  Design principles

• Representativeness

• End-to-end operation

• Heterogeneity

• Modularity » Single-concerned and loosely-coupled services

11

DEATHSTARBENCH SUITE  Design principles

• Representativeness

• End-to-end operation

• Heterogeneity

• Modularity

• Reconfigurability » Easy to update or change components with minimal effort

12

DEATHSTARBENCH SUITE  5 end-to-end applications, tens of unique microservices each

• Social Network

• Media Service

• E-Commerce Service

• Banking System

• Drone Coordination System

13

DEATHSTARBENCH SUITE  Social network

14

DEATHSTARBENCH SUITE  Media service

15

DEATHSTARBENCH SUITE  E-commerce service

16

DEATHSTARBENCH SUITE  Banking system

17

DEATHSTARBENCH SUITE  Drone coordination system

18

Client Load

Balancer NGINX Controller

Construct

Route

TargetDB

ImageDB

VideoDB

LocationDB

SpeedDB

LuminosityDB

OrientationDB

Image

Video

Location

Speed

Luminosity

Orientation

Stocking

ImageDB

MotionCtrl

Image

Recognition

Obstacle

Avoidance

Controller

Log(node.js)

Edge Router

Edge Swarm

Frontend EdgeCloud

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Read

Post

User

Timeline

Write Home

Timeline

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

Storage Read Home

Timeline

Social

Graph

Load

Balancer NGINX

Video Store

Frontend

Image Store

Frontend

Client

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Frontend Logic Caching & Storage

Index1 IndexnIndex0

CASE STUDY: SOCIAL NETWORK  User sign up/login

19

CASE STUDY: SOCIAL NETWORK  Write posts

20

Client Load

Balancer NGINX

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

Storage Read Home

Timeline

Social

Graph

Memcached MongoDB

Redis

Memcached MongoDB

Read

Post Memcached MongoDB

Memcached MongoDB User

Timeline

RabbitMQ

Write Home

Timeline

Frontend Logic Caching & Storage

Memcached MongoDB

Memcached MongoDB

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Video Store

Frontend

Image Store

Frontend Index1 IndexnIndex0

CASE STUDY: SOCIAL NETWORK  Read home timeline

21

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Read

Post

User

Timeline

Write Home

Timeline

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

Storage Read Home

Timeline

Social

Graph

Load

Balancer NGINX

Video Store

Frontend

Image Store

Frontend

Client

Video storage

Image storage

Social graph

storage

Home timeline

storage

User timeline

storage

Post storage

User storage

Frontend Logic Caching & Storage

Index1 IndexnIndex0

CASE STUDY: SOCIAL NETWORK  Search

22

Index1 IndexnIndex0

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

MongoDB

Memcached

Redis

Memcached

Memcached

Memcached

Memcached

Memcached

RabbitMQ

Unique ID

URL Shorten

Image

Video

Text

User Tag

Favorite

Search

Recommender

User

Compose

Post

Post

Storage Read Home

Timeline

Social

Graph

Read

Post

User

Timeline

Write Home

Timeline

Load

Balancer NGINXClient

Frontend Logic Caching & Storage

Video storage

Image storage

Social graph

storage

H