ING Groep N.V.: ExploitING Hotel Cassandra

44
Exploiting Hotel Cassandra cqlsh> SELECT * FROM presentations WHERE eventname='CassandraSummit' AND company='ING' AND year=2015; eventname | company | year | area | name | presentation | title | twitter -----------------+---------+------+------+---------------------+-----------------------------+-----------------------+----------- CassandraSummit | ING | 2015 | NL | Christopher Reedijk | ExploitING Hotel Cassandra | Engineer/Chapter Lead | @creedijk CassandraSummit | ING | 2015 | NL | Gary Stewart | ExploitING Hotel Cassandra | Engineer/Architect | @Gaz_GandA (2 rows)

Transcript of ING Groep N.V.: ExploitING Hotel Cassandra

Page 1: ING Groep N.V.: ExploitING Hotel Cassandra

Exploiting Hotel Cassandra

cqlsh> SELECT * FROM presentations WHERE eventname='CassandraSummit' AND company='ING' AND year=2015;

eventname | company | year | area | name | presentation | title | twitter -----------------+---------+------+------+---------------------+-----------------------------+-----------------------+-----------CassandraSummit | ING | 2015 | NL | Christopher Reedijk | ExploitING Hotel Cassandra | Engineer/Chapter Lead | @creedijkCassandraSummit | ING | 2015 | NL | Gary Stewart | ExploitING Hotel Cassandra | Engineer/Architect | @Gaz_GandA

(2 rows)

Page 2: ING Groep N.V.: ExploitING Hotel Cassandra

cqlsh> SELECT * FROM presentations WHERE eventname='CassandraSummit' AND company='ING';

eventname | company | year | area | name | presentation | title | twitter -----------------+---------+------+------+---------------------+-----------------------------+-----------------------+-----------CassandraSummit | ING | 2015 | NL | Christopher Reedijk | ExploitING Hotel Cassandra | Engineer/Chapter Lead | @creedijkCassandraSummit | ING | 2015 | NL | Gary Stewart | ExploitING Hotel Cassandra | Engineer/Architect | @Gaz_GandACassandraSummit | ING | 2014 | NL | Christopher Reedijk | C*@ING: Testing the waters | Dev Engineer | @creedijkCassandraSummit | ING | 2014 | NL | Gary Stewart | C*@ING: Testing the waters | Dev Engineer | @Gaz_GandA

(4 rows)

Exploiting Hotel Cassandra

Page 3: ING Groep N.V.: ExploitING Hotel Cassandra

Market leaders Benelux

Growth markets

Commercial Banking

Challengers

3

World mapOver 40 countries52,000+ employees

Page 4: ING Groep N.V.: ExploitING Hotel Cassandra

Market leaders Benelux

Growth markets

Commercial Banking

Challengers

4

European map

Full-service bankVery strong European baseRanked 7thLargest bank of NL150+ DevOps teams

Page 5: ING Groep N.V.: ExploitING Hotel Cassandra

• Engineer / Architect• Love/hate relationship with C*• 15 years of experience in IT• @ING since: 01-01-2012

• Engineer / Chapter Lead• Back-end guy• 11 years of experience in IT• @ING since: 01-01-2011

5

a little bit about us

Gary Stewart Christopher Reedijk

#CassandraSummit

Page 6: ING Groep N.V.: ExploitING Hotel Cassandra

6

on a dark desert highway #CassandraSummit

Page 7: ING Groep N.V.: ExploitING Hotel Cassandra

There was a need for availability

There was a need for consistency

There was a need for scalability

7

on a dark desert highway #CassandraSummit

Page 8: ING Groep N.V.: ExploitING Hotel Cassandra

• Lots of small use cases

• Lots of DevOps teams

• No ephemeral (dedicated) storage

• Zero to none NoSQL experience

my head grew heavy and my sight grew dim

8

#CassandraSummit

Page 9: ING Groep N.V.: ExploitING Hotel Cassandra

this could be Heaven or this could be Hell

9

#CassandraSummit

Page 10: ING Groep N.V.: ExploitING Hotel Cassandra

• Cassandra seemed to be the solution

• How to work with this paradigm shift

• Are we ready for it?

this could be Heaven or this could be Hell

10

#CassandraSummit

Page 11: ING Groep N.V.: ExploitING Hotel Cassandra

then she lit up a candle and she showed me the way

11

#CassandraSummit

Page 12: ING Groep N.V.: ExploitING Hotel Cassandra

APIs and Cassandra are a great fite.g. http://host/customer/id

then she lit up a candle and she showed me the way

12

#CassandraSummit

Page 13: ING Groep N.V.: ExploitING Hotel Cassandra

welcome to the Hotel Cassandra

13

#CassandraSummit

Page 14: ING Groep N.V.: ExploitING Hotel Cassandra

• Combine the small use cases

• Keeping costs in mind

• Do we dare to share?

welcome to the Hotel Cassandra

14

#CassandraSummit

Page 15: ING Groep N.V.: ExploitING Hotel Cassandra

such a lovely place..., such a lovely face

15

#CassandraSummit

Luxury

Hotel

1100+ colleagues trained4 certified trainers35+ DevOps teams12 clusters (4/5 environments)

Containment

Reduce paperwork

Unexpected load

Infrastructure

FunctionalityMonitoringAdvice

Life cyclemanagement

Security

Velocity

Training

Page 16: ING Groep N.V.: ExploitING Hotel Cassandra

16

plenty of room at the hotel cassandra #CassandraSummit

Page 17: ING Groep N.V.: ExploitING Hotel Cassandra

17

sandbox feedback containment

plenty of room at the hotel cassandra #CassandraSummit

Page 18: ING Groep N.V.: ExploitING Hotel Cassandra

some dance to remember, some dance to forget

18

#CassandraSummit

Page 19: ING Groep N.V.: ExploitING Hotel Cassandra

• Can our hardware give us containment?YES, but very expensive

• Can Cassandra offer containment?NO, not on keyspace level

• Can other technologies give us containment?PARTIALLY, but involves new technologies

• Can our software give us containment?YES, but needs ownership

some dance to remember, some dance to forget

19

#CassandraSummit

Page 20: ING Groep N.V.: ExploitING Hotel Cassandra

so I called up the Captain

20

#CassandraSummit

Page 21: ING Groep N.V.: ExploitING Hotel Cassandra

so I called up the Captain

21

#CassandraSummit

Page 22: ING Groep N.V.: ExploitING Hotel Cassandra

so I called up the Captain

22

#CassandraSummit

Page 23: ING Groep N.V.: ExploitING Hotel Cassandra

please bring me my wine

23

#CassandraSummit

Page 24: ING Groep N.V.: ExploitING Hotel Cassandra

please bring me my wine

24

#CassandraSummit

Page 25: ING Groep N.V.: ExploitING Hotel Cassandra

please bring me my wine

25

#CassandraSummit

Page 26: ING Groep N.V.: ExploitING Hotel Cassandra

please bring me my wine

26

#CassandraSummit

Page 27: ING Groep N.V.: ExploitING Hotel Cassandra

please bring me my wine

27

#CassandraSummit

Page 28: ING Groep N.V.: ExploitING Hotel Cassandra

they livin' it up at the hotel cassandra

28

#CassandraSummit

Page 29: ING Groep N.V.: ExploitING Hotel Cassandra

they livin' it up at the hotel cassandra

29

#CassandraSummit

Page 30: ING Groep N.V.: ExploitING Hotel Cassandra

what a nice surprise..., bring your alibis

30

#CassandraSummit

Page 31: ING Groep N.V.: ExploitING Hotel Cassandra

what a nice surprise..., bring your alibis

31

#CassandraSummit

Page 32: ING Groep N.V.: ExploitING Hotel Cassandra

• SSD's for SOLR/Spark• 3.6TB Nodes

• 3 Data Centers• Spinning disks• 3.6TB Nodes

• Spinning disks• 10-15TB per node• Date Tiered Compaction

32

mirrors on the ceiling..., the pink champagne on ice #CassandraSummit

Page 33: ING Groep N.V.: ExploitING Hotel Cassandra

we are all just prisoners here, of our own device

33

#CassandraSummit

Page 34: ING Groep N.V.: ExploitING Hotel Cassandra

we are all just prisoners here, of our own device

34

#CassandraSummit

Page 35: ING Groep N.V.: ExploitING Hotel Cassandra

they stab it with their steely knives

35

#CassandraSummit

Page 36: ING Groep N.V.: ExploitING Hotel Cassandra

they stab it with their steely knives

36

#CassandraSummit

Page 37: ING Groep N.V.: ExploitING Hotel Cassandra

they stab it with their steely knives

37

#CassandraSummit

Page 38: ING Groep N.V.: ExploitING Hotel Cassandra

but they just can't kill the beast

38

#CassandraSummit

Page 39: ING Groep N.V.: ExploitING Hotel Cassandra

• Tracking of queries in order to give more insight in common best practices and actual use case pattern

• Cross-environment datamodel check

• Capacity management with prediction

• ...

but they just can't kill the beast

39

#CassandraSummit

Page 40: ING Groep N.V.: ExploitING Hotel Cassandra

40

you can check-out any time you like..., but you can never leave! #CassandraSummit

Page 41: ING Groep N.V.: ExploitING Hotel Cassandra

There is the need for availability

There is the need for consistency

There is the need for scalability

41

you can check-out any time you like..., but you can never leave! #CassandraSummit

Page 42: ING Groep N.V.: ExploitING Hotel Cassandra

Thank you

[email protected]@creedijk

[email protected]@Gaz_GandA

visuals by Isabelle Hörl:www.isabellehorl.com | @IsabelleHorl

we are hiring!

Page 43: ING Groep N.V.: ExploitING Hotel Cassandra

Follow us to stay a step ahead

ING.com

YouTube.com/ING

SlideShare.net/ING@ING_News LinkedIn.com/company/ING

Flickr.com/INGGroupFacebook.com/ING

Page 44: ING Groep N.V.: ExploitING Hotel Cassandra

ING Group’s Annual Accounts are prepared in accordance with International Financial Reporting Standards as adopted by the European Union (‘IFRS-EU’).

In preparing the financial information in this document, the same accounting principles are applied as in the 2014 ING Group Annual Accounts. All figures in this document are unaudited. Small differences are possible in the tables due to rounding.

Certain of the statements contained herein are not historical facts, including, without limitation, certain statements made of future expectations and other forward-looking statements that are based on management’s current views and assumptions and involve known and unknown risks and uncertainties that could cause actual results, performance or events to differ materially from those expressed or implied in such statements. Actual results, performance or events may differ materially from those in such statements due to, without limitation: (1) changes in general economic conditions, in particular economic conditions in ING’s core markets, (2) changes in performance of financial markets, including developing markets, (3) consequences of a potential (partial) break-up of the euro, (4) the implementation of ING’s restructuring plan to separate banking and insurance operations, (5) changes in the availability of, and costs associated with, sources of liquidity such as interbank funding, as well as conditions in the credit markets generally, including changes in borrower and counterparty creditworthiness, (6) the frequency and severity of insured loss events, (7) changes affecting mortality and

morbidity levels and trends,(8) changes affecting persistency levels, (9) changes affecting interest rate levels, (10) changes affecting currency exchange rates, (11) changes in investor, customer and policyholder behaviour, (12) changes in general competitive factors, (13) changes in laws and regulations, (14) changes in the policies of governments and/or regulatory authorities, (15) conclusions with regard to purchase accounting assumptions and methodologies, (16) changes in ownership that could affect the future availability to us of net operating loss, net capital and built-in loss carry forwards, (17) changes in credit ratings, (18) ING’s ability to achieve projected operational synergies and (19) the other risks and uncertainties detailed in the Risk Factors section contained in the most recent annual report of ING Groep N.V. Any forward-looking statements made by or on behalf of ING speak only as of the date they are made, and, ING assumes no obligation to publicly update or revise any forward-looking statements, whether as a result of new information or for any other reason.

This document does not constitute an offer to sell, or a solicitation of an offer to purchase, any securities in the United States or any other jurisdiction. The securities of NN Group have not been and will not be registered under the U.S. Securities Act of 1933, as amended (the “Securities Act”), and may not be offered or sold within the United States absent registration or an applicable exemption from the registration requirements of the Securities Act.

www.ing.com

Disclaimer

44