Drupal Mysql Webinar Percona

of 29 /29
Drupal and MySQL Mike Benshoof November 13, 2013

Embed Size (px)

Transcript of Drupal Mysql Webinar Percona

  • Drupal and MySQLMike Benshoof

    November 13, 2013

  • www.percona.com

    Agenda Overview Version Differences Database Configuration Scaling/HA Tuning/Performance

  • www.percona.com

    What is Drupal?

    Open Source CMS LAMP Stack Based

    Originally PHP/MySQL Now, includes any DB supported by PDO


  • www.percona.com

    Version Differences Limiting Discussion to MySQL Differences Per Drupal's Converting 6.x to 7.x Doc:

    Drupal 7 introduces a completely new database API Some of the biggest differences include:

    Dynamic Query Builders Formal Prepared Statements Specific INSERT/UPDATE/DELETE functions PDO is now REQUIRED (major change)

  • www.percona.com

    Sample Differences Standard SELECT note the bound variables (i.e. :uid,


    Update uses db_update with bound variables

  • www.percona.com

    Conceptual Differences Drupal 6

    database.inc & database.mysql.inc are more procedural Native MySQL interaction (mysql/mysqli) mysqli_connect, mysqli_query, etc

    Drupal 7 database.inc is object oriented and driver based PDO MySQL interaction with bound parameters and

    prepared statements Database::GetConnection()->prepare()->execute()

  • www.percona.com

    Database Configuration Database configuration is broken down into

    key/target pairs Key represents active database pool Target represents single database in pool

    Valid targets are default or slave Keys are user defined Full configuration located in

    sites/default/settings.php (or specific site)

  • www.percona.com

    Configuration (cont) Basic Configuration Single server, all traffic

    sent to this host Note, this is set up by default with the installer

  • www.percona.com

    Configuration (cont) Secondary Database Manually specify this

    connection in a module Can be MySQL or other PDO data source

  • www.percona.com

    Configuration (cont) Multiple Targets This is set up for master/slave Default/Slave are the only options NOTE: if using multiple slaves, one will be chosen at random

    for each request

  • www.percona.com

    Great... Now What??

    How do you utilize different databases from code?

  • www.percona.com

    Scaling the Database This is where the Default/Slave configuration

    comes into play Drupal may try to connect to a slave server when

    appropriate and if one is not available will simply fall back to the single master server

    Send reads to the slave (with read-only user) Send writes to the master (with write user) Stick delay-sensitive queries to master Support for this is limited in Drupal Core

  • www.percona.com

    In code, por favor? As parts of the Core don't utilize by default, it may be

    needed to manually send reads to the slave Simply add 'target' => 'slave' to the options Note this can impact replication delay-sensitive queries

  • www.percona.com

    Disabling Slave Servers In lag-sensitive environments, you can explicitly

    disable the use of slave servers: db_ignore_slave()

    You can also force to the master for specific by using the same logic as slave: array('target' => 'default')

    $_SESSION['ignore_slave_server'] Skips slave for a set number of seconds Good for write followed by read(s)

  • www.percona.com

    And External Connections? This is where the distinct

    database key comes into play

    Change the active database You can also utilize

    default/slave databases here as well

  • www.percona.com

    Availability Splitting reads/writes can be complicated and in

    some cases, require manual code changes Fortunately, HA is simple (from application

    standpoint) Use base configuration (default, default) Default db server is actually a VIP Failover is hidden from the application Also, manual failover via the application is an option

  • www.percona.com

    Availability Different options

    Simple master/slave with manual failover (reconfigure application config)

    Master/Slave and shift IP on database (MHA) Master/Master behind VIP (HAProxy, BigIP, etc) PXC behind a VIP PRM to manage failover and VIP

    In most cases, the application will just shift traffic and not need to be reconfigured

  • www.percona.com

    Default Install Tuning

    Use Memcache!! Semaphore/Locking API Profile early and often!

  • www.percona.com

    Cache != Faster By default, cache is stored in MySQL This actually DOUBLES the number of

    database queries in normal page browsing This assumes populated cache

    Cache misses will actually increase the MySQL traffic even more, adding WRITE traffic Even with no content being created, you have major

    write impact on the database

  • www.percona.com

    Benchmarking Simple JMeter Test Plan

    Browse home page nodes Randomly select a few nodes/pages Data populated using the development module 10 iterations of proxy-captured test plan

  • www.percona.com

    Default Install Profiling

    # 1.3s user time, 10ms system time, 24.32M rss, 200.27M vsz# Current date: Mon Sep 30 19:46:41 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 3.50k total, 29 unique, 17.84 QPS, 0.06x concurrency __________# Time range: 2013-09-30 19:39:54 to 19:43:10

    # Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xCE1FFB25CB1ACF8E 0.5009 3.9% 530 0.0009 0.04 SELECT cache_bootstrap# 2 0xC854EADB9190C72F 1.8323 14.4% 470 0.0039 0.43 SELECT cache_menu# 3 0xE9D33E090967923F 4.7602 37.5% 403 0.0118 1.11 SELECT cache_field# 4 0x09630F7EE44B0750 0.3089 2.4% 342 0.0009 0.07 SELECT cache# 5 0xAD1BA3CA5E2D6F3B 0.1682 1.3% 139 0.0012 0.03 SELECT url_alias

    Full query capture (long_query_time=0) Running in VM, so sorted by count pt-query-digest --limit=30 /var/lib/mysql/slow.log order-by=Query_time:cnt Note: This assumes a populated cache

  • www.percona.com

    Default Install - Continued No indexing issues in top queries

    Primary key or CONST index hits Top 4 queries are all SELECT cache_%

    48% of all queries (1745 out of 3.5k) 90% of bytes sent to the client 22% of rows examined

    When considering query optimization, the best query is one that is never run...

  • www.percona.com

    Enter memcached...# 800ms user time, 100ms system time, 22.31M rss, 199.78M vsz# Current date: Mon Sep 30 20:14:26 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 1.74k total, 25 unique, 16.59 QPS, 0.05x concurrency __________# Time range: 2013-09-30 20:12:08 to 20:13:53

    # Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xAD1BA3CA5E2D6F3B 0.0304 0.6% 139 0.0002 0.00 SELECT url_alias# 2 0xB83088CAE4EA7A29 0.1669 3.2% 90 0.0019 0.06 SELECT url_alias# 3 0xBAD26E72FBCD69FE 0.3140 6.0% 90 0.0035 0.03 SELECT blocked_ips# 4 0xCC47B42511EA22DD 0.2465 4.7% 90 0.0027 0.21 SET# 5 0x09E2CE64B0B5C525 0.0758 1.4% 89 0.0009 0.03 SET

    Note the huge drop in total queries

    3.5k 1.74k Also note the 4 query difference in unique

    29 unique 25 unique Only queries missing are the cache_% tables

  • www.percona.com

    Locking API Important for preventing

    cache stampedes Wrap expensive calls with

    locking (i.e. variable_initialize) Default uses semaphore

    table in MySQL (InnoDB) Write overhead to database

    (flushing, redo log, etc) Does it need to be durable?

  • www.percona.com

    Locking API (cont) Sample DB calls for a single page load:

    [[email protected] drupal]# tail -f /var/lib/mysql/centos6-webdev-slow.log | grep -i semaphore

    INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ... INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...

    8 write queries for single page load (redo log, page flushing, etc) Depending on InnoDB configuration (innodb_flush_log_at_trx_commit, etc),

    this can lead to several IOPs (expensive)

  • www.percona.com

    Locking Alternatives Memory engine for semaphore table

    Removes InnoDB overhead No disk access needed table in RAM Native implementation (no plugin or modules) Non-durable Negative table-level locking can reduce concurrency, but this

    table is typically small (and hopefully not used often) Memcache locking

    Since you are already using memcache... (right?) Uses the (global) Memcache::Add operation Non-durable as well

  • www.percona.com

    Profiling Early and Often Finding problems early is key

    Missing indexes, etc long_query_time = 0

    Full verbosity (if using Percona Server) Pt-query-digest

    Rows_examined > Rows_sent (suboptimal index) Execution time --review track new queries over time (new

    modules, etc)

  • www.percona.com


  • [email protected]

    Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29