Drupal Mysql Webinar Percona
of 29
/29
-
Author
namtien-us -
Category
Documents
-
view
222 -
download
4
Embed Size (px)
Transcript of Drupal Mysql Webinar Percona
-
Drupal and MySQLMike Benshoof
November 13, 2013
-
www.percona.com
Agenda Overview Version Differences Database Configuration Scaling/HA Tuning/Performance
-
www.percona.com
What is Drupal?
Open Source CMS LAMP Stack Based
Originally PHP/MySQL Now, includes any DB supported by PDO
http://www.drupal.org
-
www.percona.com
Version Differences Limiting Discussion to MySQL Differences Per Drupal's Converting 6.x to 7.x Doc:
Drupal 7 introduces a completely new database API Some of the biggest differences include:
Dynamic Query Builders Formal Prepared Statements Specific INSERT/UPDATE/DELETE functions PDO is now REQUIRED (major change)
-
www.percona.com
Sample Differences Standard SELECT note the bound variables (i.e. :uid,
:type)
Update uses db_update with bound variables
-
www.percona.com
Conceptual Differences Drupal 6
database.inc & database.mysql.inc are more procedural Native MySQL interaction (mysql/mysqli) mysqli_connect, mysqli_query, etc
Drupal 7 database.inc is object oriented and driver based PDO MySQL interaction with bound parameters and
prepared statements Database::GetConnection()->prepare()->execute()
-
www.percona.com
Database Configuration Database configuration is broken down into
key/target pairs Key represents active database pool Target represents single database in pool
Valid targets are default or slave Keys are user defined Full configuration located in
sites/default/settings.php (or specific site)
-
www.percona.com
Configuration (cont) Basic Configuration Single server, all traffic
sent to this host Note, this is set up by default with the installer
-
www.percona.com
Configuration (cont) Secondary Database Manually specify this
connection in a module Can be MySQL or other PDO data source
-
www.percona.com
Configuration (cont) Multiple Targets This is set up for master/slave Default/Slave are the only options NOTE: if using multiple slaves, one will be chosen at random
for each request
-
www.percona.com
Great... Now What??
How do you utilize different databases from code?
-
www.percona.com
Scaling the Database This is where the Default/Slave configuration
comes into play Drupal may try to connect to a slave server when
appropriate and if one is not available will simply fall back to the single master server
Send reads to the slave (with read-only user) Send writes to the master (with write user) Stick delay-sensitive queries to master Support for this is limited in Drupal Core
-
www.percona.com
In code, por favor? As parts of the Core don't utilize by default, it may be
needed to manually send reads to the slave Simply add 'target' => 'slave' to the options Note this can impact replication delay-sensitive queries
-
www.percona.com
Disabling Slave Servers In lag-sensitive environments, you can explicitly
disable the use of slave servers: db_ignore_slave()
You can also force to the master for specific by using the same logic as slave: array('target' => 'default')
$_SESSION['ignore_slave_server'] Skips slave for a set number of seconds Good for write followed by read(s)
-
www.percona.com
And External Connections? This is where the distinct
database key comes into play
Change the active database You can also utilize
default/slave databases here as well
-
www.percona.com
Availability Splitting reads/writes can be complicated and in
some cases, require manual code changes Fortunately, HA is simple (from application
standpoint) Use base configuration (default, default) Default db server is actually a VIP Failover is hidden from the application Also, manual failover via the application is an option
-
www.percona.com
Availability Different options
Simple master/slave with manual failover (reconfigure application config)
Master/Slave and shift IP on database (MHA) Master/Master behind VIP (HAProxy, BigIP, etc) PXC behind a VIP PRM to manage failover and VIP
In most cases, the application will just shift traffic and not need to be reconfigured
-
www.percona.com
Default Install Tuning
Use Memcache!! Semaphore/Locking API Profile early and often!
-
www.percona.com
Cache != Faster By default, cache is stored in MySQL This actually DOUBLES the number of
database queries in normal page browsing This assumes populated cache
Cache misses will actually increase the MySQL traffic even more, adding WRITE traffic Even with no content being created, you have major
write impact on the database
-
www.percona.com
Benchmarking Simple JMeter Test Plan
Browse home page nodes Randomly select a few nodes/pages Data populated using the development module 10 iterations of proxy-captured test plan
-
www.percona.com
Default Install Profiling
# 1.3s user time, 10ms system time, 24.32M rss, 200.27M vsz# Current date: Mon Sep 30 19:46:41 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 3.50k total, 29 unique, 17.84 QPS, 0.06x concurrency __________# Time range: 2013-09-30 19:39:54 to 19:43:10
# Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xCE1FFB25CB1ACF8E 0.5009 3.9% 530 0.0009 0.04 SELECT cache_bootstrap# 2 0xC854EADB9190C72F 1.8323 14.4% 470 0.0039 0.43 SELECT cache_menu# 3 0xE9D33E090967923F 4.7602 37.5% 403 0.0118 1.11 SELECT cache_field# 4 0x09630F7EE44B0750 0.3089 2.4% 342 0.0009 0.07 SELECT cache# 5 0xAD1BA3CA5E2D6F3B 0.1682 1.3% 139 0.0012 0.03 SELECT url_alias
Full query capture (long_query_time=0) Running in VM, so sorted by count pt-query-digest --limit=30 /var/lib/mysql/slow.log order-by=Query_time:cnt Note: This assumes a populated cache
-
www.percona.com
Default Install - Continued No indexing issues in top queries
Primary key or CONST index hits Top 4 queries are all SELECT cache_%
48% of all queries (1745 out of 3.5k) 90% of bytes sent to the client 22% of rows examined
When considering query optimization, the best query is one that is never run...
-
www.percona.com
Enter memcached...# 800ms user time, 100ms system time, 22.31M rss, 199.78M vsz# Current date: Mon Sep 30 20:14:26 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 1.74k total, 25 unique, 16.59 QPS, 0.05x concurrency __________# Time range: 2013-09-30 20:12:08 to 20:13:53
# Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xAD1BA3CA5E2D6F3B 0.0304 0.6% 139 0.0002 0.00 SELECT url_alias# 2 0xB83088CAE4EA7A29 0.1669 3.2% 90 0.0019 0.06 SELECT url_alias# 3 0xBAD26E72FBCD69FE 0.3140 6.0% 90 0.0035 0.03 SELECT blocked_ips# 4 0xCC47B42511EA22DD 0.2465 4.7% 90 0.0027 0.21 SET# 5 0x09E2CE64B0B5C525 0.0758 1.4% 89 0.0009 0.03 SET
Note the huge drop in total queries
3.5k 1.74k Also note the 4 query difference in unique
29 unique 25 unique Only queries missing are the cache_% tables
-
www.percona.com
Locking API Important for preventing
cache stampedes Wrap expensive calls with
locking (i.e. variable_initialize) Default uses semaphore
table in MySQL (InnoDB) Write overhead to database
(flushing, redo log, etc) Does it need to be durable?
-
www.percona.com
Locking API (cont) Sample DB calls for a single page load:
[[email protected] drupal]# tail -f /var/lib/mysql/centos6-webdev-slow.log | grep -i semaphore
INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ... INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...
8 write queries for single page load (redo log, page flushing, etc) Depending on InnoDB configuration (innodb_flush_log_at_trx_commit, etc),
this can lead to several IOPs (expensive)
-
www.percona.com
Locking Alternatives Memory engine for semaphore table
Removes InnoDB overhead No disk access needed table in RAM Native implementation (no plugin or modules) Non-durable Negative table-level locking can reduce concurrency, but this
table is typically small (and hopefully not used often) Memcache locking
Since you are already using memcache... (right?) Uses the (global) Memcache::Add operation Non-durable as well
-
www.percona.com
Profiling Early and Often Finding problems early is key
Missing indexes, etc long_query_time = 0
Full verbosity (if using Percona Server) Pt-query-digest
Rows_examined > Rows_sent (suboptimal index) Execution time --review track new queries over time (new
modules, etc)
-
www.percona.com
Questions?
-
Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29
