Mysql query cache fragmentation, cache hit rate and Nagios monitoring
Mysql 的优化方案，在互联网上可以查找到非常多资料，今天对Mysql缓存碎片和命中率作了详细了解，个人作了简单整理。
一、Mysql查询缓存碎片和缓存命中率。
mysql> SHOW STATUS LIKE 'qcache%';
+-------------------------+-----------+
| Variable_name | Value |
+-------------------------+-----------+
| Qcache_free_blocks | 5 |
| Qcache_free_memory | 134176648 |
| Qcache_hits | 110 |
| Qcache_inserts | 245 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 7119 |
| Qcache_queries_in_cache | 9 |
| Qcache_total_blocks | 31 |
+-------------------------+-----------+
8 rows in set (0.01 sec)MySQL 查询缓存变量
变量名
说明
Qcache_free_blocks
缓存中相邻内存块的个数。数目大说明可能有碎片。FLUSH QUERY CACHE 会对缓存中的碎片进行整理，从而得到一个空闲块。
Qcache_free_memory
缓存中的空闲内存。
Qcache_hits
每次查询在缓存中命中时就增大。
Qcache_inserts
每次插入一个查询时就增大。命中次数除以插入次数就是不中比率；用 1 减去这个值就是命中率。在上面这个例子中，大约有 87% 的查询都在缓存中命中。
Qcache_lowmem_prunes
缓存出现内存不足并且必须要进行清理以便为更多查询提供空间的次数。这个数字最好长时间来看；如果这个数字在不断增长，就表示可能碎片非常严重，或者内存很少。（上面的 free_blocks 和 free_memory 可以告诉您属于哪种情况）。
Qcache_not_cached
不适合进行缓存的查询的数量，通常是由于这些查询不是 SELECT 语句。
Qcache_queries_in_cache
当前缓存的查询（和响应）的数量。
Qcache_total_blocks
缓存中块的数量。
mysql> SHOW VARIABLES LIKE '%query_cache%'; +------------------------------+-----------+ | Variable_name | Value | +------------------------------+-----------+ | have_query_cache | YES | | query_cache_limit | 1048576 | | query_cache_min_res_unit | 4096 | | query_cache_size | 134217728 | | query_cache_type | ON | | query_cache_wlock_invalidate | OFF | +------------------------------+-----------+ 6 rows in set (0.00 sec)
query_cache_min_res_unit 查询缓存分配的最小块的大小(字节)
query_alloc_block_size 为查询分析和执行过程中创建的对象分配的内存块大小
Qcache_free_blocks 代表内存自由块的多少，反映了内存碎片的情况
==========================
1）当查询进行的时候，Mysql把查询结果保存在qurey cache中，但如果要保存的结果比较大，超过query_cache_min_res_unit的值 ，这时候mysql将一边检索结果，一边进行保存结果，所以，有时候并不是把所有结果全部得到后再进行一次性保存，而是每次分配一块 query_cache_min_res_unit 大小的内存空间保存结果集，使用完后，接着再分配一个这样的块，如果还不不够，接着再分配一个块，依此类推，也就是说，有可能在一次查询中，mysql要 进行多次内存分配的操作。
2）内存碎片的产生。当一块分配的内存没有完全使用时，MySQL会把这块内存Trim掉，把没有使用的那部分归还以重 复利用。比如，第一次分配4KB,只用了3KB，剩1KB，第二次连续操作，分配4KB，用了2KB，剩2KB，这两次连续操作共剩下的 1KB+2KB=3KB，不足以做个一个内存单元分配， 这时候，内存碎片便产生了。
3）使用flush query cache，可以消除碎片
4）如果Qcache_free_blocks值过大，可能是query_cache_min_res_unit值过大，应该调小些
5）query_cache_min_res_unit的估计值：(query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache
检查查询缓存使用情况
检查是否从查询缓存中受益的最简单的办法就是检查缓存命中率
当服务器收到SELECT 语句的时候，Qcache_hits 和Com_select 这两个变量会根据查询缓存
的情况进行递增
查询缓存命中率的计算公式是：Qcache_hits/(Qcache_hits + Com_select)。
mysql> show status like '%Com_select%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Com_select | 1 |
+---------------+-------+
1 row in set (0.00 sec)
此时的查询缓存命中率：3/（3+1）=75%；由于个人的测试数据库，查询较少，更行更少，命中率颇高。
二、监控缓存命中率
通过Nagios+pnp4nagios来监控缓存命中率，并通过图表来展示。
1、监控脚本： check_mysql_qch.sh.sh
#!/bin/bash
#function:查询缓存命中率
#time:20121130
#author:system group
while getopts ":w:c:h" optname
do
case "$optname" in
"w")
WARN=$OPTARG
;;
"c")
CIRT=$OPTARG
;;
"h")
echo "Useage: check_mysql_qch.sh -w warn -c cirt"
exit
;;
"?")
echo "Unknown option $OPTARG"
exit
;;
":")
echo "No argument value for option $OPTARG"
exit
;;
*)
# Should not occur
echo "Unknown error while processing options"
exit
;;
esac
done
[ $? -ne 0 ] && echo "error: Unknown option " && exit
[ -z $WARN ] && WARN=60
[ -z $CIRT ] && CIRT=50
export selete=`/usr/local/mysql/bin/mysql -h 127.0.0.1 -uroot -Bse "SHOW GLOBAL STATUS LIKE 'Com_select';" |awk '{print $2}'`
export hits=`/usr/local/mysql/bin/mysql -h 127.0.0.1 -uroot -Bse "SHOW GLOBAL STATUS LIKE 'Qcache_hits';" |awk '{print $2}'`
a=$(($selete+$hits))
#rw_ratio=$(($a/$b))
#echo "rw_ratio=$rw_ratio"
#ratio=$(($rw_ratio*100))
#echo "ratio=$ratio"
if [ $a -ne "0" ];then
percent=`awk 'BEGIN{printf "%.2f%\n",('$hits'/'$a')*100}'`
Qch=`awk 'BEGIN{printf ('$hits'/'$a')*100}'`
fi
C=`echo "$Qch < $CIRT" | bc`
W=`echo "$Qch < $WARN" | bc`
O=`echo "$Qch > $WARN" | bc`
if [ $C == 1 ];then
echo -e "CIRT - Mysql Qcache Hits is $percent,Com_select is $selete,Qcache_hits is $hits|Qcache_hits=${Qch}%;${selete};${hits};0"
exit 2
fi
if [ $W == 1 ];then
echo -e "WARN - Mysql Qcache Hits is $percent,Com_select is $selete,Qcache_hits is $hits|Qcache_hits=${Qch}%;${selete};${hits};0"
exit 1
fi
if [ $O == 1 ];then
echo -e "OK - Mysql Qcache Hits is $percent,Com_select is $selete,Qcache_hits is $hits|Qcache_hits=${Qch}%;${selete};${hits};0"
exit 0
fi2、生成报表
Pnp4nagios templates：check_mysql_qch.php
<?php
#
# Copyright (c) 2006-2010 system (http://www.cnfol.com)
# Plugin: check_mysql_qch
#
$opt[1] = "--vertical-label hits/s -l0 --title \"Mysql Qcache Hits for $hostname / $servicedesc\" ";
#
#
#
$def[1] = rrd::def("var1", $RRDFILE[1], $DS[1], "AVERAGE");
if ($WARN[1] != "") {
$def[1] .= "HRULE:$WARN[1]#FFFF00 ";
}
if ($CRIT[1] != "") {
$def[1] .= "HRULE:$CRIT[1]#FF0000 ";
}
$def[1] .= rrd::area("var1", "#0000FF", "Mysql Qcache Hits percent") ;
$def[1] .= rrd::gprint("var1", array("LAST", "AVERAGE", "MAX"), "%6.2lf");
?>结果：
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1657
14
1415
52
1309
25
1257
29
1229
24
MySQL's Role: Databases in Web Applications
Apr 17, 2025 am 12:23 AM
The main role of MySQL in web applications is to store and manage data. 1.MySQL efficiently processes user information, product catalogs, transaction records and other data. 2. Through SQL query, developers can extract information from the database to generate dynamic content. 3.MySQL works based on the client-server model to ensure acceptable query speed.
How to start mysql by docker
Apr 15, 2025 pm 12:09 PM
The process of starting MySQL in Docker consists of the following steps: Pull the MySQL image to create and start the container, set the root user password, and map the port verification connection Create the database and the user grants all permissions to the database
Laravel Introduction Example
Apr 18, 2025 pm 12:45 PM
Laravel is a PHP framework for easy building of web applications. It provides a range of powerful features including: Installation: Install the Laravel CLI globally with Composer and create applications in the project directory. Routing: Define the relationship between the URL and the handler in routes/web.php. View: Create a view in resources/views to render the application's interface. Database Integration: Provides out-of-the-box integration with databases such as MySQL and uses migration to create and modify tables. Model and Controller: The model represents the database entity and the controller processes HTTP requests.
Solve database connection problem: a practical case of using minii/db library
Apr 18, 2025 am 07:09 AM
I encountered a tricky problem when developing a small application: the need to quickly integrate a lightweight database operation library. After trying multiple libraries, I found that they either have too much functionality or are not very compatible. Eventually, I found minii/db, a simplified version based on Yii2 that solved my problem perfectly.
Laravel framework installation method
Apr 18, 2025 pm 12:54 PM
Article summary: This article provides detailed step-by-step instructions to guide readers on how to easily install the Laravel framework. Laravel is a powerful PHP framework that speeds up the development process of web applications. This tutorial covers the installation process from system requirements to configuring databases and setting up routing. By following these steps, readers can quickly and efficiently lay a solid foundation for their Laravel project.
MySQL vs. Other Programming Languages: A Comparison
Apr 19, 2025 am 12:22 AM
Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages have advantages in their respective fields such as data analytics, enterprise applications, and system programming.
MySQL and phpMyAdmin: Core Features and Functions
Apr 22, 2025 am 12:12 AM
MySQL and phpMyAdmin are powerful database management tools. 1) MySQL is used to create databases and tables, and to execute DML and SQL queries. 2) phpMyAdmin provides an intuitive interface for database management, table structure management, data operations and user permission management.
MySQL vs. Other Databases: Comparing the Options
Apr 15, 2025 am 12:08 AM
MySQL is suitable for web applications and content management systems and is popular for its open source, high performance and ease of use. 1) Compared with PostgreSQL, MySQL performs better in simple queries and high concurrent read operations. 2) Compared with Oracle, MySQL is more popular among small and medium-sized enterprises because of its open source and low cost. 3) Compared with Microsoft SQL Server, MySQL is more suitable for cross-platform applications. 4) Unlike MongoDB, MySQL is more suitable for structured data and transaction processing.
