服务器版本:CentOS Linux release 7.1.1503 (Core)
最近老是半夜收到阿里云的短信提示服务器的磁盘使用率超标了。
使用sar -A
看到有异常情况如下。但是具体为啥会超标该怎么继续分析?
12:00:01 AM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff
12:10:01 AM 0.09 3.85 220.27 0.00 79.41 0.00 0.00 0.00 0.00
12:20:01 AM 0.01 1.91 206.10 0.00 75.79 0.00 0.00 0.00 0.00
12:30:01 AM 0.01 2.03 206.22 0.00 75.87 0.00 0.00 0.00 0.00
12:40:01 AM 0.00 3.80 210.53 0.00 77.26 0.00 0.00 0.00 0.00
12:50:01 AM 0.00 2.09 206.14 0.00 75.57 0.00 0.00 0.00 0.00
01:00:01 AM 0.00 3.18 206.27 0.00 75.88 0.00 0.00 0.00 0.00
01:10:01 AM 0.00 2.12 220.06 0.00 79.43 0.00 0.00 0.00 0.00
01:20:01 AM 0.00 1.97 206.23 0.00 75.79 0.00 0.00 0.00 0.00
01:30:01 AM 0.01 1.89 206.15 0.00 75.65 0.00 0.00 0.00 0.00
01:40:01 AM 0.00 3.05 220.17 0.00 79.93 0.00 0.00 0.00 0.00
01:50:01 AM 0.00 2.55 206.20 0.00 75.55 0.00 0.00 0.00 0.00
02:00:01 AM 0.00 2.05 206.18 0.00 75.47 0.00 0.00 0.00 0.00
02:10:01 AM 0.00 2.11 220.02 0.00 79.37 0.00 0.00 0.00 0.00
02:20:01 AM 0.00 1.85 206.08 0.00 75.73 0.00 0.00 0.00 0.00
02:30:01 AM 0.00 2.07 206.16 0.00 75.84 0.00 0.00 0.00 0.00
02:40:01 AM 0.00 2.39 210.69 0.00 76.96 0.00 0.00 0.00 0.00
02:50:01 AM 0.00 2.38 206.17 0.00 75.51 0.00 0.00 0.00 0.00
03:00:01 AM 0.00 3.42 206.28 0.00 75.62 0.00 0.00 0.00 0.00
03:10:01 AM 0.00 4.69 219.82 0.00 79.34 0.00 0.00 0.00 0.00
03:20:01 AM 0.00 4.75 217.01 0.00 79.04 0.00 0.00 0.00 0.00
03:30:01 AM 53.62 5.56 207.01 0.00 76.56 0.00 0.00 0.00 0.00
03:40:01 AM 287.65 37.00 245.74 0.15 100.08 0.00 0.00 0.00 0.00
03:50:01 AM 187.61 7.89 208.05 0.00 80.63 0.00 0.00 0.00 0.00
04:00:01 AM 19.75 7.07 217.46 0.00 88.48 0.00 0.00 0.00 0.00
04:10:01 AM 181.71 8.41 212.29 0.00 83.98 0.00 0.00 0.00 0.00
04:20:01 AM 385.70 9.50 222.33 0.03 674.64 34.72 0.00 33.86 97.53
04:30:01 AM 385.34 10.32 209.89 0.00 85.06 0.00 0.00 0.00 0.00
04:40:01 AM 136.85 7.67 211.48 0.00 83.73 0.00 0.00 0.00 0.00
04:50:01 AM 264.71 7.59 207.58 0.00 83.15 0.00 0.00 0.00 0.00
05:00:01 AM 393.60 7.53 210.64 0.00 85.45 0.00 0.00 0.00 0.00
05:10:01 AM 262.00 6.47 222.05 0.00 87.20 0.00 0.00 0.00 0.00
05:20:01 AM 1.09 3.71 206.89 0.00 76.29 0.00 0.00 0.00 0.00
05:30:01 AM 0.31 3.27 206.40 0.00 75.83 0.00 0.00 0.00 0.00
05:40:01 AM 0.00 2.50 210.33 0.00 76.70 0.00 0.00 0.00 0.00
05:50:01 AM 0.00 3.83 206.34 0.00 75.67 0.00 0.00 0.00 0.00
06:00:01 AM 0.00 2.37 206.22 0.00 75.64 0.00 0.00 0.00 0.00
06:10:01 AM 0.00 2.13 218.19 0.00 78.98 0.00 0.00 0.00 0.00
06:20:01 AM 0.00 1.91 206.10 0.00 75.84 0.00 0.00 0.00 0.00
06:30:01 AM 0.00 1.97 206.19 0.00 75.81 0.00 0.00 0.00 0.00
06:40:01 AM 0.01 2.25 210.30 0.00 77.00 0.00 0.00 0.00 0.00
06:50:01 AM 0.00 1.85 206.11 0.00 76.18 0.00 0.00 0.00 0.00
07:00:01 AM 0.00 2.11 206.10 0.00 75.74 0.00 0.00 0.00 0.00
07:10:01 AM 0.00 1.94 218.21 0.00 78.92 0.00 0.00 0.00 0.00
07:20:01 AM 0.00 1.90 206.10 0.00 75.53 0.00 0.00 0.00 0.00
07:30:01 AM 0.07 2.03 209.91 0.00 78.93 0.00 0.00 0.00 0.00
07:40:01 AM 0.10 2.25 218.15 0.00 77.00 0.00 0.00 0.00 0.00
07:50:01 AM 0.00 3.33 196.47 0.00 72.36 0.00 0.00 0.00 0.00
08:00:01 AM 1.12 3.37 209.59 0.01 76.83 0.00 0.00 0.00 0.00
08:10:01 AM 0.00 1.97 221.85 0.00 79.84 0.00 0.00 0.00 0.00
08:20:01 AM 0.00 3.62 207.75 0.00 75.96 0.00 0.00 0.00 0.00
08:30:01 AM 0.00 2.35 211.21 0.00 78.11 0.00 0.00 0.00 0.00
08:40:01 AM 0.00 2.25 200.32 0.00 86.28 0.00 0.00 0.00 0.00
08:50:01 AM 21.21 4.25 271.61 0.09 99.59 0.00 0.00 0.00 0.00
09:00:01 AM 9.65 3.29 414.17 0.05 167.70 0.00 0.00 0.00 0.00
09:10:01 AM 0.83 2.37 261.60 0.00 107.26 0.00 0.00 0.00 0.00
12:00:01 AM tps rtps wtps bread/s bwrtn/s
12:10:01 AM 0.72 0.01 0.71 0.17 7.71
12:20:01 AM 0.32 0.00 0.32 0.01 3.83
12:30:01 AM 0.32 0.00 0.32 0.01 4.07
12:40:01 AM 0.65 0.00 0.65 0.00 7.60
12:50:01 AM 0.33 0.00 0.33 0.00 4.17
01:00:01 AM 0.57 0.00 0.57 0.00 6.36
01:10:01 AM 0.35 0.00 0.35 0.00 4.24
01:20:01 AM 0.32 0.00 0.32 0.00 3.93
01:30:01 AM 0.31 0.00 0.31 0.01 3.79
01:40:01 AM 0.51 0.00 0.51 0.00 6.09
01:50:01 AM 0.43 0.00 0.43 0.00 5.09
02:00:01 AM 0.35 0.00 0.35 0.00 4.09
02:10:01 AM 0.35 0.00 0.35 0.00 4.21
02:20:01 AM 0.31 0.00 0.31 0.00 3.69
02:30:01 AM 0.31 0.00 0.31 0.00 4.13
02:40:01 AM 0.36 0.00 0.36 0.00 4.77
02:50:01 AM 0.40 0.00 0.40 0.00 4.76
03:00:01 AM 0.59 0.00 0.59 0.00 6.84
03:10:01 AM 0.89 0.00 0.89 0.00 9.37
03:20:01 AM 0.89 0.00 0.89 0.00 9.51
03:30:01 AM 2.23 1.64 0.59 107.24 11.12
03:40:01 AM 11.69 9.81 1.88 574.98 74.00
03:50:01 AM 7.83 7.13 0.70 375.55 15.77
04:00:01 AM 2.13 1.49 0.64 39.51 14.15
04:10:01 AM 8.36 7.63 0.74 363.42 16.81
04:20:01 AM 14.69 13.78 0.91 771.41 19.00
04:30:01 AM 14.70 13.66 1.04 770.68 20.64
04:40:01 AM 8.09 7.45 0.64 273.70 15.33
04:50:01 AM 11.26 10.64 0.62 529.42 15.19
05:00:01 AM 13.89 13.27 0.62 787.20 15.05
05:10:01 AM 9.42 8.83 0.59 523.99 12.93
05:20:01 AM 0.74 0.08 0.66 2.17 7.43
05:30:01 AM 0.60 0.02 0.59 0.63 6.55
05:40:01 AM 0.44 0.00 0.44 0.00 5.00
05:50:01 AM 0.71 0.00 0.71 0.00 7.65
06:00:01 AM 0.41 0.00 0.41 0.00 4.73
06:10:01 AM 0.36 0.00 0.36 0.00 4.25
06:20:01 AM 0.33 0.00 0.33 0.00 3.83
06:30:01 AM 0.34 0.00 0.34 0.00 3.95
06:40:01 AM 0.38 0.00 0.37 0.03 4.51
06:50:01 AM 0.32 0.00 0.32 0.00 3.71
07:00:01 AM 0.36 0.00 0.36 0.00 4.21
07:10:01 AM 0.33 0.00 0.33 0.00 3.88
07:20:01 AM 0.33 0.00 0.33 0.00 3.80
07:30:01 AM 0.36 0.01 0.34 0.15 4.07
07:40:01 AM 0.36 0.00 0.36 0.20 4.49
07:50:01 AM 0.58 0.00 0.58 0.00 6.65
08:00:01 AM 0.65 0.05 0.60 2.24 6.75
08:10:01 AM 0.34 0.00 0.34 0.00 3.95
08:20:01 AM 0.67 0.00 0.67 0.00 7.24
08:30:01 AM 0.40 0.00 0.40 0.00 4.69
08:40:01 AM 0.38 0.00 0.38 0.00 4.51
08:50:01 AM 1.44 0.87 0.57 42.43 8.51
09:00:01 AM 0.93 0.42 0.50 19.31 6.57
You can use a tool called atop, which can view and record the process list of disk I/O usage from high to low at a certain moment.
Service
/etc/init.d/atop
will run/etc/atop/atop.daily
The log collection interval (INTERVAL) is defined in atop.daily as 600 seconds.
Log storage location
/var/log/atop/atop_YYYYMMDD
is binary content. You can view it like this:atop -r /var/log/atop/atop_YYYYMMDD
Press the t key to switch to the next collection point. , press d to check the disk I/O status.Compile and install the latest version of atop:
Use atop (written in C) or iotop (written in Python) to check the I/O status of a specific process, and use iostat (sysstat) to check the global I/O load of the system.
http://www.atoptool. nl/screen...
atop interactive command:
t (manual refresh, the default refresh interval is 10 seconds, i can modify the time interval) g (default output) d (disk) m (memory)
s (scheduling) v (variable) c (command line) p (process statistics) u (user statistics) n (network, requires kernel module netatop support)
C (sort by CPU) M (sort by memory MEM) D(Sort by disk DSK) N(Sort by network NET)
atop -d View system disk overview:
The data above is the system overview: CPU and its core/MEM (memory)/SWP (swap space)/PAG (page swap)/LVM (logical volume)/DSK ( Disk)/NET (Network)
The data below is the specific process: PID RDDSK (read) WRDSK (write) WCANCL DSK (disk usage of the process) CMD
atop can also display the disk I/O and network I/O status of the system and process, which is better than top. It even developed a kernel module netatop in order to monitor the network I/O of specific processes.