Linux Performance Co-Pilot with WebUI
Performance Co-Pilot allow sysadmins to collect and measure data from various systems, it comes in RPM packages for Red Hat 6 to 7
website http://pcp.io/
installing Performance CO-Pilot
yum -y install pcp
pcp packages comes with different services pmcd,pmlogger
pmcd : performance metrics collector daemon
[root@localhost ~]# systemctl enable pmcd ; systemctl start pmcd ; systemctl status pmcd
pmcd.service - Performance Metrics Collector Daemon
Loaded: loaded (/usr/lib/systemd/system/pmcd.service; enabled)
Active: active (exited) since Sat 2016-07-23 07:17:24 EDT; 1min 1s ago
Docs: man:pmcd(8)
Main PID: 4717 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/pmcd.service
├─4862 /usr/libexec/pcp/bin/pmcd
├─4865 /var/lib/pcp/pmdas/root/pmdaroot
├─4866 /var/lib/pcp/pmdas/proc/pmdaproc -d 3
├─4867 /var/lib/pcp/pmdas/xfs/pmdaxfs -d 11
└─4868 /var/lib/pcp/pmdas/linux/pmdalinux
Jul 23 07:17:24 localhost.localdomain systemd[1]: Starting Performance Metrics Collector Daemon...
Jul 23 07:17:24 localhost.localdomain pmcd[4717]: Rebuilding PMNS ...
Jul 23 07:17:24 localhost.localdomain pmcd[4717]: Starting pmcd ...
Jul 23 07:17:24 localhost.localdomain systemd[1]: Started Performance Metrics Collector Daemon.
Jul 23 07:18:26 localhost.localdomain systemd[1]: Started Performance Metrics Collector Daemon.
[root@localhost ~]#
pcp packages come with many commands to gather information about the machine like pmatop, pmstat, pminfo, pmval
pmatop one of my favourite tool as it gives you a big picture
it shows information about disk , memory, cpu , network , process , swap , lvm
pmstat it show loadavg, memory , swap , io , system , cpu come with -s ( sample counter [how many times it should collect this data ] ) and -t for the time interval
[root@localhost ~]# pmstat
@ Sat Jul 23 07:27:54 2016
loadavg memory swap io system cpu
1 min swpd free buff cache pi po bi bo in cs us sy id
0.08 2720 119224 0 257332 0 0 1 0 139 129 1 1 98
0.07 2716 118768 0 257404 0 0 32 6 590 912 6 8 81
0.07 2716 118752 0 257404 0 0 0 1 138 112 1 1 99
0.06 2716 118752 0 257404 0 0 0 6 107 102 0 0 100
0.06 2716 118720 0 257404 0 0 0 80 131 113 1 1 99
0.05 2716 118516 0 257404 0 0 0 0 98 98 0 0 100
0.05 2716 118564 0 257404 0 0 0 0 123 111 1 1 99
0.12 2716 118564 0 257404 0 0 0 1 94 96 0 0 100
0.11 2716 118548 0 257412 0 0 0 4 118 109 0 1 99
0.10 2716 118548 0 257412 0 0 0 0 98 97 0 0 100
0.10 2716 118500 0 257412 0 0 0 115 130 114 0 1 99
0.09 2716 118500 0 257412 0 0 0 0 102 100 0 0 100
0.08 2716 118696 0 257412 0 0 0 0 132 122 1 1 99
^C
pminfo command to list all available metrics
[root@localhost ~]# pminfo |grep kernel kernel.all.load kernel.all.intr kernel.all.pswitch kernel.all.sysfork kernel.all.hz kernel.all.uptime kernel.all.idletime kernel.all.nusers kernel.all.lastpid kernel.all.runnable kernel.all.nprocs kernel.all.cpu.user kernel.all.cpu.nice kernel.all.cpu.sys kernel.all.cpu.idle kernel.all.cpu.intr kernel.all.cpu.steal kernel.all.cpu.guest kernel.all.cpu.vuser --more--
lets use metrics called network.interface.in.bytes to see how many bytes we receive in our interface
[root@localhost ~]# pmval -s 5 network.interface.in.bytes
metric: network.interface.in.bytes
host: localhost.localdomain
semantics: cumulative counter (converting to rate)
units: byte (converting to byte / sec)
samples: 5
interval: 1.00 sec
eno16777736 lo
131.9 0.0
65.97 0.0
65.96 0.0
65.96 0.0
65.96 0.0
let’s start pmlogger
[root@localhost ~]# systemctl enable pmlogger ; systemctl start pmlogger; systemctl status pmlogger
pmlogger.service - Performance Metrics Archive Logger
Loaded: loaded (/usr/lib/systemd/system/pmlogger.service; enabled)
Active: active (exited) since Sat 2016-07-23 07:17:25 EDT; 19min ago
Docs: man:pmlogger(1)
Main PID: 4902 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/pmlogger.service
└─9035 /usr/libexec/pcp/bin/pmlogger -P -r -T24h10m -c config.default -m pmlogger_check 20160723.07.17
Jul 23 07:17:24 localhost.localdomain systemd[1]: Starting Performance Metrics Archive Logger...
Jul 23 07:17:25 localhost.localdomain pmlogger[4902]: Starting pmlogger ...
Jul 23 07:17:25 localhost.localdomain systemd[1]: Started Performance Metrics Archive Logger.
Jul 23 07:37:15 localhost.localdomain systemd[1]: Started Performance Metrics Archive Logger.
[root@localhost ~]#
the pmlogger service will save the log archievs into /var/log/pcp/pmlogger/[hostname]/date-day
we can use pmval with parameter -a to tell it to use this archive and set the matrix
[root@localhost ~]# pmval -a /var/log/pcp/pmlogger/localhost.localdomain/20150918.17.37.0 kernel.all.load 17:56:18.812 7.000E-02 8.000E-02 0.2100 17:56:19.812 7.000E-02 8.000E-02 0.2100 17:56:20.812 7.000E-02 8.000E-02 0.2100 17:56:21.812 7.000E-02 8.000E-02 0.2100 17:56:22.812 7.000E-02 8.000E-02 0.2100 17:56:23.812 7.000E-02 8.000E-02 0.2100 17:56:24.812 7.000E-02 8.000E-02 0.2100 17:56:25.812 7.000E-02 8.000E-02 0.2100 17:56:26.812 7.000E-02 8.000E-02 0.2100 17:56:27.812 7.000E-02 8.000E-02 0.2100 17:56:28.812 7.000E-02 8.000E-02 0.2100 17:56:29.812 7.000E-02 8.000E-02 0.2100 17:56:30.812 7.000E-02 8.000E-02 0.2100
we can assign specified start and end time for pmval
-S start time, -T end time
with ISO-formatted date example -S ‘@ Wed Feb 25 05:01:00 2016’ -T ‘@ Wed Feb 25 06:01:00 2016’
this will query 2 hours from 5am Feb 25 2016 TO 7am Feb 25 2016 with the metrics you like
not the fun part pcp offer a web real-time monitor in different flavours and styles one of my best is called vector.
first we need to install and run the pcp web service
[root@localhost ~]# yum install pcp-webjs
[root@localhost ~]# systemctl start pmwebd
[root@localhost ~]# systemctl status pmwebd
pmwebd.service - Performance Metrics Web Daemon
Loaded: loaded (/usr/lib/systemd/system/pmwebd.service; disabled)
Active: active (exited) since Sat 2016-07-23 07:53:20 EDT; 5s ago
Docs: man:pmwebd(8)
Process: 11173 ExecStart=/usr/share/pcp/lib/pmwebd start (code=exited, status=0/SUCCESS)
Main PID: 11173 (code=exited, status=0/SUCCESS)
Jul 23 07:53:19 localhost.localdomain systemd[1]: Starting Performance Metrics Web Daemon...
Jul 23 07:53:20 localhost.localdomain pmwebd[11173]: Starting pmwebd ...
Jul 23 07:53:20 localhost.localdomain systemd[1]: Started Performance Metrics Web Daemon.
check which port this service use
[root@localhost ~]# ss -anultp|grep pmweb
tcp LISTEN 0 32 *:44323 *:* users:(("pmwebd",11199,4))
tcp LISTEN 0 32 :::44323 :::* users:(("pmwebd",11199,5))
[root@localhost ~]#
access to localhost:44323/vector
it shows nice metrics of disk IOPS, Throughputs, Network Packets and more
happy debugging folks
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Leave a Reply