{{tag>howto diagnose}}
====== Diagnose 101 ======
===== Memory =====
This will show you top 10 process that using the most memory:
ps aux --sort=-%mem | awk 'NR<=10{print $0}'
using top: when you opened top press m will short process based on memory usage.
But this will not solve your problem, in Linux everything is either file or process. so the files you opened will eating the memory too. so this will not help.
//lsof// will give you all opened files with the size of the file or the file offset in bytes.
===== Filesysteme =====
mit //df -h// nachschauen ob die NFS/SAMBA-remote-filesysteme eingehängt sind (wenn es lange dauert bis df Ausgabe produziert, dann stimmt an dieser Stelle etwas nicht)
df -h
df: /orabase/ACFS/DG1/V1/OGG: Input/output error
Filesystem Size Used Avail Use% Mounted on
devtmpfs 125G 8.0K 125G 1% /dev
tmpfs 125G 1.6G 124G 2% /dev/shm
tmpfs 125G 6.9M 125G 1% /run
tmpfs 125G 0 125G 0% /sys/fs/cgroup
/dev/sda3 2.0G 1.1G 981M 53% /
/dev/mapper/system-usr 8.0G 4.1G 4.0G 51% /usr
tmpfs 8.0G 120M 7.9G 2% /tmp
/dev/mapper/system-nosql 100G 180M 100G 1% /nosql
/dev/mapper/system-var 10G 1.8G 8.2G 18% /var
/dev/mapper/system-opt 4.0G 809M 3.3G 20% /opt
/dev/mapper/system-oemcc 10G 33M 10G 1% /opt/oemcc
/dev/mapper/system-zel 20G 9.7G 11G 49% /zel
/dev/mapper/system-home 8.0G 3.2G 4.9G 40% /home
/dev/mapper/system-var--crash 32G 25G 7.3G 78% /var/crash
/dev/mapper/system-var--log 8.0G 1.3G 6.8G 16% /var/log
/dev/mapper/system-orasw 80G 52G 29G 65% /orasw
/dev/mapper/system-var_log_sa 5.0G 407M 4.6G 8% /var/log/sa
/dev/mapper/system-vl_audit 749M 516M 234M 69% /var/log/audit
/dev/mapper/system-sudolog 2.0G 40M 2.0G 2% /var/log/sudolog
/dev/mapper/system-orabase 80G 21G 60G 26% /orabase
tmpfs 25G 0 25G 0% /run/user/2000
/dev/mapper/filevg_fs9709 639G 345G 295G 54% /zed9709
tmpfs 25G 0 25G 0% /run/user/1845
===== CPU =====
load average
sar -q
Linux 5.15.5-1-default (dellirio) 12/09/21 _x86_64_ (12 CPU)
02:41:45 LINUX RESTART (12 CPU)
02:50:11 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
03:00:11 1 1701 1.25 0.98 0.99 0
03:10:11 3 1692 2.20 1.47 1.22 0
03:20:11 4 1706 1.50 1.29 1.20 0
03:30:00 1 1755 0.70 1.30 1.35 0
03:40:11 1 1796 1.17 2.09 1.80 0
03:50:11 0 1787 1.51 1.81 1.86 0
04:00:11 3 1806 0.88 1.17 1.48 0
04:10:11 6 1753 0.77 0.78 1.13 0
04:20:11 3 1754 0.51 0.72 0.93 0
04:30:11 1 1757 0.62 0.65 0.80 0
04:40:11 0 1750 0.71 0.70 0.78 0
04:50:11 1 1753 0.69 0.72 0.74 0
05:00:11 1 1751 0.52 0.75 0.77 0
05:10:11 0 1751 0.61 0.71 0.73 0
05:20:11 0 1759 1.03 1.04 0.89 0
05:30:11 1 1749 0.75 0.92 0.89 0
05:40:11 0 1753 0.88 0.83 0.83 0
05:50:11 2 1756 0.91 0.88 0.84 0
06:00:11 3 1759 1.01 0.83 0.83 0
06:10:11 0 1754 1.09 0.86 0.81 0
06:20:11 1 1753 1.14 1.03 0.90 0
06:30:11 0 1758 0.78 0.81 0.84 0
06:40:11 0 1753 0.67 0.76 0.81 0
06:50:11 0 1745 0.77 0.78 0.81 0
07:00:11 1 1755 1.00 0.88 0.83 0
07:10:11 3 1753 0.84 0.77 0.80 0
07:20:11 0 1749 0.86 0.77 0.79 0
07:30:11 1 1749 0.67 0.66 0.73 0
07:40:11 0 1754 0.99 0.89 0.82 0
07:50:11 1 1746 0.75 0.76 0.80 0
08:00:11 1 1754 0.86 0.73 0.78 0
08:10:11 2 1750 0.70 0.74 0.77 0
08:20:11 0 1754 0.70 0.71 0.77 0
08:30:11 0 1752 0.67 0.74 0.82 0
08:40:11 1 1750 0.91 0.83 0.84 0
08:50:11 1 1749 0.71 0.75 0.81 0
09:00:11 3 1755 0.43 0.68 0.78 0
09:10:11 0 1754 0.68 0.75 0.76 0
09:20:11 2 1759 1.44 1.04 0.86 0
09:30:11 0 1754 0.64 0.80 0.85 0
09:40:11 0 1753 0.76 0.79 0.82 0
09:50:11 0 1751 0.80 0.82 0.81 0
10:00:11 2 1754 0.79 0.78 0.80 0
10:10:11 1 1756 0.82 0.90 0.86 0
10:20:11 0 1754 1.10 0.88 0.85 0
10:30:11 1 1751 0.60 0.70 0.76 0
10:40:11 0 1754 0.83 0.83 0.81 0
10:50:11 1 1753 1.04 0.91 0.86 0
11:00:11 1 1779 1.05 0.92 0.88 0
11:10:11 0 1763 0.76 0.76 0.81 0
11:20:11 1 1757 0.56 0.75 0.80 0
11:30:00 0 1762 1.00 0.84 0.80 0
11:40:01 1 1758 0.77 0.69 0.73 0
11:50:02 0 1753 1.11 0.96 0.82 0
12:00:03 2 1760 0.82 0.85 0.83 0
12:10:03 0 1759 0.80 0.78 0.82 0
12:20:04 2 1763 1.12 0.86 0.83 0
12:20:04 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
12:30:00 3 1771 0.55 0.66 0.77 0
12:40:01 3 1767 0.54 0.71 0.77 0
12:50:02 7 1766 0.74 0.72 0.77 0
13:00:03 6 1765 0.69 0.88 0.87 0
13:10:05 5 1760 0.77 0.80 0.82 0
13:20:05 4 1756 0.77 0.86 0.84 0
13:30:06 5 1754 0.59 0.70 0.80 0
13:40:07 0 1756 0.73 0.86 0.85 0
Average: 1 1755 0.86 0.87 0.88 0
===== user/system/idle/wait =====
sar -u
Linux 5.15.5-1-default (dellirio) 12/09/21 _x86_64_ (12 CPU)
02:41:45 LINUX RESTART (12 CPU)
02:50:11 CPU %user %nice %system %iowait %steal %idle
03:00:11 all 8.33 0.00 0.93 0.02 0.00 90.72
03:10:11 all 9.00 0.00 1.13 0.02 0.00 89.85
03:20:11 all 10.42 0.00 1.42 0.01 0.00 88.14
03:30:00 all 12.78 0.00 1.00 0.02 0.00 86.20
03:40:11 all 19.48 0.00 1.01 0.02 0.00 79.49
03:50:11 all 17.14 0.01 1.15 0.01 0.00 81.69
04:00:11 all 9.82 0.00 1.25 0.01 0.00 88.91
04:10:11 all 8.07 0.00 0.89 0.02 0.00 91.02
04:20:11 all 6.65 0.00 0.50 0.01 0.00 92.84
04:30:11 all 6.45 0.00 0.48 0.01 0.00 93.06
04:40:11 all 6.53 0.00 0.47 0.01 0.00 92.98
04:50:11 all 6.44 0.00 0.46 0.01 0.00 93.08
05:00:11 all 6.55 0.00 0.49 0.01 0.00 92.96
05:10:11 all 6.54 0.00 0.47 0.01 0.00 92.98
05:20:11 all 6.54 0.00 0.50 0.01 0.00 92.95
05:30:11 all 6.36 0.00 0.46 0.01 0.00 93.17
05:40:11 all 6.62 0.00 0.49 0.01 0.00 92.88
05:50:11 all 6.54 0.00 0.48 0.01 0.00 92.97
06:00:11 all 6.47 0.00 0.47 0.01 0.00 93.05
06:10:11 all 6.64 0.00 0.51 0.01 0.00 92.83
06:20:11 all 6.55 0.00 0.49 0.01 0.00 92.95
06:30:11 all 6.49 0.00 0.48 0.01 0.00 93.02
06:40:11 all 6.61 0.00 0.48 0.01 0.00 92.90
06:50:11 all 6.52 0.00 0.48 0.01 0.00 92.99
07:00:11 all 6.46 0.00 0.48 0.01 0.00 93.04
07:10:11 all 6.66 0.00 0.50 0.01 0.00 92.83
07:20:11 all 6.48 0.00 0.49 0.01 0.00 93.02
07:30:11 all 6.43 0.00 0.47 0.01 0.00 93.08
07:40:11 all 6.62 0.00 0.49 0.01 0.00 92.87
07:50:11 all 6.50 0.00 0.47 0.01 0.00 93.02
08:00:11 all 6.51 0.00 0.48 0.01 0.00 93.00
08:10:11 all 6.46 0.00 0.48 0.01 0.00 93.04
08:20:11 all 6.50 0.00 0.47 0.01 0.00 93.01
08:30:11 all 6.44 0.00 0.47 0.01 0.00 93.08
08:40:11 all 6.58 0.00 0.48 0.01 0.00 92.92
08:50:11 all 6.44 0.00 0.48 0.01 0.00 93.07
09:00:11 all 6.51 0.00 0.47 0.01 0.00 93.01
09:10:11 all 6.60 0.00 0.49 0.02 0.00 92.89
09:20:11 all 6.44 0.00 0.48 0.01 0.00 93.07
09:30:11 all 6.53 0.00 0.48 0.01 0.00 92.98
09:40:11 all 6.61 0.00 0.50 0.01 0.00 92.88
09:50:11 all 6.40 0.00 0.47 0.01 0.00 93.11
10:00:11 all 6.49 0.00 0.46 0.01 0.00 93.04
10:10:11 all 6.53 0.00 0.49 0.01 0.00 92.97
10:20:11 all 6.55 0.00 0.48 0.01 0.00 92.95
10:30:11 all 6.44 0.00 0.48 0.01 0.00 93.07
10:40:11 all 6.58 0.00 0.48 0.01 0.00 92.93
10:50:11 all 6.45 0.00 0.48 0.01 0.00 93.06
11:00:11 all 6.64 0.00 0.54 0.01 0.00 92.80
11:10:11 all 6.60 0.00 0.50 0.01 0.00 92.89
11:20:11 all 6.54 0.00 0.49 0.01 0.00 92.96
11:30:00 all 6.52 0.00 0.48 0.01 0.00 92.99
11:40:01 all 6.63 0.00 0.51 0.01 0.00 92.85
11:50:02 all 6.60 0.00 0.49 0.01 0.00 92.90
12:00:03 all 6.42 0.00 0.49 0.01 0.00 93.08
12:10:03 all 6.63 0.00 0.48 0.01 0.00 92.88
12:20:04 all 6.52 0.00 0.49 0.01 0.00 92.98
12:20:04 CPU %user %nice %system %iowait %steal %idle
12:30:00 all 6.56 0.00 0.51 0.01 0.00 92.92
12:40:01 all 6.57 0.00 0.49 0.01 0.00 92.93
12:50:02 all 6.52 0.00 0.48 0.01 0.00 92.98
13:00:03 all 6.52 0.00 0.50 0.01 0.00 92.97
13:10:05 all 6.61 0.00 0.49 0.01 0.00 92.89
13:20:05 all 6.48 0.00 0.48 0.01 0.00 93.02
13:30:06 all 6.45 0.00 0.48 0.01 0.00 93.05
13:40:07 all 6.69 0.00 0.53 0.01 0.00 92.76
13:50:08 all 9.78 0.00 1.37 0.04 0.00 88.81
14:00:05 all 10.89 0.00 3.08 0.39 0.00 85.63
14:10:09 all 10.17 0.01 5.68 0.37 0.00 83.77
14:20:11 all 7.78 0.06 1.45 0.10 0.00 90.62
14:30:11 all 6.33 0.00 0.48 0.01 0.00 93.18
14:40:11 all 6.43 0.00 0.48 0.01 0.00 93.08
14:50:11 all 6.39 0.00 0.46 0.01 0.00 93.14
15:00:11 all 6.51 0.00 0.47 0.01 0.00 93.01
15:10:11 all 6.59 0.00 0.49 0.01 0.00 92.92
15:20:11 all 6.55 0.00 0.48 0.00 0.00 92.97
15:30:11 all 6.46 0.00 0.47 0.01 0.00 93.06
15:40:11 all 6.63 0.00 0.53 0.01 0.00 92.84
15:50:11 all 6.42 0.00 0.47 0.01 0.00 93.10
16:00:11 all 6.42 0.00 0.49 0.00 0.00 93.09
16:10:11 all 6.54 0.00 0.49 0.01 0.00 92.97
16:20:11 all 6.57 0.00 0.48 0.01 0.00 92.94
16:30:11 all 6.43 0.00 0.47 0.00 0.00 93.10
16:40:11 all 6.55 0.00 0.51 0.01 0.00 92.94
16:50:11 all 6.47 0.00 0.48 0.01 0.00 93.05
17:00:11 all 6.59 0.00 0.52 0.01 0.00 92.89
17:10:11 all 6.59 0.00 0.49 0.01 0.00 92.91
17:20:11 all 6.94 0.00 0.64 0.01 0.00 92.42
17:30:11 all 6.49 0.00 0.49 0.00 0.00 93.02
17:40:11 all 6.57 0.00 0.49 0.01 0.00 92.92
17:50:11 all 6.43 0.00 0.46 0.00 0.00 93.10
18:00:11 all 6.41 0.00 0.47 0.00 0.00 93.11
18:10:11 all 6.59 0.00 0.49 0.01 0.00 92.92
18:20:11 all 6.52 0.00 0.50 0.01 0.00 92.97
18:30:11 all 5.75 0.00 3.30 0.02 0.00 90.93
18:40:11 all 6.59 0.00 0.52 0.01 0.00 92.89
18:50:11 all 6.54 0.00 0.50 0.01 0.00 92.95
19:00:11 all 6.44 0.00 0.48 0.01 0.00 93.07
19:10:11 all 6.47 0.00 0.49 0.01 0.00 93.04
19:20:11 all 6.52 0.00 0.50 0.00 0.00 92.98
19:30:11 all 6.38 0.00 0.49 0.00 0.00 93.13
19:40:11 all 6.56 0.00 0.49 0.01 0.00 92.95
19:50:11 all 6.53 0.00 0.48 0.00 0.00 92.99
20:00:11 all 6.45 0.00 0.48 0.01 0.00 93.07
20:10:11 all 6.55 0.00 0.50 0.01 0.00 92.95
20:20:11 all 6.56 0.00 0.49 0.01 0.00 92.94
20:30:11 all 6.50 0.00 0.47 0.00 0.00 93.02
20:40:11 all 6.51 0.00 0.48 0.01 0.00 93.01
20:50:11 all 6.46 0.00 0.52 0.00 0.00 93.01
21:00:11 all 6.45 0.00 0.47 0.00 0.00 93.07
21:10:11 all 6.50 0.00 0.49 0.01 0.00 93.00
21:20:11 all 6.54 0.00 0.49 0.01 0.00 92.97
21:30:11 all 6.73 0.00 0.52 0.01 0.00 92.75
21:40:11 all 6.48 0.00 0.50 0.01 0.00 93.02
21:50:11 all 6.56 0.00 0.47 0.00 0.00 92.97
21:50:11 CPU %user %nice %system %iowait %steal %idle
22:00:11 all 6.45 0.00 0.49 0.00 0.00 93.05
22:10:09 all 6.61 0.00 0.53 0.01 0.00 92.85
22:20:10 all 6.89 0.00 0.57 0.01 0.00 92.53
22:30:11 all 6.46 0.00 0.48 0.01 0.00 93.06
22:40:11 all 6.48 0.00 0.48 0.01 0.00 93.04
22:50:11 all 8.18 0.01 0.94 0.02 0.00 90.86
23:00:11 all 10.03 0.00 1.32 0.01 0.00 88.64
Average: all 7.02 0.00 0.64 0.02 0.00 92.32
===== Page scans =====
# pgscand/s
sar -B
Linux 5.15.5-1-default (dellirio) 12/09/21 _x86_64_ (12 CPU)
02:41:45 LINUX RESTART (12 CPU)
02:50:11 pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff
03:00:11 2159.79 474.13 2044.61 0.43 2694.87 684.83 3.15 969.08 140.86
03:10:11 14.27 291.08 2818.88 0.46 3140.79 42.18 8.94 64.69 126.55
03:20:11 22.18 322.18 4364.56 0.09 4605.91 228.94 10.70 186.91 77.99
03:30:00 211.07 435.64 6575.73 0.51 4641.67 574.13 0.00 484.18 84.33
03:40:11 227.27 575.39 8227.81 0.68 7136.55 932.59 7.78 1169.61 124.38
03:50:11 133.24 274.62 9761.47 0.77 9079.62 426.59 1.83 752.30 175.60
04:00:11 68.96 196.59 3620.86 0.43 3839.63 10.41 0.97 22.01 193.50
04:10:11 108.41 343.67 2161.80 1.69 2952.70 6.90 3.54 19.28 184.71
04:20:11 31.92 170.10 937.49 0.05 975.63 0.00 0.00 0.00 0.00
04:30:11 0.57 103.90 589.12 0.00 697.15 0.00 0.00 0.00 0.00
04:40:11 2.06 235.75 649.01 0.00 744.08 0.00 0.00 0.00 0.00
04:50:11 0.20 118.81 556.53 0.00 600.60 0.00 0.00 0.00 0.00
05:00:11 2.40 114.81 545.01 0.05 595.20 0.00 0.00 0.00 0.00
05:10:11 0.45 246.77 689.25 0.03 819.35 0.00 0.00 0.00 0.00
05:20:11 0.40 129.98 661.70 0.01 1109.42 0.00 0.00 0.00 0.00
05:30:11 0.08 117.62 521.24 0.00 582.06 0.00 0.00 0.00 0.00
05:40:11 2.76 249.75 643.12 0.01 780.15 0.00 0.00 0.00 0.00
05:50:11 6.73 126.30 596.49 0.00 671.95 0.00 0.00 0.00 0.00
06:00:11 0.01 96.68 530.67 0.00 618.66 0.00 0.00 0.00 0.00
06:10:11 2.57 266.10 720.01 0.03 858.60 0.00 0.00 0.00 0.00
06:20:11 0.12 121.25 635.55 0.00 717.99 0.00 0.00 0.00 0.00
06:30:11 0.26 113.68 574.20 0.00 767.13 0.00 0.00 0.00 0.00
06:40:11 0.17 261.27 665.27 0.00 784.80 0.00 0.00 0.00 0.00
06:50:11 0.04 118.45 551.86 0.00 629.77 0.00 0.00 0.00 0.00
07:00:11 2.53 112.97 565.24 0.00 613.99 0.00 0.00 0.00 0.00
07:10:11 0.23 285.62 686.18 0.00 835.89 0.00 0.00 0.00 0.00
07:20:11 0.05 138.78 628.65 0.00 682.12 0.00 0.00 0.00 0.00
07:30:11 0.00 122.99 537.63 0.00 599.31 0.00 0.00 0.00 0.00
07:40:11 0.10 274.37 652.92 0.00 818.27 0.00 0.00 0.00 0.00
07:50:11 0.00 118.21 556.10 0.00 616.73 0.00 0.00 0.00 0.00
08:00:11 0.05 120.01 572.02 0.00 637.66 0.00 0.00 0.00 0.00
08:10:11 14.22 266.43 687.34 0.01 825.56 0.00 0.00 0.00 0.00
08:20:11 2.17 139.62 666.28 0.00 793.85 0.00 0.00 0.00 0.00
08:30:11 7.75 131.67 561.53 0.02 716.39 0.00 0.00 0.00 0.00
08:40:11 0.00 256.77 662.32 0.00 859.19 0.00 0.00 0.00 0.00
08:50:11 0.03 123.38 563.36 0.00 674.44 0.00 0.00 0.00 0.00
09:00:11 0.00 114.11 547.27 0.00 705.90 0.00 0.00 0.00 0.00
09:10:11 12.79 252.19 702.99 0.26 920.45 0.00 0.00 0.00 0.00
09:20:11 0.39 147.49 638.85 0.05 760.01 0.00 0.00 0.00 0.00
09:30:11 0.04 136.47 564.92 0.00 708.38 0.00 0.00 0.00 0.00
09:40:11 0.03 255.35 652.99 0.00 878.96 0.00 0.00 0.00 0.00
09:50:11 0.06 132.95 581.55 0.00 698.58 0.00 0.00 0.00 0.00
10:00:11 0.03 139.92 576.65 0.00 706.77 0.00 0.00 0.00 0.00
10:10:11 0.11 255.14 684.26 0.00 872.31 0.00 0.00 0.00 0.00
10:20:11 0.03 143.43 634.54 0.00 817.30 0.00 0.00 0.00 0.00
10:30:11 0.05 141.67 570.88 0.00 703.73 0.00 0.00 0.00 0.00
10:40:11 0.21 260.65 691.39 0.01 879.73 0.00 0.00 0.00 0.00
10:50:11 0.03 126.31 569.04 0.00 692.55 0.00 0.00 0.00 0.00
11:00:11 13.99 225.77 752.19 0.45 992.62 0.00 0.00 0.00 0.00
11:10:11 0.07 252.40 741.12 0.01 1017.54 0.00 0.00 0.00 0.00
11:20:11 4.35 140.42 667.81 0.02 802.82 0.00 0.00 0.00 0.00
11:30:00 0.37 122.16 563.25 0.00 744.91 0.00 0.00 0.00 0.00
11:40:01 0.11 271.16 680.93 0.00 892.33 0.00 0.00 0.00 0.00
11:50:02 5.03 151.65 614.87 0.00 791.47 0.00 0.00 0.00 0.00
12:00:03 0.04 132.86 575.88 0.00 712.23 0.00 0.00 0.00 0.00
12:10:03 0.03 252.91 676.79 0.00 908.54 0.00 0.00 0.00 0.00
12:20:04 0.39 147.20 688.58 0.01 848.35 0.00 1.81 1.65 91.53
12:20:04 pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff
12:30:00 3.05 195.14 724.28 0.01 930.38 0.00 0.00 0.00 0.00
12:40:01 0.05 265.53 725.85 0.00 935.13 0.00 0.28 0.27 97.62
12:50:02 0.08 136.79 598.39 0.00 743.05 0.00 0.00 0.00 0.00
13:00:03 0.15 164.60 585.57 0.00 750.99 0.00 0.46 0.65 140.29
13:10:05 10.86 270.11 743.23 0.32 1012.90 0.00 0.00 0.00 0.00
13:20:05 0.25 133.39 714.48 0.01 831.17 0.00 1.43 2.24 157.06
13:30:06 3.19 163.83 595.92 0.00 765.62 0.00 0.00 0.00 0.00
13:40:07 17.53 295.47 753.81 0.04 1006.05 0.00 0.74 0.92 124.89
13:50:08 1500.23 2825.21 4220.89 2.36 4988.38 1102.40 0.53 913.63 82.84
14:00:05 3497.65 19664.66 17760.30 21.32 19182.88 4749.71 0.00 4638.51 97.66
14:10:09 7508.38 26189.41 28168.63 12.67 37140.56 3818.99 0.42 4797.61 125.61
14:20:11 7462.33 1301.18 1929.30 2.10 51368.51 2506.62 9.02 3611.18 143.55
14:30:11 106.62 275.94 624.41 0.52 760.10 74.37 0.00 73.46 98.77
14:40:11 35.00 461.89 674.80 0.13 867.32 81.00 0.00 108.68 134.17
14:50:11 4.60 272.93 559.00 0.03 690.39 0.00 0.00 0.00 0.00
15:00:11 9.38 283.60 563.90 0.15 682.50 15.57 0.00 27.36 175.78
15:10:11 1.03 395.51 655.12 0.03 908.90 25.17 0.00 25.16 99.97
15:20:11 2.63 259.14 633.32 0.02 787.45 0.00 0.00 0.00 0.00
15:30:11 23.03 243.15 568.04 0.27 751.19 0.00 0.00 0.00 0.00
15:40:11 249.32 478.42 839.47 3.06 1220.78 102.94 2.02 177.56 169.18
15:50:11 0.30 263.99 579.86 0.01 738.63 0.00 0.00 0.00 0.00
16:00:11 6.16 304.27 566.86 0.03 727.45 0.00 0.00 0.00 0.00
16:10:11 22.66 461.11 797.63 0.11 1114.12 42.73 1.76 78.06 175.47
16:20:11 20.59 289.50 686.36 0.13 892.29 0.00 12.96 25.27 194.96
16:30:11 6.28 260.99 585.05 0.20 824.54 0.00 4.31 8.32 193.04
16:40:11 63.72 464.46 788.90 1.87 1000.21 0.00 0.00 0.00 0.00
16:50:11 0.17 312.16 595.96 0.01 798.00 0.00 0.00 0.00 0.00
17:00:11 15.58 353.77 703.13 0.50 928.24 0.00 0.00 0.00 0.00
17:10:11 6.35 463.17 683.95 0.14 965.99 50.44 0.00 87.16 172.80
17:20:11 2.24 404.11 852.33 0.12 1148.76 0.00 0.00 0.00 0.00
17:30:11 0.79 329.23 600.76 0.07 775.42 0.00 2.64 5.13 194.32
17:40:11 0.97 462.83 734.01 0.03 985.74 7.37 3.28 20.87 195.99
17:50:11 0.06 269.99 572.75 0.01 744.20 0.00 0.43 0.78 184.31
18:00:11 2.95 282.43 559.24 0.02 735.91 0.00 7.18 12.81 178.33
18:10:11 7.49 426.80 754.54 0.02 1029.93 14.22 9.14 39.07 167.23
18:20:11 38.10 343.63 722.21 0.73 923.66 0.00 7.30 13.22 181.06
18:30:11 1890.43 475.87 1852.15 0.38 3877.73 1477.39 7.27 1530.73 103.10
18:40:11 12.11 362.11 758.64 0.09 972.39 0.00 0.00 0.00 0.00
18:50:11 5.01 185.02 622.25 0.04 792.26 0.00 0.00 0.00 0.00
19:00:11 1.17 161.32 569.85 0.06 697.54 0.00 0.69 1.20 173.49
19:10:11 3.57 248.52 707.11 0.10 883.61 3.27 3.24 12.73 195.80
19:20:11 3.03 145.74 707.86 0.03 838.90 0.00 0.00 0.00 0.00
19:30:11 0.53 163.73 613.13 0.07 729.05 10.95 0.34 22.07 195.45
19:40:11 17.60 262.27 737.98 0.19 945.50 8.70 1.79 20.73 197.55
19:50:11 0.03 129.05 579.51 0.01 715.93 0.00 0.00 0.00 0.00
20:00:11 0.04 178.64 574.52 0.01 692.40 1.09 0.00 2.14 196.64
20:10:11 52.93 267.99 849.40 0.22 1146.37 39.39 0.00 78.38 198.98
20:20:11 33.46 140.97 666.55 0.45 841.07 0.00 0.00 0.00 0.00
20:30:11 6.99 108.92 564.66 0.06 707.29 0.00 1.06 1.94 183.60
20:40:11 0.03 228.57 683.28 0.01 873.53 0.00 5.95 10.87 182.75
20:50:11 1.78 142.83 615.92 0.02 849.07 0.00 6.25 12.14 194.24
21:00:11 0.02 110.33 536.53 0.01 697.26 0.00 5.52 9.34 169.10
21:10:11 26.75 253.67 718.77 1.10 926.06 0.00 5.15 8.09 156.92
21:20:11 0.78 108.53 664.89 0.18 828.10 0.00 6.73 9.97 148.16
21:30:11 0.36 107.63 634.68 0.03 864.87 0.00 14.52 27.44 188.91
21:40:11 0.16 265.75 704.65 0.01 907.50 0.00 0.34 0.53 157.21
21:50:11 0.17 107.22 607.32 0.00 801.65 0.00 0.00 0.00 0.00
21:50:11 pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff
22:00:11 0.13 147.05 570.40 0.02 735.08 0.00 4.46 8.37 187.45
22:10:09 190.58 506.01 1000.52 0.50 1301.50 23.45 0.00 46.35 197.63
22:20:10 117.12 584.55 1828.52 0.10 2143.24 111.97 0.64 223.33 198.32
22:30:11 19.58 125.50 589.88 0.04 761.71 0.00 0.00 0.00 0.00
22:40:11 4.71 273.35 790.63 0.12 982.89 0.00 1.96 3.35 170.75
22:50:11 346.74 188.52 2135.91 14.42 2501.70 0.00 0.00 0.00 0.00
23:00:11 33.03 253.85 5366.47 0.88 5466.65 43.77 0.00 86.02 196.54
23:10:11 9.29 300.25 4808.97 0.18 4991.85 56.38 2.89 116.08 195.84
Average: 217.45 639.20 1442.44 0.59 2100.62 141.75 1.41 168.89 117.98
===== free =====
Speicher/SWAP-Auslastung in GB
free -g
total used free shared buff/cache available
Mem: 16318416 7295948 451604 524528 8570864 8159504
Swap: 16322880 4127488 12195392
===== Netzwerk =====
# RX-ERR
netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
bond0 1500 0 500615690 0 680848 0 495289774 0 0 0 BMmRU
bond1 1500 0 7118264 0 387171 0 5325736 0 0 0 BMmRU
bond1:1 1500 0 - no statistics available - BMmRU
bond1:2 1500 0 - no statistics available - BMmRU
bond1:4 1500 0 - no statistics available - BMmRU
bond1:5 1500 0 - no statistics available - BMmRU
bond1:6 1500 0 - no statistics available - BMmRU
bond2 1500 0 730890 0 227226 0 4492 0 0 0 BMmRU
eth0 1500 0 680901 0 680846 0 0 0 0 0 BMsRU
eth1 1500 0 386965 0 386965 0 0 0 0 0 BMsRU
eth2 1500 0 227224 0 227224 0 0 0 0 0 BMsRU
eth3 1500 0 503666 0 0 0 4492 0 0 0 BMsRU
eth4 1500 0 499934789 0 0 0 495289774 0 0 0 BMsRU
eth5 1500 0 6731299 0 0 0 5325736 0 0 0 BMsRU
eth6 9000 0 656029363 0 214 0 703127744 0 0 0 BMRU
eth6:1 9000 0 - no statistics available - BMRU
eth7 9000 0 1173507454 0 216 0 1037376088 0 0 0 BMRU
eth7:1 9000 0 - no statistics available - BMRU
lo 65536 0 736520436 0 0 0 736520436 0 0 0 LRU
Im Block "Tcp" das Verhältnis von segments retransmitted zu segments send out (Diagnose wie viele Pakete retransmitted werden mussten)
netstat -st
IcmpMsg:
InType0: 622204
InType3: 6052
InType8: 746391
InType11: 23
InType13: 3
InType15: 3
InType17: 3
OutType0: 745713
OutType3: 5401
OutType8: 622447
OutType11: 57
OutType14: 3
Tcp:
3519494 active connections openings
761050 passive connection openings
2016541 failed connection attempts
451582 connection resets received
291 connections established
565428806 segments received
582593439 segments send out
2846212 segments retransmited
10 bad segments received.
1231155 resets sent
InCsumErrors: 0
UdpLite:
InDatagrams: 0
NoPorts: 0
InErrors: 0
OutDatagrams: 0
RcvbufErrors: 0
SndbufErrors: 0
InCsumErrors: 0
IgnoredMulti: 0
TcpExt:
1 packets pruned from receive queue because of socket buffer overrun
35 ICMP packets dropped because they were out-of-window
ArpFilter: 0
217878 TCP sockets finished time wait in fast timer
4115511 delayed acks sent
1135 delayed acks further delayed because of locked socket
Quick ack mode was activated 157740 times
4919622 packets directly queued to recvmsg prequeue.
2106470 packets directly received from backlog
728738337 packets directly received from prequeue
260056903 packets header predicted
1756921 packets header predicted and directly queued to user
TCPPureAcks: 12917450
TCPHPAcks: 256486617
TCPRenoRecovery: 0
TCPSackRecovery: 21917
TCPSACKReneging: 0
TCPFACKReorder: 36
TCPSACKReorder: 0
TCPRenoReorder: 0
TCPTSReorder: 1
TCPFullUndo: 4
TCPPartialUndo: 0
TCPDSACKUndo: 1056
TCPLossUndo: 978
TCPLostRetransmit: 1401
TCPRenoFailures: 0
TCPSackFailures: 421
TCPLossFailures: 7
TCPFastRetrans: 46262
TCPForwardRetrans: 1889
TCPSlowStartRetrans: 1309
TCPTimeouts: 1915
TCPLossProbes: 2790807
TCPLossProbeRecovery: 3514
TCPRenoRecoveryFail: 0
TCPSackRecoveryFail: 258
TCPSchedulerFailed: 0
TCPRcvCollapsed: 43
TCPDSACKOldSent: 157742
TCPDSACKOfoSent: 0
TCPDSACKRecv: 2783642
TCPDSACKOfoRecv: 0
TCPAbortOnData: 166874
TCPAbortOnClose: 4068
TCPAbortOnMemory: 0
TCPAbortOnTimeout: 237
TCPAbortOnLinger: 0
TCPAbortFailed: 0
TCPMemoryPressures: 0
TCPSACKDiscard: 0
TCPDSACKIgnoredOld: 4
TCPDSACKIgnoredNoUndo: 2550554
TCPSpuriousRTOs: 0
TCPMD5NotFound: 0
TCPMD5Unexpected: 0
TCPSackShifted: 14373
TCPSackMerged: 25768
TCPSackShiftFallback: 263888
TCPBacklogDrop: 0
TCPMinTTLDrop: 0
TCPDeferAcceptDrop: 0
IPReversePathFilter: 285686
TCPTimeWaitOverflow: 0
TCPReqQFullDoCookies: 0
TCPReqQFullDrop: 0
TCPRetransFail: 0
TCPRcvCoalesce: 1677684
TCPOFOQueue: 5972
TCPOFODrop: 0
TCPOFOMerge: 0
TCPChallengeACK: 11751
TCPSYNChallenge: 10
TCPFastOpenActive: 0
TCPFastOpenActiveFail: 0
TCPFastOpenPassive: 0
TCPFastOpenPassiveFail: 0
TCPFastOpenListenOverflow: 0
TCPFastOpenCookieReqd: 0
TCPSpuriousRtxHostQueues: 57
BusyPollRxPackets: 0
TCPAutoCorking: 229104566
TCPFromZeroWindowAdv: 263
TCPToZeroWindowAdv: 263
TCPWantZeroWindowAdv: 185
TCPSynRetrans: 2858
TCPOrigDataSent: 542987167
TCPHystartTrainDetect: 432795
TCPHystartTrainCwnd: 8644006
TCPHystartDelayDetect: 12
TCPHystartDelayCwnd: 225
TCPACKSkippedSynRecv: 0
TCPACKSkippedPAWS: 0
TCPACKSkippedSeq: 7
TCPACKSkippedFinWait2: 0
TCPACKSkippedTimeWait: 0
TCPACKSkippedChallenge: 0
TCPWinProbe: 81
TCPKeepAlive: 533011
TCPMTUPFail: 0
TCPMTUPSuccess: 0
IpExt:
InNoRoutes: 0
InTruncatedPkts: 0
InMcastPkts: 3618291
OutMcastPkts: 811102
InBcastPkts: 13248
OutBcastPkts: 31
InOctets: 3464926929670
OutOctets: 2631875144172
InMcastOctets: 580452056
OutMcastOctets: 42202479
InBcastOctets: 4579903
OutBcastOctets: 12340
InCsumErrors: 0
InNoECTPkts: 3072086529
InECT1Pkts: 0
InECT0Pkts: 0
InCEPkts: 0
===== AD =====
==== winbind ====
wbinfo -p
Ping to winbindd succeeded
wbinfo -P
checking the NETLOGON for domain[IDST] dc connection to "N4363022.idst.ibaintern.de" succeeded
==== AD-abfrage id gegen alle user ====
cd /home/BADA/
for X in *; do id BADA\\$X; done;
uid=409768(BADA\ax) gid=04402(BADA\do) groups=409768(BADA\ax),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),50061(BADA\z9),71422(BADA\dsl),42766(BADA\z9),06508(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),011016(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=85757(BADA\ba) gid=04402(BADA\do) groups=85757(BADA\ba),04402(BADA\do),78915(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),78914(BADA\l9)
uid=461093(BADA\ba) gid=04402(BADA\do) groups=461093(BADA\ba),04402(BADA\do),497671(BADA\z9),78915(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),78914(BADA\l9)
uid=471050(BADA\ba) gid=04402(BADA\do) groups=471050(BADA\ba),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),42766(BADA\z9),497827(BADA\z9),06508(BADA\z9),497742(BADA\z9),497768(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),019314(BADA\l9),011016(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),42765(BADA\l9)
uid=226129(BADA\bo) gid=04402(BADA\do) groups=226129(BADA\bo),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),50061(BADA\z9),42766(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=461970(BADA\bo) gid=04402(BADA\do) groups=461970(BADA\bo),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),413441(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=461276(BADA\br) gid=04402(BADA\do) groups=461276(BADA\br),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),497827(BADA\z9),413441(BADA\z9),06508(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),011016(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=468484(BADA\ch) gid=04402(BADA\do) groups=468484(BADA\ch),04402(BADA\do),497893(BADA\z9),77856(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),77855(BADA\l9)
uid=28157(BADA\co) gid=04402(BADA\do) groups=28157(BADA\co),04402(BADA\do),387321(BADA\z9),06500(BADA\z9),78915(BADA\z9),387329(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),06501(BADA\l9),28253(BADA\l9),06597(BADA\l9),387320(BADA\l9),78914(BADA\l9),387318(BADA\l9)
uid=461042(BADA\co) gid=04402(BADA\do) groups=461042(BADA\co),04402(BADA\do),497844(BADA\z9),497671(BADA\z9),06500(BADA\z9),78915(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),06501(BADA\l9),28253(BADA\l9),06597(BADA\l9),78914(BADA\l9)
uid=460539(BADA\da) gid=04402(BADA\do) groups=460539(BADA\da),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),497827(BADA\z9),413441(BADA\z9),06508(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),011016(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=89047(BADA\di) gid=04402(BADA\do) groups=89047(BADA\di),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),50061(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=006856(BADA\en) gid=04402(BADA\do) groups=006856(BADA\en),04402(BADA\do),78915(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),78914(BADA\l9)
uid=461280(BADA\en) gid=04402(BADA\do) groups=461280(BADA\en),04402(BADA\do),497671(BADA\z9),78915(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),78914(BADA\l9)
uid=85723(BADA\ga) gid=04402(BADA\do) groups=85723(BADA\ga),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),50061(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=461942(BADA\ga) gid=04402(BADA\do) groups=461942(BADA\ga),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),413441(BADA\z9),497742(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=461131(BADA\he) gid=04402(BADA\do) groups=461131(BADA\he),04402(BADA\do),497778(BADA\z),06900(BADA\z9),065357(BADA\z9),50061(BADA\z9),42766(BADA\z9),497686(BADA\z9),497827(BADA\z9),06508(BADA\z9),497742(BADA\z9),497768(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),78907(BADA\z9),498933(BADA\z9),497746(BADA\z9),47231(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),019314(BADA\l9),011016(BADA\l9),065356(BADA\l9),78906(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),47230(BADA\l9),42765(BADA\l9)
uid=063520(BADA\he) gid=04402(BADA\do) groups=063520(BADA\he),04402(BADA\do),41663(BADA\l9),019271(BADA\l9),77855(BADA\l9),011016(BADA\l9),06597(BADA\l9),28253(BADA\l9),019273(BADA\l9),76053(BADA\l9),71422(BADA\dsl),41664(BADA\z9),019272(BADA\z9),06596(BADA\z9),77856(BADA\z9),019274(BADA\z9)
uid=006852(BADA\ho) gid=04402(BADA\do) groups=006852(BADA\ho),04402(BADA\do),78914(BADA\l9),06597(BADA\l9),28253(BADA\l9),76053(BADA\l9),71422(BADA\dsl),06596(BADA\z9),78915(BADA\z9)
uid=468680(BADA\kr) gid=04402(BADA\do) groups=468680(BADA\kr),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),497827(BADA\z9),413441(BADA\z9),06508(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),011016(BADA\l9),28063(BADA\l9),50070(BADA\l9),065087(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=06680(BADA\lu) gid=04402(BADA\do) groups=06680(BADA\lu),04402(BADA\do),387318(BADA\l9),50070(BADA\l9),28063(BADA\l9),019314(BADA\l9),387320(BADA\l9),06597(BADA\l9),28253(BADA\l9),06901(BADA\l9),78980(BADA\l9),011487(BADA\l9),50060(BADA\l9),76053(BADA\l9),71422(BADA\dsl),78981(BADA\z9),06596(BADA\z9),28064(BADA\z9),387329(BADA\z9),01148(BADA\z9),50061(BADA\z9),387321(BADA\z9),06900(BADA\z9)
uid=76375(BADA\mi) gid=04402(BADA\do) groups=76375(BADA\mi),04402(BADA\do),06512(BADA\z9),065049(BADA\z9),71422(BADA\dsl),019274(BADA\z9),01148(BADA\z9),011015(BADA\z9),77856(BADA\z9),06596(BADA\z9),065041(BADA\z9),019272(BADA\z9),76053(BADA\l9),11339,19597(IB\vi),019273(BADA\l9),011487(BADA\l9),065040(BADA\l9),28253(BADA\l9),011014(BADA\l9),06597(BADA\l9),065038(BADA\l9),011016(BADA\l9),06513(BADA\l9),77855(BADA\l9),019271(BADA\l9)
id: ‘BADA\\mo’: no such user
uid=298526(BADA\ni) gid=04402(BADA\do) groups=298526(BADA\ni),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),50061(BADA\z9),42766(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),78907(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),78906(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=461234(BADA\ni) gid=04402(BADA\do) groups=461234(BADA\ni),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),497686(BADA\z9),413441(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),78907(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),78906(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=468663(BADA\pa) gid=04402(BADA\do) groups=468663(BADA\pa),04402(BADA\do),50061(BADA\z9),497827(BADA\z9),06508(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),28253(BADA\l9),06597(BADA\l9),019314(BADA\l9),011016(BADA\l9),50070(BADA\l9),065087(BADA\l9)
id: ‘BADA\\pa’: no such user
uid=231922(BADA\pa) gid=04402(BADA\do) groups=231922(BADA\pa),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),42766(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),42765(BADA\l9)
uid=461078(BADA\pa) gid=04402(BADA\do) groups=461078(BADA\pa),04402(BADA\do),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),413441(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=06681(BADA\re) gid=04402(BADA\do) groups=06681(BADA\re),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),71422(BADA\dsl),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=461086(BADA\re) gid=04402(BADA\do) groups=461086(BADA\re),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),413441(BADA\z9),497742(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=434520(BADA\sc) gid=04402(BADA\do) groups=434520(BADA\sc),04402(BADA\do),77480(BADA\z9),434596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),442071(BADA\l9),442073(BADA\l9)
uid=006853(BADA\sc) gid=04402(BADA\do) groups=006853(BADA\sc),04402(BADA\do),78914(BADA\l9),06597(BADA\l9),28253(BADA\l9),76053(BADA\l9),71422(BADA\dsl),06596(BADA\z9),78915(BADA\z9)
uid=006855(BADA\sc) gid=04402(BADA\do) groups=006855(BADA\sc),04402(BADA\do),78914(BADA\l9),06597(BADA\l9),28253(BADA\l9),76053(BADA\l9),71422(BADA\dsl),06596(BADA\z9),78915(BADA\z9)
uid=461308(BADA\si) gid=04402(BADA\do) groups=461308(BADA\si),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),413441(BADA\z9),497742(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9)
uid=06684(BADA\si) gid=04402(BADA\do) groups=06684(BADA\si),04402(BADA\do),06900(BADA\z9),50061(BADA\z9),42766(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),019314(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
uid=89048(BADA\si) gid=04402(BADA\do) groups=89048(BADA\si),04402(BADA\do),06900(BADA\z9),06512(BADA\z9),50061(BADA\z9),71422(BADA\dsl),28064(BADA\z9),06596(BADA\z9),78981(BADA\z9),76053(BADA\l9),11339,19597(IB\vi),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),06597(BADA\l9),019314(BADA\l9),011016(BADA\l9),06513(BADA\l9),28063(BADA\l9),50070(BADA\l9)
uid=006857(BADA\so) gid=04402(BADA\do) groups=006857(BADA\so),04402(BADA\do),387318(BADA\l9),78914(BADA\l9),387320(BADA\l9),06597(BADA\l9),28253(BADA\l9),76053(BADA\l9),71422(BADA\dsl),06596(BADA\z9),387329(BADA\z9),78915(BADA\z9),387321(BADA\z9)
uid=461460(BADA\so) gid=04402(BADA\do) groups=461460(BADA\so),04402(BADA\do),387321(BADA\z9),413438(BADA\z9),413441(BADA\z9),497671(BADA\z9),78915(BADA\z9),387329(BADA\z9),06596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),06597(BADA\l9),387320(BADA\l9),78914(BADA\l9),387318(BADA\l9)
uid=404600(BADA\uf) gid=04402(BADA\do) groups=404600(BADA\uf),04402(BADA\do),011800(BADA\z0),71422(BADA\dsl),380450(BADA\z9)
uid=434521(BADA\vo) gid=04402(BADA\do) groups=434521(BADA\vo),04402(BADA\do),77480(BADA\z9),434596(BADA\z9),71422(BADA\dsl),76053(BADA\l9),28253(BADA\l9),442071(BADA\l9),442073(BADA\l9)
uid=461334(BADA\za) gid=04402(BADA\do) groups=461334(BADA\za),04402(BADA\do),434639(BADA\z9),06900(BADA\z9),387321(BADA\z9),413438(BADA\z9),50061(BADA\z9),42766(BADA\z9),413441(BADA\z9),497742(BADA\z9),497768(BADA\z9),387329(BADA\z9),28064(BADA\z9),06596(BADA\z9),497702(BADA\z9),78981(BADA\z9),497746(BADA\z9),71422(BADA\dsl),76053(BADA\l9),50060(BADA\l9),78980(BADA\l9),06901(BADA\l9),28253(BADA\l9),019318(BADA\l9),06597(BADA\l9),387320(BADA\l9),019314(BADA\l9),442190(BADA\l9),28063(BADA\l9),50070(BADA\l9),387318(BADA\l9),42765(BADA\l9)
===== Hardware =====
==== hplog ====
hplog -v
ID Severity Initial Time Update Time Count
-------------------------------------------------------------
0000 Information 14:37 12/11/2012 14:37 12/11/2012 0001
LOG: Maintenance note: IML Cleared (IML Log cleared by the
0001 Caution 09:08 02/08/2013 09:08 02/08/2013 0001
LOG: POST Error: 1704-Slot X Drive Array - Array Accelerator Super-Cap is charging. The Array Accelerator Cache will be enabled once Super-Cap has been charged. No action is required.
0002 Repaired 08:13 02/11/2013 05:14 04/15/2018 0015
LOG: Network Adapter Link Down (Slot 3, Port 2)
0003 Repaired 00:12 03/08/2013 05:14 04/15/2018 0012
LOG: Network Adapter Link Down (Slot 3, Port 1)
0004 Repaired 12:46 06/20/2013 20:59 12/27/2016 0006
LOG: Network Adapters Redundancy Reduced (Slot 0, Port 4)
0005 Repaired 12:46 06/20/2013 05:17 04/15/2018 0008
LOG: Network Adapter Link Down (Slot 2, Port 2)
0006 Repaired 12:46 06/20/2013 05:17 04/15/2018 0008
LOG: Network Adapter Link Down (Slot 2, Port 1)
0007 Repaired 13:41 06/21/2013 05:19 04/15/2018 0006
LOG: Network Adapter Link Down (Slot 0, Port 2)
0008 Repaired 13:42 06/21/2013 20:40 12/27/2016 0004
LOG: Network Adapters Redundancy Reduced (Slot 0, Port 3)
0009 Repaired 13:42 06/21/2013 05:19 04/15/2018 0006
LOG: Network Adapters Redundancy Reduced (Slot 0, Port 1)
0010 Caution 06:09 06/17/2014 06:09 06/17/2014 0001
LOG: POST Error: 1792-Drive Array Reports Valid Data Found in Array Accelerator
0011 Repaired 14:38 11/08/2016 15:20 11/08/2016 0001
LOG: System Power Supply: General Failure (Power Supply 1)
0012 Repaired 14:38 11/08/2016 15:20 11/08/2016 0001
LOG: System Power Supplies Not Redundant
0013 Critical 11:35 03/21/2017 11:35 03/21/2017 0001
LOG: ASR Detected by System ROM
0014 Repaired 05:43 10/22/2017 13:57 04/15/2018 0002
LOG: Internal Storage Enclosure Device Failure (Bay 2, Box 1, Port 1I, Slot 0)
0015 Caution 18:59 11/30/2017 18:59 11/30/2017 0001
LOG: POST Error: 1786-Drive Array Recovery Needed
0016 Critical 10:50 01/31/2018 10:50 01/31/2018 0001
LOG: Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00000000)
0017 Caution 11:12 01/31/2018 11:12 01/31/2018 0001
LOG: POST Error: 1720-S.M.A.R.T. Hard Drive Detects Imminent Failure
0018 Critical 09:46 02/01/2018 09:46 02/01/2018 0001
LOG: Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00000000)
0019 Caution 10:01 02/01/2018 10:01 02/01/2018 0001
LOG: POST Error: 1720-S.M.A.R.T. Hard Drive Detects Imminent Failure
0020 Critical 15:51 04/13/2018 15:51 04/13/2018 0001
LOG: Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00000000)
0021 Caution 00:00 00/00/0000 00:00 00/00/0000 0001
LOG: System Power Supply: General Failure (Power Supply 1)
0022 Caution 05:09 04/15/2018 05:09 04/15/2018 0001
LOG: POST Error: 1615-Power Supply Failure or Power Supply Unplugged in Bay 1
0023 Caution 05:10 04/15/2018 05:10 04/15/2018 0001
LOG: POST Error: 1720-S.M.A.R.T. Hard Drive Detects Imminent Failure
0024 Critical 12:37 07/18/2018 12:37 07/18/2018 0001
LOG: Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00000000)
==== hpasmcli ====
hpasmcli
HPE management CLI for Linux (v2.0)
Copyright 2015 Hewlett Packard Enterprise Development LP.
--------------------------------------------------------------------------
NOTE: Some hpasmcli commands may not be supported on all Proliant servers.
Type 'help' to get a list of all top level commands.
--------------------------------------------------------------------------
hpasmcli> show dimm
DIMM Configuration
------------------
Processor #: 1
Module #: 1
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 2
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 3
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 4
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 5
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 6
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 7
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 1
Module #: 8
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 1
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 2
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 3
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 4
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 5
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 6
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 7
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
Processor #: 2
Module #: 8
Present: Yes
Form Factor: 9h
Memory Type: DDR3(18h)
Size: 16384 MB
Speed: 1333 MHz
Supports Lock Step: No
Configured for Lock Step: No
Status: Ok
hpasmcli> quit
----
~~DISCUSSION~~