Advisory: Support Portal Maintenance. Login is currently unavailable, more info available here.
Hi
We have a Zabbix server running
Red Hat Enterprise Linux release 8.2 (Ootpa)
16gb Memory
4gb swap
and
Sophos Anti-Virus = 10.5.2Build Revision = 2840690
We get a lot of these messages in the messages file?
Jan 13 04:44:28 ireprdzbxapp01 kernel: talpa-deny: Timeout occurred while closing /var/log/zabbix/zabbix_agentd.log on behalf of process zabbix_agentd[518545/518545] owned by 993(993)/990(990) <62>Jan 13 04:45:22 ireprdzbxapp01 kernel: talpa-deny: Timeout occurred while closing /var/log/zabbix/zabbix_server.log on behalf of process zabbix_server[2256/2256] owned by 993(993)/990(990) <62>Jan 13 04:47:08 ireprdzbxapp01 kernel: traps: mysqld[1716] general protection fault ip:55d354863b34 sp:7fce98f08d18 error:0 in mysqld[55d353c1a000+11d5000]Jan 13 04:47:11 ireprdzbxapp01 systemd[1]: Started Process Core Dump (PID 518569/UID 0).Jan 13 04:47:12 ireprdzbxapp01 kernel: talpa-vfshook: Patching hugetlbfsJan 13 04:47:12 ireprdzbxapp01 kernel: talpa-vfshook: Patched hugetlbfsJan 13 04:47:35 ireprdzbxapp01 kernel: mtdd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0Jan 13 04:47:41 ireprdzbxapp01 kernel: mtdd cpuset=/ mems_allowed=0Jan 13 04:47:41 ireprdzbxapp01 kernel: CPU: 2 PID: 488355 Comm: mtdd Kdump: loaded Tainted: G OE --------- - - 4.18.0-193.19.1.el8_2.x86_64 #1Jan 13 04:47:41 ireprdzbxapp01 kernel: Hardware name: Amazon EC2 m5.xlarge/, BIOS 1.0 10/16/2017Jan 13 04:47:41 ireprdzbxapp01 kernel: Call Trace:Jan 13 04:47:41 ireprdzbxapp01 kernel: dump_stack+0x5c/0x80Jan 13 04:47:41 ireprdzbxapp01 kernel: dump_header+0x6e/0x27aJan 13 04:47:41 ireprdzbxapp01 kernel: ? try_to_free_pages+0xe8/0x1c0Jan 13 04:47:41 ireprdzbxapp01 kernel: oom_kill_process.cold.28+0xb/0x10Jan 13 04:47:41 ireprdzbxapp01 kernel: out_of_memory+0x1ba/0x490Jan 13 04:47:41 ireprdzbxapp01 kernel: __alloc_pages_slowpath+0xc40/0xd60Jan 13 04:47:41 ireprdzbxapp01 kernel: ? avc_has_perm_noaudit+0x6c/0x140Jan 13 04:47:41 ireprdzbxapp01 kernel: __alloc_pages_nodemask+0x245/0x280Jan 13 04:47:41 ireprdzbxapp01 kernel: __get_free_pages+0xa/0x30Jan 13 04:47:41 ireprdzbxapp01 kernel: inode_doinit_with_dentry+0x266/0x480Jan 13 04:47:41 ireprdzbxapp01 kernel: security_d_instantiate+0x2f/0x50Jan 13 04:47:41 ireprdzbxapp01 kernel: d_splice_alias+0x4c/0x3c0Jan 13 04:47:41 ireprdzbxapp01 kernel: proc_tgid_net_lookup+0x39/0x70Jan 13 04:47:41 ireprdzbxapp01 kernel: path_openat+0x852/0x14d0Jan 13 04:47:41 ireprdzbxapp01 kernel: ? __switch_to_asm+0x41/0x70Jan 13 04:47:41 ireprdzbxapp01 kernel: ? __switch_to_asm+0x35/0x70Jan 13 04:47:41 ireprdzbxapp01 kernel: do_filp_open+0x93/0x100Jan 13 04:47:41 ireprdzbxapp01 kernel: ? unuse_pde+0x20/0x20Jan 13 04:47:41 ireprdzbxapp01 kernel: ? simple_attr_release+0x20/0x20Jan 13 04:47:41 ireprdzbxapp01 kernel: do_sys_open+0x184/0x220Jan 13 04:47:41 ireprdzbxapp01 kernel: do_syscall_64+0x5b/0x1a0Jan 13 04:47:41 ireprdzbxapp01 kernel: entry_SYSCALL_64_after_hwframe+0x65/0xcaJan 13 04:47:41 ireprdzbxapp01 kernel: RIP: 0033:0x7f56b5fea861Jan 13 04:47:41 ireprdzbxapp01 kernel: Code: 89 54 24 08 e8 80 cd 01 00 8b 74 24 0c 48 8b 3c 24 41 89 c0 44 8b 54 24 08 b8 01 01 00 00 89 f2 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 77 2d 44 89 c7 89 44 24 08 e8 ab cd 01 00 8b 44Jan 13 04:47:41 ireprdzbxapp01 kernel: RSP: 002b:00007f56af7fd380 EFLAGS: 00000293 ORIG_RAX: 0000000000000101Jan 13 04:47:41 ireprdzbxapp01 kernel: RAX: ffffffffffffffda RBX: 00007f56a04e02a0 RCX: 00007f56b5fea861Jan 13 04:47:41 ireprdzbxapp01 kernel: RDX: 0000000000000000 RSI: 00007f56bbb6f94e RDI: 00000000ffffff9cJan 13 04:47:41 ireprdzbxapp01 kernel: RBP: 0000000000000008 R08: 0000000000000000 R09: 0000000000000001Jan 13 04:47:41 ireprdzbxapp01 kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 00007f56b69a1bf5Jan 13 04:47:41 ireprdzbxapp01 kernel: R13: 00007f56b69a1bf5 R14: 0000000000000001 R15: 00007f53478fd164
When we switch sophos off, like we did over the weekend, the server was fine and didn;t send out a flood of zabbix alerts.
Upon turning Sophos back on we are seeing the same regular issues.
I have setup a cron job to kill mtdd every 6 hours as well.
Please advise.
Tariq
Hello Tariq,
apparently there's an out-of-memory condition.I assume MTD is enabled - if it's indeed mtdd you shouldn't encounter this problem if you turn MTD off. Have you checked the mtdd_.log?
Christian
Hi Christian
I'm assuming mtdd is enabled. Can it be disabled? Is it wise to disable it?And what should i be looking for in particular in the mtdd....log?
Thank you
MTD can be enabled or disabled within the Threat Protection Policy settings. It offers additional protection but it is not necessarily essential.
what should i be looking forI can't say, anything abnormal should stand out. Might be necessary to enable verbose logging.