Unix & Linux Asked by althor on October 31, 2021
The system has killed a process due to "Out of memory" but I cannot understand these messages.
I am not able to find the memory issue.
[Mon Jul 20 21:20:39 2020] crawler invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0
[Mon Jul 20 21:20:39 2020] crawler cpuset=/ mems_allowed=0
[Mon Jul 20 21:20:39 2020] CPU: 0 PID: 24357 Comm: crawler Not tainted 4.4.0-1098-aws #109-Ubuntu
[Mon Jul 20 21:20:39 2020] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
[Mon Jul 20 21:20:39 2020] 0000000000000286 1f6ee74fe469971b ffff8800978779e0 ffffffff81408801
[Mon Jul 20 21:20:39 2020] ffff880097877b98 ffff880204a5ee00 ffff880097877a50 ffffffff81216217
[Mon Jul 20 21:20:39 2020] 0000000000000000 ffff8800e2e2e9c0 ffff8801148cc4c0 ffff880097877a38
[Mon Jul 20 21:20:39 2020] Call Trace:
[Mon Jul 20 21:20:39 2020] [<ffffffff81408801>] dump_stack+0x63/0x82
[Mon Jul 20 21:20:39 2020] [<ffffffff81216217>] dump_header+0x5a/0x1c3
[Mon Jul 20 21:20:39 2020] [<ffffffff813a04e1>] ? apparmor_capable+0x131/0x1b0
[Mon Jul 20 21:20:39 2020] [<ffffffff8119a94b>] oom_kill_process+0x20b/0x3d0
[Mon Jul 20 21:20:39 2020] [<ffffffff8119ad58>] out_of_memory+0x1f8/0x460
[Mon Jul 20 21:20:39 2020] [<ffffffff811a0db3>] __alloc_pages_slowpath.constprop.89+0x943/0xaf0
[Mon Jul 20 21:20:39 2020] [<ffffffff811a11ff>] __alloc_pages_nodemask+0x29f/0x2b0
[Mon Jul 20 21:20:39 2020] [<ffffffff811ec7cc>] alloc_pages_current+0x8c/0x110
[Mon Jul 20 21:20:39 2020] [<ffffffff81196a8b>] __page_cache_alloc+0xab/0xc0
[Mon Jul 20 21:20:39 2020] [<ffffffff811993b0>] filemap_fault+0x160/0x440
[Mon Jul 20 21:20:39 2020] [<ffffffff812af5c6>] ext4_filemap_fault+0x36/0x50
[Mon Jul 20 21:20:39 2020] [<ffffffff811c6b87>] __do_fault+0x77/0x110
[Mon Jul 20 21:20:39 2020] [<ffffffff811cab29>] handle_mm_fault+0x1259/0x1b80
[Mon Jul 20 21:20:39 2020] [<ffffffff8183ee61>] ? __schedule+0x301/0x810
[Mon Jul 20 21:20:39 2020] [<ffffffff8183ee6d>] ? __schedule+0x30d/0x810
[Mon Jul 20 21:20:39 2020] [<ffffffff8183ee61>] ? __schedule+0x301/0x810
[Mon Jul 20 21:20:39 2020] [<ffffffff8183ee6d>] ? __schedule+0x30d/0x810
[Mon Jul 20 21:20:39 2020] [<ffffffff8183ee61>] ? __schedule+0x301/0x810
[Mon Jul 20 21:20:39 2020] [<ffffffff8106de74>] __do_page_fault+0x1a4/0x410
[Mon Jul 20 21:20:39 2020] [<ffffffff810fa96c>] ? ktime_get_ts64+0x4c/0x100
[Mon Jul 20 21:20:39 2020] [<ffffffff8106e102>] do_page_fault+0x22/0x30
[Mon Jul 20 21:20:39 2020] [<ffffffff81846938>] page_fault+0x28/0x30
[Mon Jul 20 21:20:39 2020] Mem-Info:
[Mon Jul 20 21:20:39 2020] active_anon:1971173 inactive_anon:13648 isolated_anon:0
active_file:0 inactive_file:21 isolated_file:0
unevictable:913 dirty:0 writeback:0 unstable:0
slab_reclaimable:7341 slab_unreclaimable:9130
mapped:1013 shmem:20671 pagetables:5035 bounce:0
free:25558 free_pcp:31 free_cma:0
[Mon Jul 20 21:20:39 2020] Node 0 DMA free:15900kB min:128kB low:160kB high:192kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[Mon Jul 20 21:20:39 2020] lowmem_reserve[]: 0 3729 7951 7951 7951
[Mon Jul 20 21:20:39 2020] Node 0 DMA32 free:48524kB min:31640kB low:39548kB high:47460kB active_anon:3694868kB inactive_anon:31728kB active_file:28kB inactive_file:56kB unevictable:1984kB isolated(anon):0kB isolated(file):0kB present:3915776kB managed:3835212kB mlocked:1984kB dirty:0kB writeback:0kB mapped:2068kB shmem:40836kB slab_reclaimable:11392kB slab_unreclaimable:17676kB kernel_stack:1488kB pagetables:9948kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:552 all_unreclaimable? yes
[Mon Jul 20 21:20:39 2020] lowmem_reserve[]: 0 0 4221 4221 4221
[Mon Jul 20 21:20:39 2020] Node 0 Normal free:37808kB min:35808kB low:44760kB high:53712kB active_anon:4189824kB inactive_anon:22864kB active_file:0kB inactive_file:28kB unevictable:1668kB isolated(anon):0kB isolated(file):0kB present:4456448kB managed:4322624kB mlocked:1668kB dirty:0kB writeback:0kB mapped:1984kB shmem:41848kB slab_reclaimable:17972kB slab_unreclaimable:18844kB kernel_stack:1440kB pagetables:10192kB unstable:0kB bounce:0kB free_pcp:124kB local_pcp:4kB free_cma:0kB writeback_tmp:0kB pages_scanned:212 all_unreclaimable? yes
[Mon Jul 20 21:20:39 2020] lowmem_reserve[]: 0 0 0 0 0
[Mon Jul 20 21:20:39 2020] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15900kB
[Mon Jul 20 21:20:39 2020] Node 0 DMA32: 358*4kB (UME) 132*8kB (UME) 281*16kB (UME) 128*32kB (UME) 98*64kB (UME) 30*128kB (UME) 23*256kB (E) 14*512kB (ME) 8*1024kB (E) 3*2048kB (E) 0*4096kB = 48584kB
[Mon Jul 20 21:20:39 2020] Node 0 Normal: 423*4kB (UME) 127*8kB (UE) 153*16kB (UME) 172*32kB (UE) 103*64kB (UE) 39*128kB (UE) 21*256kB (UE) 10*512kB (ME) 1*1024kB (E) 0*2048kB 1*4096kB (H) = 37860kB
[Mon Jul 20 21:20:39 2020] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[Mon Jul 20 21:20:39 2020] 21375 total pagecache pages
[Mon Jul 20 21:20:39 2020] 0 pages in swap cache
[Mon Jul 20 21:20:39 2020] Swap cache stats: add 0, delete 0, find 0/0
[Mon Jul 20 21:20:39 2020] Free swap = 0kB
[Mon Jul 20 21:20:39 2020] Total swap = 0kB
[Mon Jul 20 21:20:39 2020] 2097053 pages RAM
[Mon Jul 20 21:20:39 2020] 0 pages HighMem/MovableOnly
[Mon Jul 20 21:20:39 2020] 53619 pages reserved
[Mon Jul 20 21:20:39 2020] 0 pages cma reserved
[Mon Jul 20 21:20:39 2020] 0 pages hwpoisoned
[Mon Jul 20 21:20:39 2020] Out of memory: Kill process 24355 (crawler) score 957 or sacrifice child
[Mon Jul 20 21:20:39 2020] Killed process 24355 (crawler) total-vm:9099416kB, anon-rss:7805456kB, file-rss:0kB
Regards
Thanks in advance
The kernel killed:
Killed process 24355 (crawler) total-vm:9099416kB, anon-rss:7805456kB, file-rss:0kB
The process tried to allocate close to 9GB of RAM which is more than your system can handle.
Looks like you have just 2GB of RAM and you've got SWAP disabled. I'd say in advance that having SWAP in a situation like this is unlikely to help at all. If you have tasks which eat that much RAM you must have at the very least 12GB of RAM.
Answered by Artem S. Tashkinov on October 31, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP