TransWikia.com

kern.log and syslog grow 40+ GB in few minutes

elementary OS Asked by Surendrajat on December 18, 2021

I’ve read most of the answers and nothing really seems to be related to case except force shut down and cleaning up kern.log and syslog files manually on next boot. Happened a few times already. I’ve tried to take a sample of 20000 lines from kern.log(which is almost 40GB) and uploaded here. While running below script:

for log in test;  
  echo "${log} :"
  sed -e 's/[[^]]+]//' -e 's/.*[0-9]{2}:[0-9]{2}:[0-9]{2}//' ${log} 
  | sort | uniq -c | sort -hr | head -10
done

I get the following lines:

 559  npc kernel:   worker_thread+0x34/0x410
    559  npc kernel:  wlp5s0:  Failed check-sdata-in-driver check, flags: 0x4
    559  npc kernel:  WARNING: CPU: 3 PID: 19976 at /build/linux-hwe-pGQy9q/linux-hwe-5.3.0/net/mac80211/driver-ops.h:19 drv_sta_state+0x165/0x400 [mac80211]
    559  npc kernel:   sta_info_move_state+0x284/0x360 [mac80211]
    559  npc kernel:   __sta_info_flush+0x139/0x1b0 [mac80211]
    559  npc kernel:   __sta_info_destroy_part2+0x36/0x1a0 [mac80211]
    559  npc kernel:   ret_from_fork+0x35/0x40
    559  npc kernel:   ? _raw_spin_unlock_bh+0x1e/0x20
    559  npc kernel:   ? process_one_work+0x3f0/0x3f0
    559  npc kernel:   process_one_work+0x1fd/0x3f0

My system details:

OS: elementary OS Hera 5.1.5
CPU: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
DISK: 150GB SSD (partition-dual boot w/ windows10)

Additional WLAN related hwinfo, which could be relevant:

*-network
                description: Wireless interface
                product: Wireless 8265 / 8275
                vendor: Intel Corporation
                physical id: 0
                bus info: pci@0000:05:00.0
                logical name: wlp5s0
                version: 78
                serial: 48:a4:72:xx:xx:xx
                width: 64 bits
                clock: 33MHz
                capabilities: bus_master cap_list ethernet physical wireless
                configuration: broadcast=yes driver=iwlwifi driverversion=5.3.0-59-generic firmware=36.77d01142.0 ip=192.168.43.85 latency=0 link=yes multicast=yes wireless=IEEE 802.11
                resources: irq:132 memory:f1000000-f1001fff

When it happens, suddenly the CPU is at 30-40%, and the next thing I know that no space on deceive left.

4 Answers

By the way the bash script is wrong. Should be:

#!/bin/bash
for log in test; do
  echo "${log} :"
  sed -e 's/[[^]]+]//' -e 's/.*[0-9]{2}:[0-9]{2}:[0-9]{2}//' ${log} 
  | sort | uniq -c | sort -hr | head -10
done

I tested it, but have no idea what test in your case is.

Answered by Sysadmin on December 18, 2021

List below file:

cat /etc/logrotate.d/rsyslog

should contain this:

/var/log/syslog
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
            /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
    rotate 4
    weekly
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
            /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

If it does not contain the above , edit it and replace

sudo nano /etc/logrotate.d/rsyslog

Then force the logrotate manually:

sudo logrotate --force  /etc/logrotate.d/rsyslog

If it will show similar error:

error: skipping "/var/log/syslog" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.

Then type in terminal:

sudo chmod 755 /var/log/ && sudo chown root:root /var/log/
sudo chmod 755 /var && sudo chown root:root /var
sudo logrotate --force  /etc/logrotate.d/rsyslog

Answered by Sysadmin on December 18, 2021

How to upgrade Elementary OS to the 5.7.0 kernel.

Install necessary packages

sudo apt-get install build-essential libncurses-dev bison flex libssl-dev libelf-dev

To check hardware

sudo lspci

To check missing firmware

sudo update-initramfs -u

This step is important, because it shows missing firmware.

Make directories for firmware

mkdir /home/username/Downloads/kernel
mkdir /home/username/Downloads/amdgpu
mkdir /home/username/Downloads/i915
mkdir /home/username/Downloads/rtl

To get rid of missing firmware errors for amdgpu

cd /home/username/Downloads/amdgpu
wget https://github.com/M-Bab/linux-kernel-amdgpu-binaries/files/4124959/firmware-radeon-ucode-rock_2.110_all.deb.zip
unzip firmware-radeon-ucode-rock_2.110_all.deb.zip
sudo dpkg -i firmware-radeon-ucode-rock_2.110_all.deb

Or use eddy to install this deb

Source: https://github.com/M-Bab/linux-kernel-amdgpu-binaries/issues/82

To get rid of missing firmware errors for i915

cd /home/username/Downloads/i915
git clone https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
sudo cp -R /linux-firmware/i915/* /lib/firmware/i915/

To get rid of missing firmware errors for rtl (Realtek LAN and Wi-Fi adapters in my DELL)

cd /home/username/Downloads/rtl
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/rtl_nic/rtl8168fp-3.fw
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/rtl_nic/rtl8125a-3.fw
sudo cp rtl8168fp-3.fw /lib/firmware/rtl_nic 
sudo cp rtl8125a-3.fw /lib/firmware/rtl_nic

Download kernel 5.7.0

cd /home/username/Downloads/kernel
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.7.tar.xz
tar -xf linux-5.7.tar.xz
cd linux-5.7
cp -v /boot/config-$(uname -r) .config
make menuconfig 

save config and then exit.

Use 4 core/thread - if your processor have 4 cores

make -j 4

Use 8 core/thread - if your processor have 8 cores

make -j 8

Then type below commands:

sudo make modules_install 
sudo make install 
sudo update-initramfs -c -k 5.7.0

ignore these errors below, just because those three bin files are not made yet W: Possible missing firmware /lib/firmware/amdgpu/navi12_ta.bin for module amdgpu W: Possible missing firmware /lib/firmware/amdgpu/navi10_mes.bin for module amdgpu W: Possible missing firmware /lib/firmware/amdgpu/navi12_dmcu.bin for module amdgpu

Type in terminal:

sudo update-grub
sudo reboot
uname -mrs

To check errors after reboot:

sudo journalctl -b
sudo dmesg

Answered by Sysadmin on December 18, 2021

It looks like you are using hwe kernel. Can you remove it together with headers and then update the grub and also initramfs?

Answered by Sysadmin on December 18, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP