Linux Mint: Watchdog configuration

From Edge
Jump to navigation Jump to search

List of tested devices

  • The guide was tested on a systems with the specs listed below, but should be easily adaptable
Device: fitlet3
CPU: Intel(R) Atom(TM) Processor X6425E
BIOS: 05/14/2018 American Megatrends Inc. FLT3.PRD.00.10
OS: Linux Ubuntu 24.04 LTS
Kernel: 6.11.0-26-generic
Kernel module: wdat_wdt

Device: Airtop3
CPU: Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz
BIOS: 10/28/2019 American Megatrends Inc. ATOP3.PRD.0.29.4
OS: Linux Mint 19.2 Tina
Kernel: 4.15.0-54-generic
Kernel module: wdat_wdt

Device: fitlet2
CPU: Intel(R) Atom(TM) Processor E3950
BIOS: 05/14/2018 American Megatrends Inc. FLT2.0.40.01.00
OS: Linux Mint 19 Tara
Kernel: 4.15.0-23-generic
Kernel module: iTCO_wdt

Device: fitlet2
CPU: Intel(R) Atom(TM) Processor E3950
BIOS: 09/17/2018 American Megatrends Inc. FLT2.0.46.01.00
OS: Linux Mint 20 Ulyana
Kernel: 5.4.0-42-generic
Kernel module: iTCO_wdt

Device: fitlet1
CPU: AMD A10 Micro-6700T APU+AMD Radeon R6 Graphics
BIOS: 11/30/2015 American Megatrends Inc. SBCFLT_0.08.08
OS: Linux Mint 19 Tara
Kernel: 4.15.0-23-generic
Kernel module: sp5100_tco

Device: IPC2
CPU: Intel(R) Core(TM) i5-4300U
BIOS: 05/30/2017 Compulab Ltd. IPC2_3.330.5
OS: Linux Mint 19 Tara
Kernel: 4.15.0-23-generic
Kernel module: iTCO_wdt

Device: IPC3
CPU: Intel(R) Core(TM) i5-7200U
BIOS: 08/09/2018 American Megatrends Inc. 5.12_IPC3K.PRD.0.25.7
OS: Linux Mint 19 Tara
Kernel: 4.15.0-23-generic
Kernel module: wdat_wdt


Notes

fitlet1

kernel module i2c_piix4 should be blacklisted Add the following lines to the /etc/modprobe.d/blacklist.conf 

# fitlet1: sp5100_tco conflicts with i2c_piix4 for I/O address 0x0cd6
blacklist i2c_piix4

Then, unload i2c_piix4 and update initramfs 

$ sudo modprobe -r i2c_piix4
$ sudo update-initramfs -u

IPC2-Haswell

iTCO Watchdog does not reset ACPI Watchdog timer

  • if the iTCO BIOS does not reset the ACPI Watchdog timer, you may need to disable the BIOS configuration from Linux
   BIOS --> South bridge --> Watchdog

Get super-user permissions: 

   $ su
   Password: 

   $ apt install ioport
  • Note: On Ubuntu versions below 18.04, ioport should be installed manually from:
   https://packages.ubuntu.com/bionic/ioport


And then run the following command: 

   $ outl 0x1854 0x2000

To read it for validation please run (also as super-user): 

   $ printf "0x%x\n" $(inl 0x1854)
   0x2000

To run this command automatically on each boot add it to file named /etc/rc.local. If you do not have such file, create it. 

   #!/bin/sh -e
   #
   # rc.local
   #
   # This script is executed at the end of each multiuser runlevel.
   # Make sure that the script will "exit 0" on success or any other
   # value on error.
   #
   # In order to enable or disable this script just change the execution
   # bits.
   #
   # By default this script does nothing.
   outl 0x1854 0x2000
   exit 0

Make it executable 

   $ sudo chmod +x /etc/rc.local

Airtop3

Make sure you have correct BIOS settings:

  1. Press [ESC] on startup to enter BIOS settings
  2. Chipset -> PCH-IO Configuration -> Enable TCO Timer is set to [Enabled]
  3. If not, set to [Enabled]
  4. Navigate to "Save & Exit" tab and choose Save changes and exit

fitlet2

Make sure you have correct BIOS settings:

  1. Press [ESC] on startup to enter BIOS settings
  2. At the bottom of the main screen check OS selection is set to [Linux]
  3. If not, set to [Linux]
  4. Navigate to "Save & Exit" tab and choose Save changes and exit.

Usage example

fitlet2

  • Load the kernel module iTCO_wdt
$ sudo modprobe iTCO_wdt
  • Verify the module successfully load
$ dmesg | grep -i iTCO
[   34.741071] iTCO_vendor_support: vendor-support=0
[   34.742749] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
[   34.742824] iTCO_wdt: Found a Apollo Lake SoC TCO device (Version=5, TCOBASE=0x0460)
[   34.742993] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
  • Install watchdog program, it should run in the background automatically on each system start
$ sudo apt install watchdog
  • Modify the following configuration files:
/etc/default/watchdog
change:                watchdog_module="none"
to:                    watchdog_module="iTCO_wdt"

/etc/watchdog.conf
uncomment line:        watchdog-device  = /dev/watchdog
  • NOTE: bugfix required for Linux Mint 18.3
/lib/systemd/system/watchdog.service 
add "WantedBy=default.target" line under [Install] header, as per below:

[Install]
WantedBy=default.target
  • Try watchdog service and ensure it is running correctly
$ sudo systemctl start watchdog
$ sudo systemctl status watchdog
● watchdog.service - watchdog daemon
  Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled)
  Active: active (running) since Tue 2018-04-17 03:30:58 EDT; 3min 3s ago
Main PID: 1143 (watchdog)
  CGroup: /system.slice/watchdog.service
          └─1143 /usr/sbin/watchdog

Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: int=1s realtime=yes sync=no soft=no mla=0 mem=0
Apr 17 03:30:58 denis-fitlet2 systemd[1]: Started watchdog daemon.
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: ping: no machine to check
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: file: no file to check
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: pidfile: no server process to check
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: interface: no interface to check
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: temperature: no sensors to check
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=root no_act=no force=no
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: watchdog now set to 60 seconds
Apr 17 03:30:58 denis-fitlet2 watchdog[1143]: hardware watchdog identity: iTCO_wdt
  • Enable autostart of the watchdog service
$ sudo systemctl enable watchdog
Synchronizing state of watchdog.service with SysV init with /lib/systemd/systemd-sysv-install...
Executing /lib/systemd/systemd-sysv-install enable watchdog
  • Reboot
  • Verify the driver was loaded automatically
$ dmesg | grep -i iTCO
  • Simulate kernel panic (as root user):
$ su
Password:
$ echo 1 > /proc/sys/kernel/sysrq
$ echo c > /proc/sysrq-trigger
  • As a result, the watchdog mechanism should reboot the system within 30 seconds.

IPC3, Airtop3, fitlet3

  • Load the kernel module wdat_wdt

NOTE: kernel module wdat_wdt does not produce debug messages, dmesg output will be empty 

$ sudo modprobe wdat_wdt
  • Check if the module is loaded
$ lsmod | grep wdat_wdt
wdat_wdt               24576  0
  • Install watchdog program, it should run in the background automatically on each system start
$ sudo apt install watchdog
  • Modify the following configuration files:
/etc/default/watchdog
change:                watchdog_module="none"
to:                    watchdog_module="wdat_wdt"

/etc/watchdog.conf
uncomment line:        watchdog-device  = /dev/watchdog
  • Try watchdog service and ensure it is running correctly
$ sudo systemctl start watchdog
$ sudo systemctl status watchdog
● watchdog.service - watchdog daemon
     Loaded: loaded (/usr/lib/systemd/system/watchdog.service; enabled; preset: enabled)
     Active: active (running) since Thu 2025-06-12 14:44:18 IDT; 46min ago
   Main PID: 3704 (watchdog)
      Tasks: 1 (limit: 8746)
     Memory: 564.0K (peak: 1.5M)
        CPU: 649ms
     CGroup: /system.slice/watchdog.service
             └─3704 /usr/sbin/watchdog

Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  file: no file to check
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  pidfile: no server process to check
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  interface: no interface to check
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  temperature: no sensors to check
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  no test binary files
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  no repair binary files
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  error retry time-out = 60 seconds
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  repair attempts = 1
Jun 12 14:44:18 test-fitlet3 watchdog[3704]:  alive=[none] heartbeat=[none] to=root no_act=no force=no
Jun 12 14:44:18 test-fitlet3 systemd[1]: Started watchdog.service - watchdog daemon.
  • Reboot
  • Check if the module was loaded
$ lsmod | grep wdat_wdt
wdat_wdt               24576  0
  • Simulate kernel panic (as root user):
$ echo 1 > /proc/sys/kernel/sysrq
$ echo c > /proc/sysrq-trigger
  • As a result, the system will freeze, and the watchdog mechanism should reboot the system within 60 seconds.
Retrieved from "https://wiki.edge.compulab.com/index.php?title=Linux_Mint:_Watchdog_configuration&oldid=193"