Linux Watchdog – Watchdog automatically detects Linux lockups and reboots your system—free, lightweight, and especially useful for headless servers.
A locked-up Linux machine is more than an annoyance—it’s lost time, missed work, and sometimes a full reboot you didn’t schedule.
Misryoum has a simple way to make that recovery automatic using Watchdog, a free tool that monitors whether the system is still responding. When it detects a lockup, it can reboot the machine so you don’t have to wait, troubleshoot blindly, or remote into a host that never comes back.
Watchdog works by using a “kicker” concept: something regularly signals “we’re alive. ” and if that signal stops. the watchdog assumes the system is stuck.. In Linux terms, software Watchdog typically relies on a kernel component called softdog paired with a user-space service.. That combination creates a device node at /dev/watchdog and expects periodic resets of a timer—if the timer isn’t kicked. a reboot is triggered.
There’s also a hardware watchdog option, and it’s generally more reliable, but it requires specialized support. Software Watchdog isn’t perfect, but in day-to-day environments—especially labs and small home infrastructures—it can be a practical safety net.
Why the “autonomous reboot” matters for real setups
If you run desktops, Watchdog can help in situations where a system freezes and you can’t reach the GUI.. The bigger payoff shows up with servers—particularly “headless” machines without keyboards, monitors, or easy physical access.. Misryoum readers often run these as home lab nodes, media boxes, or lightweight services, and they frequently need unattended uptime.
The human impact is straightforward: fewer hours spent staring at a frozen screen. fewer forced hard power cycles. and less time restoring services after an unrecoverable stall.. When the machine automatically comes back. workflows that depend on it—SSH access. scheduled jobs. or internal network services—resume with less downtime.
How Watchdog detects lockups (and why it’s not magic)
At a high level, Watchdog’s job is to decide whether Linux is still progressing.. With softdog loaded, the system exposes a watchdog device (/dev/watchdog).. A service then “kicks” that device periodically, which resets the countdown.. If Linux locks up hard enough that the kicking process can’t run, the countdown expires and the machine reboots.
That design explains both the usefulness and the limitations. It doesn’t understand software crashes like a human debugger would, and it can’t fix a deeper failure. But it does handle one painful category well: systems that stop responding even though the hardware is technically still powered on.
In other words, Watchdog is best viewed as an automated fallback. It buys you reliability by turning indefinite hangs into scheduled-like recoveries.
Install and configure the free Watchdog service
Misryoum’s walkthrough stays focused on getting you to a working baseline quickly. The commands below assume you’re comfortable using sudo and a typical systemd-based distro.
On Ubuntu, install Watchdog with:
sudo apt-get install watchdog -y
On Fedora-based systems:
sudo dnf install watchdog -y
On Arch (using your preferred AUR helper workflow):
yay -S watchdog
Next, load the kernel module:
sudo modprobe softdog
Verify it loaded:
lsmod | grep softdog
Check that the device node exists:
ls -la /dev/watchdog
A key detail for persistence: make sure softdog is loaded at boot. If the module isn’t present after a reboot, the watchdog service won’t be doing its work.
Then configure /etc/watchdog.conf. Open it with:
sudo nano /etc/watchdog.conf
Look for lines like watchdog-timeout and interval. If those settings are commented out, remove the comment marker so they take effect. If the watchdog-timeout line isn’t present, add it so you control how long the system gets before rebooting.
Finally, enable and start the service:
sudo systemctl enable –now watchdog
Testing it safely (and what to watch for)
Misryoum also recommends a controlled test, because “configured” doesn’t always mean “working.” One method is to trigger an unresponsive state using kernel SysRq. If you do this on a machine where downtime won’t break anything important, you can verify whether Watchdog reboots the host.
The general idea is to set kernel.sysrq=1, then trigger a SysRq action through /proc/sysrq-trigger. If your system becomes unresponsive and Watchdog reboots it, you’ve confirmed the monitoring loop is functioning.
For systems where you can’t afford even brief disruptions, treat the test as a planned maintenance event—especially if the device is running services you can’t quickly restart.
Hardware watchdog: the “better reliability” path
If you have hardware watchdog support, systemd can be configured to kick the watchdog device and rely on the firmware-level behavior for reboot. The workflow typically involves editing systemd configuration entries such as RuntimeWatchdogSec, RebootWatchdogSec, and WatchdogDevice.
In practical terms. you adjust those values in /etc/systemd/system.conf. restart the systemd daemon with daemon-reload. and verify that systemd is now responsible for watchdog kicking.. This route is often favored when stability requirements are higher. because it removes some of the ambiguity that can come with software-only monitoring.
The takeaway: automate the boring worst-case
Watchdog won’t prevent every root cause behind a lockup, and it won’t replace good monitoring or log review. But it can eliminate the most frustrating part of many freezes: waiting forever for a system to self-recover.
For anyone running Linux machines they don’t want to babysit, Misryoum’s suggestion is simple—install Watchdog, tune the timeout, confirm the device exists, and enable the service. Once it’s doing its periodic “kick,” lockups become recoverable events rather than outages you have to actively manage.