Working log

Short notes on keeping small servers boring.

Things worth writing down after they broke once. Mostly Linux, networking and backups, kept brief on purpose.

Recent

Networking

Queueing discipline does not survive a reboot

Setting a queueing discipline with tc applies immediately but is forgotten on restart. The interface quietly falls back to the distribution default, and throughput drops without any obvious error.

The fix is dull: a one-shot unit that reapplies it after the network is up. Worth checking with tc qdisc show after every reboot rather than trusting that it stuck.

Linux

Package locks during first boot

A fresh image often runs unattended upgrades in the background. Any install started in the first few minutes fails on a lock held by another process, and the error text points at the lock file rather than the cause.

Waiting for the lock to clear is more reliable than forcing it. Removing the lock by hand leaves the package database half-written often enough to regret it.

Certificates

Renewal fails quietly more often than issuance

Issuing a certificate is visible and loud. Renewal happens on a timer, and when the webroot path moved six weeks earlier nobody notices until the expiry warning arrives.

Checking the renewal path on the day the certificate is issued costs a minute. Finding out at expiry costs an evening.

Backups

An untested restore is not a backup

Snapshots that run nightly for a year prove only that the job exits zero. The first real restore is where missing permissions, absent database roles and an unmounted volume all surface at once.

Restoring to a spare host once a quarter turns the question from whether it works into how long it takes.