SKYHOUSE.dev Journal

Why: After the release upgrade to Ubuntu 26.04.1 LTS, SSH on 2222 was refused — sshd was back on 22 — so we audited every manual override in the journal to see what else the upgrade had quietly reverted.

The dist-upgrade ran 14:30–14:58 today. I walked every /etc, /usr/local, and systemd path the journal has ever touched and compared it with the live system. Most of it survived. Four things didn't: the sshd config, the third-party apt repos, the sudo implementation itself, and one sudoers drop-in. All four are fixed now, and SSH and the repos are set up so the next release upgrade can't do the same thing.

1. sshd_config replaced wholesale

The upgrade installed the stock /etc/ssh/sshd_config and kept ours as /etc/ssh/sshd_config.ucf-old. Comparing the two showed three lines gone: Port 2222, PermitRootLogin no, and PasswordAuthentication yes. UFW and fail2ban were both still set for 2222. That left sshd listening on a port fail2ban wasn't watching and UFW wasn't letting in from the WAN.

• Action: rather than re-editing the main file, I put our settings in a drop-in, /etc/ssh/sshd_config.d/10-skyhouse.conf. sshd_config pulls that directory in at the top, and in sshd the first value wins, so the drop-in takes precedence. A future upgrade can replace sshd_config again and our settings survive. Checked with sshd -t, then restarted ssh. sshd -T now reports port 2222 / permitrootlogin no / passwordauthentication yes.
• Knock-on: ~/lock_ssh.sh and ~/unlock_ssh.sh toggle PasswordAuthentication with sed on the main file, so after this change they would have done nothing. Both now edit the drop-in instead.
• ssh still runs as ssh.service, not socket-activated (ssh.socket is inactive), so the Port line is all that's needed. If socket activation ever gets turned on, the port has to be set on the socket unit instead.

2. Third-party apt repos disabled

A release upgrade disables every non-Ubuntu repo. The two old-format .list files couldn't be converted to the new .sources format, so they became plex.list.disabled and docker.list.disabled with their deb lines commented out. netdata and google-chrome were converted but left as Enabled: no. The packages stayed installed and running, but nothing would update them again. The Plex repo matters most: plex-check / plex-update depend on apt seeing new Plex versions, which is the same quiet failure the 2026-09-10 entry was about.

• Action: I wrote plexmediaserver.sources (suite public) and docker.sources using the existing key files. Docker and netdata both already publish resolute (26.04) suites, so I pointed both at those instead of the old noble suite. Chrome is re-enabled. I removed the .disabled files after copying them, plus the original netdata/chrome .sources, to /home/plex/archive/2026-10-03-upgrade/.
• apt update is clean. Plex and netdata are already current. docker-ce has a same-version 26.04 build waiting (29.8.2 ~noble → ~resolute). We didn't install it today; it'll come in with the next normal upgrade, and it restarts every container when it does.

3. sudo became sudo-rs

26.04 replaces classic sudo with sudo-rs, a Rust rewrite, through update-alternatives. sudo-rs doesn't accept two settings our drop-ins rely on. It rejects Defaults:plex timestamp_type=global in claude-audit-tty, which broke the Claude sudo lease: a sudo -v in one terminal no longer reaches Claude's shell. It also rejects the wildcard log-read rules on line 10 of claude-readonly. It printed a parse warning on every sudo call and skipped those rules.

• Action: switched back with sudo update-alternatives --set sudo /usr/bin/sudo.ws (classic sudo 1.9.17p2; visudo follows it). We chose this over rewriting the drop-ins because classic sudo is still shipped and supported, and our sudoers files work on it unchanged. visudo -c passes on every file, the lease works again, and the NOPASSWD log reads work again.
• Gotcha we hit along the way: a sudo -v run under sudo-rs creates a sudo-rs timestamp, which classic sudo doesn't see. After switching, you have to run sudo -v again.

4. claude-readonly-hardware drop-in missing

/etc/sudoers.d/claude-readonly-hardware (from the 2026-05-31 entry) was gone. The upgrade modified /etc/sudoers.d at 14:38, but I can't prove the file was still there right before the upgrade.

• Action: recreated it from the journal's description (read-only only: du, smartctl -a/-x/-H/-i/--scan, dmidecode, blkid, fdisk -l). Validated with visudo -cf, then installed it 0440 in one step.

5. What survived

Everything else checked out: the GRUB cmdline (usb-storage.quirks is active in the running kernel; the bad-RAM memmap exclusions and the 10 s menu are still set), the modprobe.d/modules-load.d files, UFW rules (81 still LAN-only), jail.local, Docker's daemon.json log limits, every systemd drop-in (mount ordering, unmount timeouts, the Plex memory cap and TMPDIR), smartd.conf, the /etc/hosts split-horizon lines, DNS through the router only, fstab and both swapfiles, the /usr/local scripts, the Telegram/netdata configs, the user crontab, and the backup sudo grants. OpenVPN is still disabled. The upgrade also added a crashkernel= reservation to the kernel cmdline (the kdump default). It's harmless and we left it.

Unrelated, but it turned up during the audit: /media/plex3_backup isn't mounted, and its drive (Seagate WP001CYM) isn't visible to the kernel at all. That points at the DAS bay or hardware, not the upgrade. Until it's back, the config backup's mount guard means that job will skip its runs.

Net: SSH is back on 2222, the four third-party repos get updates again, and sudo is classic sudo again so the lease works. The sshd settings and the repo files are now in formats a future release upgrade won't overwrite or disable.