SKYHOUSE.dev Journal

Maintaining the Cloud Fortress

The Flow That Never Runs

Why: Damien deliberately left Don't Worry Darling alone after it wedged rdt-client, to find out what the system should do about a failed request on its own. The answer turned out to be: nothing, forever, silently — and not because anything was broken.

1. "Missing" is a state, not a work queue

Radarr's scheduled tasks contain no search for missing movies. The complete list of things that run on their own:

Rss Sync                      every 30 min
Refresh Monitored Downloads   every 1 min
Refresh Movie                 every 1440 min
Import List Sync / Housekeeping / Backup / Health ...

Rss Sync is the only acquisition mechanism, and RSS feeds carry only newly published releases. A 2022 catalogue title will essentially never appear in one again. So a monitored, available, missing movie sits in that state indefinitely with nothing looking for it. On 2026-08-10 that described 55 titles.

This is not a misconfiguration. It is how Radarr works, and it is reasonable — unattended searching for arbitrary back-catalogue is a good way to hammer indexers. But it means "I requested this and it never came" has no owner.

2. Why this particular failure left no trace

Radarr's entire history for the film was one line:

2026-08-08T23:43:06  grabbed  Dont.Worry.Darling.2022.1080p.BluRay.AVC.Tru

No downloadFailed, no downloadIgnored. The grab did not fail — it vanished, when the wedged rdt-client record was deleted during remediation. A disappearing queue item is not a failing one, so no failure event was recorded, the release was never blocklisted, and nothing downstream had a trigger to fire on.

And even a properly detected failure would have stopped there, because autoRedownloadFailed was False. The official flow — grab → fail → blocklist → search a replacement — was broken at both the detection step and the retry step.

3. Three fixes, each covering a different link

gapfix
a failure that is detected does not retryautoRedownloadFailed = True
this class of stall is never detected at allnew rule in radarr-queue-cleanup.py
nothing notices a title stranded for a weekarr-stranded-check.py, daily

autoRedownloadFailedFromInteractiveSearch was deliberately left off: when Damien picks a release by hand, silently substituting a different one would defeat the purpose of choosing it. A failed manual pick should come back to him.

4. The rule that makes the stall detectable

The 2026-08-08 shape matched none of the existing cleanup rules: no errorMessage, not importPending, and at the provider rather than not-at-provider. It simply read as "downloading" for sixteen hours. Two new gates:

provider-ready      + no local progress >=  6h  -> remove, blocklist, re-search
provider-incomplete + no local progress >= 24h  -> remove, blocklist, re-search

The split matters. provider-ready means TorBox holds it complete and cached, so if nothing has downloaded, the client is the broken party — six hours is generous. provider-incomplete means TorBox is still fetching from the swarm, which is legitimately slow, so that gets a day. An uncached grab is not a fault, just patience.

Progress comes from the pipeline probe's state file rather than new bookkeeping. The probe already samples sizeleft and first-sighting every 30 minutes; duplicating that would guarantee the two eventually disagree. Unknown history counts as "not yet" — acting on absent evidence is how a cleanup starts eating healthy downloads.

This is also the only rule that sets skipRedownload=false. The others deliberately avoid a re-search storm, but here the stranded title is the problem, so a replacement search is the entire point.

5. The check that would have told him

The pipeline probe watches the queue — items that entered and did not leave. It correctly reported UP the moment the stuck item was removed, because the pipeline genuinely was flowing. Nothing watched the other question.

~/bin/arr-stranded-check.py (daily 09:20) alerts on titles newly past a 7-day threshold. The standing backlog is deliberately silent: a monitor that is permanently red is a monitor that gets muted, which is the exact failure the 2026-08-05 alerting redesign exists to prevent. The first run seeds quietly — 55 titles recorded, zero alerts — because blasting a pre-existing backlog as 55 discoveries is noise, not news. Verified: seed run silent, second run silent, no Telegram sent.

Its Kuma monitor is a heartbeat only, always pushing up. It says "the checker ran"; it never says "you have a backlog". The Telegram digest is the alert, and it names the fix rather than just the problem.

6. A monitor created that should not have been

Running the provisioner with --apply to add the stranded monitor also created Job · dr-backup — a spec staged on 2026-08-09 and deliberately left unapplied, because that job has no repo and no cron entry. A dead-man's switch on a job that will never ping would have fired a false alert within 27 hours. Paused rather than deleted, so the spec survives for whoever completes the DR work. The lesson is small and general: a provisioner that converges to a file converges to everything in that file, including things staged for later.

7. A trap caught live, mid-write

While this entry was being drafted Damien found a Russian-dubbed film in Plex, deleted it from the Plex app, and picked a replacement in Radarr. The state five minutes later:

Radarr:  hasFile = True
         still tracks "Война против всех  War on Everyone (2016) ... .avi"
Disk:    folder does not exist

Plex deleting a file is invisible to Radarr until something rescans. Radarr still believed it held the file, so the incoming replacement would have been refused on import — upgradeAllowed=false blocks an upgrade over a file it thinks exists — leaving a completed download that never imports. The rule to keep: delete via Radarr, or rescan after deleting via Plex.

The system was not failing to do its job. It was doing exactly what it was configured to do, which was nothing, and there was no mechanism anywhere that considered that worth mentioning.

← Back to Admin Hub