SKYHOUSE.dev Journal

Why: Telegram had been accumulating Radarr and rdt-client alerts — a “51 media request(s) stranded” digest, repeated DID NOT STICK lines from the hourly Radarr cleanup, and an rdt-client WEDGED that did not match the wedge signature we built the watchdog for. We went looking for whether these wanted cleanup services or hands, and found that most of them were one disease.

Radarr's queue held 19 records, rdt-client held 82 across both instances, and the TorBox account held 4 torrents. Those three numbers should track each other. The gap between them is the whole story, and it is not three problems — it is one loop with three places to break.

1. TorBox flaps, and rdt-client eats the status page

TorBox went degraded at 17:16. Their API does not fail cleanly when this happens: it 302-redirects to status.torbox.app, which answers 200 with HTML. rdt-client hands that to a JSON parser:

• Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
• …thrown from TorBoxNET.TorrentsApi.GetCurrentAsync, unhandled in ProviderUpdater.ExecuteAsync, retried every ~15 seconds forever.

The dashboard stayed up throughout, which is what made this confusing from the outside — the web UI and /torrents/mylist are different backend paths. Measured directly: 2 of 6 API calls succeeded, 4 hung past 12 s. This is a flap, not an outage, and a flap is worse, because every layer below treats a hung call as a temporary condition it will silently forget about.

2. Why nothing self-heals

While ProviderUpdater spins, rdt-client's qBittorrent shim stops answering. Radarr talks to rdt-client through that shim, so its “remove from client” call times out at ~103 s — 116 times in the preceding 7 days. And the arrs rebuild their queues from the download client. So:

• rdt-client's post-import FinishedAction (remove from provider) fails → the torrent stays in the TorBox account forever. Two were sitting there fully imported: The Secret of NIMH and the Criterion Nowhere.
• Radarr's remove-from-client fails → the rdt-client record survives → Radarr re-creates the queue entry from it on the next sync, with the same queue id.
• No layer records a failed delete as retryable. The failure leaves no state anywhere, so nothing ever comes back to finish the job.

Damien called this one before we had the logs open: rdt-client sends a message, expects a response, doesn't get it, and the record stays stuck. That is exactly it — it is just three layers deep rather than two.

3. The cleanup that was written and never scheduled

bin/rdtclient-cleanup.py has existed since 2026-08-04 and was never put in cron. This is the same gap radarr-queue-cleanup.py itself sat in until 2026-08-05 — the tool written, documented, correct, and simply not wired to anything.

The consequence was visible hourly: seven Joy.Ride.2021 releases had been resurrecting in Radarr's queue for nine days, each cleanup run dutifully deleting them and logging DID NOT STICK on the next. Joy Ride (2021) had in fact imported successfully on 2026-08-11; the seven were losing candidates from a parallel-grab burst, all rejected by TorBox with DIFF_ISSUE: Download already queued, all kept forever by rdt-client.

I added it at :05, twelve minutes ahead of the Radarr cleanup at :17, so the client layer is clean before the arr layer re-syncs. It requires --include-arr-referenced: without it the two layers protect each other into deadlock — rdt-client keeps the record because an arr references it, and the arr references it only because rdt-client holds it. Both real gates still apply (the record must carry an error and be ≥24 h old), so an in-flight download is never a candidate.

First run cleared 8 records and Radarr's queue dropped 19 → 12. The Joy Rides did not come back.

4. The class the cleanup cannot reach

Eight queue entries survived, and they turned out to be a second failure shape that rdtclient-cleanup.py is blind to by design: rdt-client records that are completed with error = None, whose files no longer exist. The cleanup only considers errored records, so these are unreachable by it.

• Six showed “Movie title mismatch, automatic import is not possible” — all six had movieId = None because the movie is no longer in the Radarr library at all, and their staging files were already gone. Nothing to import, nothing to reassociate, no film to attach to.
• Stop! That! Train! and Finding Nemo were redundant: both films already hold a better file in the library (a 5.46 GB REPACK and a 5.23 GB BluRay respectively), which is why Radarr correctly refused the queued copies.

Cleared by hand this time, with the documented record-only flags (deleteData=True, the other two false). Queue is now 19 → 4, and the four remaining are real: two genuinely stalled downloads, one TorBox-side reported missing, and one delete still in flight.

5. A movie that was a Windows executable

Queue entry 001FF871… was filed against Toy Story 2. It was an 877 MB PE32+ executable (x86-64, stripped to external PDB, for MS Windows) padded to feature length. Radarr's “Caution: Found executable file” refusal is the only reason it was still sitting in staging rather than in the library.

Inert on this box — nothing here executes PE binaries, and Plex would never have touched it — but it is worth recording that the pipeline does surface these, and that the guard which caught it is Radarr's, not ours. Deleted, along with the last two staging leftovers (a 1.5 GB redundant Finding Nemo and a Microcosmos directory containing only a 23 MB sample.mkv). /media/plex1/torbox_downloads is now empty — 2.4 GB reclaimed.

6. The 51 stranded requests were one alert, seven days late

This one was an alerting bug, not 51 incidents. arr-stranded-check.py seeds quietly on its first run so it cannot blast the pre-existing backlog — but that guard only suppressed run #1. Every seeded title kept first_missing = seed time, so all of them crossed the 7-day threshold on the same run. The state file is unambiguous:

• 51 ids sharing an identical first_missing of 1786407536.58 — 2026-08-10 17:18, the seed run.
• All 51 alerted at 1787070001.85 — 2026-08-18 09:20. Seed + 7 days + one cron tick.

Precisely the pre-existing-condition blast the seeding was written to prevent, arriving a week late. Seeded ids now live in their own seeded bucket and are excluded from “newly stranded” permanently; they leave the bucket only by arriving, and a title that goes missing again afterwards alerts normally. Backfilled the 51 into the existing state, and --report now marks them [seeded backlog — silent]. The check now reads 63 missing, 53 stranded, 51 seeded-backlog, none new, correctly leaving All Man: The International Male Story and War on Everyone as the only genuine strandings.

7. The overlap worth remembering

Three of the “stranded” titles — War on Everyone (stuck at 99%), Everything's Gone Green (5%), Picture Day (0%) — are simultaneously sitting in rdt-client as non-completed records, some for nearly a month. The stranded check reports “nothing is happening” while something is happening and has been permanently stalled for weeks. Neither monitor can see the other's half of the story. These are uncached torrents fetching from the swarm, which is the exact condition bin/arr-cached-search.py exists to prevent — and it, too, is not scheduled.

Net effect: Radarr's queue went 19 → 4, rdt-client shed 16 dead records, staging went empty, one alert class was a bug and is now silent, and the loop that regenerated all of it hourly is finally wired to run on its own.