SKYHOUSE.dev Journal

Why: Ellis and Damien both had Google Takeout exports sitting on disk — roughly half a terabyte of photos across eleven zip parts — and neither had made it into Immich properly. Damien wanted his originals gone afterwards, which meant the import had to be actually trustworthy, not just apparently finished.

This took most of an evening and ran unattended after Damien left. The short version: both libraries are in, verified against Postgres rather than the API, and the only permanent losses are six files that were already corrupt before they ever reached this server. The longer version is more interesting, because immich-go has two failure modes that look like success.

1. Where things actually stood

The starting picture was messier than the summary suggested. Ellis's export was seven parts, not five. Part 002's zip was corrupt and had been hand-recovered into a folder. Parts 003–007 existed in two places with differing integrity. Both Immich accounts already held substantial partial imports from earlier attempts — Ellis 91,072 assets, Damien 82,026 — so this was never a clean-slate import; it was a reconciliation.

That prior state is why we used immich-go rather than the generic CLI in ~/process_takeout.sh: it parses Takeout's sidecar JSON, recreates albums, and backfills album membership onto assets that are already uploaded. Running it over content that's mostly present is not wasted work — it's the point.

2. Ellis's import took three passes

The first pass processed 133,439 assets and reported zero errors. It also reported 647 assets "pending — did not reach a final state," which is the sort of line that's easy to skim past. It mattered.

• Pass 1 uploaded 35,019 assets and made 27,899 album placements.
• Pass 2 re-ran the identical command to retry album and tag assignments that had failed with HTTP 400/500 on individual batches. Album errors dropped 10 → 1, tag errors 4 → 0, and another 6,321 album placements landed.
• Pass 3 was a different tool mode entirely, and it's the one that actually fixed the pending assets.

Mid-run there was a genuine scare: Immich's API showed Ellis's video count collapsing from 8,648 to 1,873 while an import was running. That turned out to be benign — /api/assets/statistics excludes Live Photo motion parts, and Immich had just finished linking 15,544 motion-photo MP4s to their parent stills. Postgres showed 135,089 visible assets and zero soft-deleted. Lesson: for anything that matters, query the asset table directly rather than trusting the statistics endpoint.

3. Two ways immich-go loses files quietly

These are the findings worth keeping, because both produce a summary that says Errors: 0.

• A bad CRC makes it abandon the rest of the archive. On zip: checksum error, immich-go doesn't skip the entry and continue — it stops processing that zip entirely. Parts 003 and 005 each hit one, and everything after the bad entry in those archives was silently never read. That alone accounted for most of the 647 pending assets. Never feed it a zip with known-bad CRCs; extract first, since unzip skips bad members and continues.
• from-google-photos skips files that have no sidecar JSON. They're logged only as WRN missing metadata and end up neither uploaded nor discarded — visible only as the Pending count in the final summary. Re-running the same command does not fix them; the count stayed at 646 across two passes. The fix is upload from-folder --folder-as-album=FOLDER, which doesn't require metadata and reconstructs album names from the directory structure.

So the recovery procedure is: pull the file list out of the WRN missing metadata log lines, extract exactly those members with their directory structure intact, and upload the staging folder in from-folder mode. Structure matters — the first attempt used unzip -j and 645 paths collapsed into 385 distinct basenames, because names like FullSizeRender(1).MP4 recur across albums.

Worth noting how much of the apparent loss was illusory: of Ellis's 645 recovered files, only 113 were genuinely absent. The rest were content-duplicates already present under different filenames. Immich deduplicates on content hash, so "this filename isn't in the database" never proves an asset is missing.

4. Damien's import, with the lessons applied

Integrity testing found three of his four May zips damaged — but only six bad files between them. Rather than let immich-go abort mid-archive three separate times, I extracted 002, 003 and 004 up front and fed it the clean 001 zip plus three extracted folders in a single invocation, so albums resolve across part boundaries.

It ran with zero ERR lines — a visibly cleaner run than Ellis's, entirely because of the pre-extraction. It still left 1,313 pending assets, exactly as predicted, and the from-folder recovery cleared them: 386 genuinely new files, 1,149 album placements, pending to zero.

5. Where things landed

⚠️ Correction added 2026-08-09 — the growth figures originally in this entry were wrong. The "before" numbers (Ellis 91,072, Damien 82,026) came from /api/assets/statistics, which excludes Live Photo motion parts; the "after" numbers came from Postgres, which includes them. Comparing the two overstated how much was added — badly in Damien's case. Measured properly by createdAt, the imports added 35,626 assets for Ellis and 518 for Damien (matching his logs exactly: 126 uploaded + 386 recovered). Damien's number is small because his account already held nearly the entire May batch. The absolute totals below are correct as database counts; only the implied growth was wrong. Lesson: never compare an API count against a DB count — the gap is ~16,000 assets for Ellis alone.

• Ellis: 135,202 visible assets, 799 albums, 0 soft-deleted. (API reports 119,149 — the difference is motion parts.)
• Damien: 94,305 visible assets, 406 albums. (API reports ~82,436.)
• Permanently lost: six files, all corrupt at source before arriving here — IMG_4397.MOV, DSC09122.JPG, IMG_6796.HEIC, dsc00386.jpg, and two in part 006. Six out of 133,439.
• Reclaimed: ~300 GB by deleting Ellis's redundant SSD copies once his import verified. The root disk went from 70% to 37% used.

One thing deliberately not done on the night: Damien asked for his May originals to be deleted once his import verified, but he'd left before that point and never gave the explicit go-ahead. Deleting the only remaining copy of someone's photo library is not a call to make on their behalf while they're asleep, so it waited.

6. Cleanup — completed 2026-08-10

Damien checked his library, was satisfied, and authorised the deletions. Removed: his May batch from both plex3 and plex3_backup, every derived staging and extraction folder, and — after a separate explicit confirmation, since it is Ellis's archive rather than his own — Ellis's seven original zips from both drives too.

Before recommending that last one I checked whether backup-plex1.sh was actually running, because the whole justification is "Immich is backed up." It is: active in cron at 04:30 nightly, with a matching Photos tree on plex1_backup. Worth verifying rather than assuming, given this same server has had a backup cron sitting silently commented out before.

No Google Takeout archives remain on the server. Immich on plex1, mirrored to plex1_backup, is now the sole copy. Freed roughly 1.6 TB across plex3 and plex3_backup, on top of the ~300 GB reclaimed earlier from the root SSD — which went from 70% to 37% used, and plex3 from 45% to 37%.

Two libraries reconciled into Immich, six files lost to pre-existing corruption, 300 GB of SSD reclaimed — and a documented recovery procedure for the two ways immich-go reports success while skipping your files.