SKYHOUSE.dev Journal

Why: Stack creation in Portainer started failing with an instant "Network Error" in the browser, leaving orphaned stack records with no containers. Symptom appeared after the Portainer container was recreated on May 27.

The browser was aborting the POST /api/stacks/create/standalone/string request after ~20 seconds while Portainer was still pulling images server-side — so the stack row got written but the container never appeared in the UI (even though it sometimes did get created behind the scenes). NPM's access log made the diagnosis trivial: every POST returned HTTP 499 (client closed connection), while Portainer's own logs showed "Stack deployment successful" for the same requests.

1. Root cause

Default nginx buffers the entire upstream response before forwarding it to the client. Portainer's stack-create endpoint is a long-lived streaming response that emits image-pull progress over many seconds. With buffering on, the browser sees zero bytes for the duration of the pull and the fetch aborts — well before NPM's own proxy_read_timeout 90s would have fired.

Damien had already entered the right directives into NPM's per-host Advanced field, but without trailing semicolons. NPM does not auto-append ;, so the rendered 27.conf failed nginx -t and NPM deleted the file entirely instead of leaving the old one in place. That's why the host briefly served the wrong cert and produced ERR_SSL_UNRECOGNIZED_NAME_ALERT — portainer.skyhouse.dev had no server block at all and nginx fell through to a default.

2. Fix applied

• Updated the advanced_config column for proxy_host.id = 27 in the NPM MariaDB so the stored directives are syntactically valid (each line ends in ;).
• Hand-wrote /data/nginx/proxy_host/27.conf inside the npm-app-1 container to restore the host immediately, with the new directives inside the location / block:

  proxy_read_timeout 600s;
  proxy_send_timeout 600s;
  proxy_buffering off;
  proxy_request_buffering off;

• Validated with nginx -t and reloaded with nginx -s reload.
• Cleared the stale nginx_online: false / nginx_err values in the meta JSON column so the NPM UI no longer shows the host as broken.

3. NPM advanced-field gotcha to remember

If you paste directives into the per-host Advanced tab and forget a semicolon (or use any directive nginx rejects), NPM does not warn you — it silently deletes the rendered conf file. The host appears to vanish from the proxy, browsers get the wrong cert from a fallback server block, and the only trace is the meta.nginx_err field on the row in the proxy_host table. Worth checking that field first whenever a single host appears to drop off.

End result: stack creation through portainer.skyhouse.dev works again, and the underlying foot-gun (silent conf deletion on bad advanced config) is documented for next time.