Emendant Feed & Website Down

Incident Report for Emendant

Postmortem

Summary

On 12 September 2026, feed.emendant.com stopped answering for about 40 minutes. The cause was an infrastructure migration: the emendant.com DNS zone was moved to a separate Cloudflare account, while the storage bucket serving the feed stayed on the original account. A hostname can be claimed by one Cloudflare account at a time, so the moment the zone moved, the claim on feed.emendant.com no longer resolved to the bucket behind it.

The website and the status heartbeat endpoint were unavailable for the same reason and over the same window.

Impact

emendant feed update failed for the duration. Clients that could not reach the origin fall back to the feed bundled in their installed version, so scans continued to run and continued to report findings, against a feed that did not include entries published since that build.

No data was lost, altered or exposed. No signing key was involved at any point. The published snapshot history, its sequence number and every signature are unchanged: the migration copied the stored objects and re-signed nothing. All 434 objects were verified byte for byte against their originals after the move, and the restored origin was checked with the client's own verification code before the incident was closed.

Timeline

Time (UTC) Event
17:05 Last successful origin poll
17:10 First failed poll. Origin unreachable
17:15 Second consecutive failure. Alert raised, incident opened
17:20 Cause identified as the account split described above
17:31 Bucket recreated on the account now holding the zone, and the feed contents restored to it
17:33 feed.emendant.com re-pointed at the restored bucket
17:37 Website and heartbeat services redeployed
~17:45 All hostnames serving again
19:45 Snapshot publishing verified end to end against the restored origin

The restoration time is approximate. The monitor that would normally record it was itself affected, which is covered below.

Root cause

The DNS zone and the resources serving it were moved separately. Cloudflare treats a custom hostname as owned by one account, so moving the zone alone left every hostname pointing at an account that no longer had authority to answer for it. The failure was immediate and total for the affected names rather than gradual, which is why detection was fast.

A second, smaller delay came after the resources were restored. The zone move carried over DNS records for the affected hostnames, and Cloudflare will not bind a service to a hostname that already has an unmanaged record. Those records had to be removed before the new bindings would take effect.

Detection

The outage was detected in five minutes and alerted on in ten, by the origin monitor polling every five minutes and holding its alert until a second consecutive failure. That behaved exactly as designed.

Known gap in monitoring

The five-minute origin monitor has not resumed. It runs as a Cloudflare scheduled job, and scheduled jobs are not currently being delivered on the account. This matches a Cloudflare incident, "Workers Cron Triggers degraded", raised on 9 September and marked resolved on 12 September, which continues to reproduce for us on a minimal test case created after that resolution. It has been reported to Cloudflare.

Until it resumes, origin freshness is checked once a day by an independent job that runs outside Cloudflare. That job exists precisely so that a Cloudflare failure cannot take down both the service and the thing watching it. The practical effect is that detection of a future origin outage would take up to a day rather than ten minutes. The status page origin component may therefore be slower to update than usual.

What is changing

The credential used to publish feed snapshots was documented as needing write access only. Publishing also reads the origin three times, so a correctly minimal credential failed. The documentation has been corrected.

The publisher reports that it cannot read the origin, but not why. A permission refusal and an empty origin currently reach it as the same answer. We are treating that as a defect: for a full publication it could be read as the origin having no history. It did not occur during this incident, and the snapshot sequence is confirmed intact.

Posted Sep 12, 2026 - 19:51 UTC

Resolved

This incident has been resolved.
Posted Sep 12, 2026 - 19:25 UTC

Monitoring

We applied a fix at 17:40. feed.emendant.com serves the current signed snapshot again, and emendant feed update succeeds.

If your last update failed, run emendant feed update again. Nothing else is needed.

We are watching the origin for the next hour before we close this.
Posted Sep 12, 2026 - 17:46 UTC

Identified

We have found the cause: the origin bucket stopped serving the pointer document after a configuration change.

A fix is being applied now. The effect on users is unchanged: emendant feed update fails, and every other command works from the snapshot already on your machine.
Posted Sep 12, 2026 - 17:23 UTC
This incident affected: Emendant Feed (Emendant Feed).