Owns: whether a competitor product is active, no_longer_supplied (walk-delisted), or discontinued (scan-delisted) — and the guards that stop us delisting a product that's actually still for sale.
active.Guard: a walk below
TRUNCATION_FRACTIONof its trailing baseline is flagged truncated and cannot delist.
| "Gone" signal | Mechanism | How we detect it | Action |
|---|---|---|---|
| Absent from catalogue | WALK reaper | Not seen across N consecutive complete walks | → no_longer_supplied |
| Hard 404 on the PDP | SCAN reaper | HTTP 404, N consecutive | → discontinued (kind http4xx) |
| "No longer available" soft page | SCAN reaper | Bunnings isActive:false; T&W category-redirect (CollectionPage/ItemList, no Product node) |
→ discontinued (kind stub-page, after 3 strikes) |
| Trap | Why it fools us | The guard |
|---|---|---|
| Truncated walk | A short walk looks like mass-absence | TRUNCATION_FRACTION gate — a walk must reach ≥ the fraction of baseline to delist (currently 0.97) |
| Transient WAF block | A reese84 challenge returns a non-product page | the block/challenge kind is not a delist signal — retry |
| Timeout / 5xx | Momentary failure | not a delist signal — retry with cooldown |
| Re-slugged product | URL changes; the old URL 404s but the product still exists at a new URL | (open) discovery re-walk should re-match before delisting |
lib/catalogue/supply-reaper.ts, called from finalizeCatalogueRun in lib/catalogue/refresh.ts. A product absent from N consecutive complete walks → no_longer_supplied. Completeness is gated by TRUNCATION_FRACTION (detectTruncation). Runs where the walk terminates — on the fleet worker for fleet-delegated walks — so a change here needs both the catalogue-runs daemon restart AND the fleet pin to advance.lib/scrape/apply-result.ts. Consecutive failures of a delist-kind (stub-page, http4xx) stamp discontinued_at at DISCONTINUED_THRESHOLD (3). Transient blocks/timeouts do not count.isActive:false + T&W category-redirect served HTTP 200 → logged generic no_data → retried forever. Now routed to stub-page → discontinued after 3 strikes. This is the gap this whole wiki exists to have caught.TRUNCATION_FRACTION 0.90→0.97 + un-delist.