> ## Documentation Index
> Fetch the complete documentation index at: https://xoxno.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Indexer disruption runbook
> Contingency actions when the indexing and API layer stalls or serves stale data, and why protocol solvency does not depend on it.
What to do when the indexing and API layer stops keeping up with the chain.
The indexer reads the chain and serves that data to dashboards and integrators. **No contract reads from it.** The protocol cannot be made unsafe by an indexer failure, because nothing on-chain depends on its output. An indexer disruption is a visibility problem, and the main risk it creates is people acting on data that is quietly out of date.
## Blast radius
| System | Effect |
| -------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| Contract state, prices, accrual, liquidation eligibility | Unaffected. The protocol reads prices from the oracle and state from its own storage. |
| Dashboards and market pages | Stale or unavailable. |
| Historical series (TVL, revenue, volume) | Gaps for the affected window, recoverable by backfill. |
| Integrators reading the public API | Serve stale values unless they check freshness. |
| Liquidators using the API to find targets | **Will miss liquidatable positions.** See below. |
## The one real risk
The protocol's safety depends on unhealthy positions being liquidated promptly. If liquidators discover targets through the API and the API is stale, positions can become liquidatable without anyone acting.
The protocol stays correct here: a liquidation submitted against on-chain state either qualifies or reverts, so no bad liquidation results from stale data. The risk is inaction, not wrong action.
Confirm every liquidation against on-chain state before you submit it. Treat the API as a discovery aid, never as the authority. `is_liquidatable(account_id)` on the controller is the authority.
If an indexer outage runs long during a volatile market, assume liquidation coverage is degraded and monitor on-chain health directly rather than through the dashboard.
## Four traps that silently corrupt a rebuild
A backfill can finish cleanly and still be wrong. These four are protocol-specific, and each one produces a gap that looks like healthy data.
1. **Market state comes from the pool, not the controller.** `PoolMarketStateBatchEvent` and `PoolMarketParamsBatchEvent` are published by the pool contract. An indexer subscribed only to the controller address silently indexes no market state at all.
2. **One liquidation can publish two position batches.** A `SeizeMode::Credit` liquidation writes the liquidated account's batch first, then the receiving account's. Key on `account_id`; never assume one operation yields one batch.
3. **A `Credit(0)` liquidation creates an account with no creation event.** The new account is announced only inside the second batch's `account_attributes`. An indexer that discovers accounts from a creation event will never see it.
4. **`LiqSeize` is gross, `LiqCredit` is net.** The protocol fee sits between them. Summing both tags as the same quantity double-counts the seizure.
See [Events](/docs/stellar-lending/dev/events) for the full wire shapes. If your rebuild predates any of these fixes, re-check the affected range rather than trusting the row count.
## Triage
| Symptom | Likely cause |
| ------------------------------------------------- | ---------------------------------------------------------------------------------- |
| API responds, values frozen at a past ledger | Ingestion stalled; serving layer healthy. |
| API unreachable | Serving layer or infrastructure fault. |
| Some endpoints current, others behind | Partial pipeline failure; one series is stuck. |
| Ingestion stalled and RPC also failing | Upstream RPC fault. See [RPC disruption](/docs/stellar-lending/dev/ops/rpc-disruption). |
| Positions look wrong only for liquidated accounts | One of the liquidation event traps above, not an ingestion fault. |
Always determine the ledger the indexer has reached and compare it to the chain's current ledger. That gap is the actual severity measure, and it distinguishes "slightly behind" from "stopped".
## Recovery
Compare the last indexed ledger to the current ledger. Record the gap before restarting anything, so you know the window that needs backfill.
Check RPC health first. An indexer that cannot read the chain is a symptom, not the fault.
Restart from the last successfully indexed ledger, not from the chain head. Starting at the head leaves a permanent hole in the historical series.
Replay the missing ledger range. The chain is the source of truth and the range is fully recoverable, so a gap is repairable as long as you resume from the right point.
Reconcile a sample of derived values against a direct contract read. For one market, compare your totals to the pool's `get_supplied_amount(hub_asset)`, `get_borrowed_amount(hub_asset)`, `get_reserves(hub_asset)`, and `get_revenue(hub_asset)`, and the controller's `get_market_indexes_detailed`. Matching values confirm the backfill was complete, not merely finished.
## Communicating during an outage
State whether protocol funds are affected, because that is the question users will actually be asking. During an indexer outage the answer is that they are not: balances and positions are on-chain and intact, and only the display of them is degraded.
Tell users that positions remain safe, that on-chain actions still work if the application can reach RPC, and that displayed figures may be behind until backfill completes.
## Do not
* **Do not resume from the chain head to make the API look current.** It hides the gap instead of repairing it and leaves a permanent hole in history.
* **Do not treat API totals as authoritative** when reconciling protocol accounting. Read the contracts.
* **Do not pause the protocol for an indexer failure.** The contracts are unaffected, and pausing would convert a display outage into a real one for users.