Skip to main content
An indexer outage makes API and dashboard data stale while contracts continue using on-chain state. Restore ingestion, recover the missing event range, and check liquidation coverage during the outage.

Impact

Triage

Compare the durable ingestion cursor and last fully committed ledger with the chain’s current ledger. Record the gap, event-source retention, and affected endpoints before restarting services.

Maintain liquidation coverage

Use direct contract reads to monitor account health when API discovery is stale. Build the candidate inventory from retained NFT and lending events, an archival inventory, or paginated v1 reads for known owners. Known-owner reads do not enumerate every account. Preserve this coverage limit in incident reporting. The controller’s is_liquidatable(account_id) reads on-chain eligibility; re-simulate the complete liquidation and apply execution limits before submitting. A valid target read does not guarantee that a later transaction succeeds. Delayed repayments and liquidations can increase financial risk. Assess protocol controls from observed risk and their documented effects; pausing does not repair indexing.

Recovery

1

Restore the event source

Confirm RPC health and access to the missing ledger range. A source whose retention begins after the outage cannot provide a complete backfill. Public REST endpoints do not establish a canonical historical event source; confirm the chosen source’s actual retention and pagination.
2

Resume from the durable checkpoint

Restart from the exact saved cursor. Commit decoded events, derived state, and the cursor atomically. If a ledger was only partly processed, replay it with deduplication. Starting at the chain head leaves history unrecovered.
3

Replay the gap

Preserve ledger continuity and event order. Deduplicate parent events by (txHash, eventId) and batch children by (txHash, eventId, childIndex). Apply state only from successful transactions and successful contract events. Keep raw records and unknown action tags for later decoder repair. Current contract state cannot reconstruct events that the source no longer retains.
4

Check event coverage

Subscribe to both controller and pool events, plus position-NFT mint, transfer, and burn events. Apply all account batches and their children in transaction order. Follow the event decoding and indexing rules, including gross seizure versus net credit and account creation without a lending batch.
5

Reconcile derived state

Compare sampled markets with the pool’s get_supplied_amount(hub_asset), get_borrowed_amount(hub_asset), get_reserves(hub_asset), and get_revenue(hub_asset). Check the controller’s get_market_indexes_detailed for indexes and price diagnostics.Matching samples do not prove complete history. Also verify checkpoints, ledger continuity, source coverage, and deduplication across the full gap.

Recovery criteria

Close the incident when ingestion advances with the chain, affected API series are fresh, the retained event range is replayed, and reconciliation succeeds. Record any unrecoverable historical range explicitly. Communicate the last indexed ledger, affected services, and observed liquidation coverage. Ledger balances persist, but stale displays and delayed protective transactions can change account risk. On-chain actions require a healthy RPC.