> ## Documentation Index
> Fetch the complete documentation index at: https://xoxno.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# RPC disruption runbook

> Contingency actions when the Soroban RPC endpoint is unreachable, lagging, or rejecting submissions, and what stays safe while it is down.

What to do when the protocol's RPC access degrades. **Read first:** [Addresses](/docs/stellar-lending/dev/addresses).

RPC is how off-chain systems read and write the chain. It is not part of the protocol's trust model. An RPC outage stops you from observing and transacting; it does not change on-chain state, and it cannot make a position unsafe on its own.

## What breaks, and what does not

| System                                  | Effect of an RPC outage                                                                                                                                             |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| On-chain state, balances, indexes, debt | Unaffected. Ledger state is independent of your endpoint.                                                                                                           |
| Interest accrual                        | Unaffected. Accrual is computed from ledger time on the next interaction, not pushed by any service.                                                                |
| Application submissions                 | Fail. Users cannot supply, borrow, repay, or withdraw.                                                                                                              |
| Liquidations                            | Blocked for anyone using that endpoint, including third-party liquidators pointed at it.                                                                            |
| Storage rent keeper                     | Cannot read TTLs, extend them, or restore archived entries. Safe for a short outage, dangerous past about three weeks. See [Prolonged outages](#prolonged-outages). |
| Indexer ingestion                       | Stalls, then backfills once the endpoint returns. See [Indexer disruption](/docs/stellar-lending/dev/ops/indexer-disruption).                                            |

## Triage

Separate an endpoint problem from a network problem before acting. They look identical from one client.

| Check                                         | Endpoint fault | Network fault       |
| --------------------------------------------- | -------------- | ------------------- |
| A second, unrelated RPC provider              | Works          | Also fails          |
| Public network status                         | Normal         | Reports degradation |
| Ledger sequence advancing on another provider | Yes            | No                  |

If a second provider works, the fault is yours and failover fixes it. If no provider advances, the network itself is degraded and there is nothing to fail over to. In that case the correct action is to wait; the protocol's state is intact and no user is exposed by the delay.

## Failover

Each deployment reads its endpoint from configuration rather than a hardcoded value, so failover is a configuration change and a restart, not a code change.

<Steps>
  <Step title="Confirm the fault is the endpoint">
    Query a second provider for the current ledger. If it advances, your endpoint is the problem.
  </Step>

  <Step title="Repoint the affected services">
    Update the RPC URL for the application, the keeper, and the exporter or indexer. The network passphrase does not change — you are switching endpoints, not networks.
  </Step>

  <Step title="Restart and confirm reads">
    Confirm each service reads current ledger state before you consider it recovered.
  </Step>

  <Step title="Confirm submissions land">
    A successful read does not prove submissions work. Submit one low-value operation and confirm it lands.
  </Step>
</Steps>

## Prolonged outages

The one component with a real deadline is the storage rent keeper. Soroban ledger entries expire if their time-to-live (TTL) is not extended. An expired entry archives, and it must be restored before the contract can use it again.

The keeper exists to stay ahead of that. On mainnet it wakes every **6 hours** and extends anything due to expire within the next **21 days** (`ttl_safety_margin_days: 21` in `services/keeper/config/mainnet.yaml`; testnet uses 14). It also restores archived entries it finds.

That margin is the outage budget. A few hours of downtime costs you a tick or two and nothing expires. Losing the keeper for **three weeks** is what actually risks expiry.

So if RPC access looks like it will be out for days, restore keeper connectivity **before** the application. Delayed user transactions cost users time. Expired storage costs a restore transaction per entry, and the contract cannot run until it is back.

## Do not

* **Do not point production at an endpoint you do not trust.** RPC responses drive what your application shows users and what the keeper decides to submit. A hostile endpoint can lie about state. Prefer a known provider over the first reachable one.
* **Do not assume the chain is down because your endpoint is.** Check a second provider before declaring a network incident.
* **Do not pause the protocol for an RPC outage.** Pausing is an on-chain action addressing an on-chain problem. It does not repair connectivity, and it requires a governance action to reverse.