Planning an Indexer Backfill: log-range budgeting across chains

A backfill runs for six hours, then dies on one chain with an eth_getLogs range error, at a width that worked on the chain before it. The quieter failure is the one that finished, reported success, and left a hole you found a week later.

Neither is a loop bug. Both come from starting before the ceiling was known. The work is to budget the range before the first call: find each chain's real ceiling, spend it deliberately, checkpoint so a restart is cheap, and recognise the failures that are not about range at all.

A backfill is a range problem, not a loop

The naive version — one eth_getLogs call from the deployment block to the tip — fails per chain, and the ceiling is not the same on every chain or behind every provider. Three things get chosen before the first call: range width, concurrency, and checkpoint granularity. Leave the third until later and the first failure decides it for you.

The ceiling can bind on block span or on result count, and the two are not the same kind of limit. A block-span cap is a number somebody configured. In geth it is the RangeLimit config field, documented with a default of 0, which means disabled [1]; when it is set, the filter rejects the query with exceed maximum block range %d [2]. Hosted providers republish that setting as a plan property: Chainstack documents 100 blocks on its Developer plan and 10,000 on the paid plans [3], and Alchemy publishes a per-plan block-range table with a 150MB response cap [4].

A result cap is a property of the data. Infura allows a maximum of 10,000 results per query and returns error -32005 with the message query returned more than 10000 results [5]. geth has no equivalent: its LogQueryLimit caps the number of addresses and topics in a filter, default 1000, not the number of results [1].

That difference decides how you plan. A block-span cap is a ceiling you can look up. A result cap is one you have to measure, because it depends on how busy the contract was in the range you asked for. That is why a width that is fine on a quiet chain fails on a busy one at the same width.

Find the real ceiling for each chain

Guessing the width from another chain is a common way to lose an afternoon. Client-level ceilings vary by client, and Chainstack's documentation says to check the node client's docs or source for the chain you are targeting [7]. Two nodes behind one endpoint can answer the same request with different caps [8].

Measure it instead. Start with a width you believe is safe, double it until the call fails, then binary-search between the last width that worked and the first that did not. Record the exact error body at the boundary, not just the fact of failure, because the wording tells you who enforced the limit. A client-side cap reads exceed maximum block range: 50000 [8], which tracks geth's own format string [2]. A plan-side cap reads:

eth_getLogs range of 100001 blocks exceeds the 500-block limit for this plan

[8]

Same failure, different owner, and only one of them is worth raising with support.

The boundary moves with block density. A width that clears a quiet week on one contract will hit the result cap during a busy one, so re-measure after a launch, an airdrop, or any week the chain got busy. Provider guidance points the same way: query smaller ranges and paginate on high-activity chains or contracts [9]. The exact messages a node returns at the range boundary are collected on our /errors page, which is worth reading before you write the matcher.

We do not publish our own per-chain numbers here. The mechanics and the adaptive chunking pattern are covered in How eth_getLogs Range Caps Bite in Production; measure each chain yourself rather than importing a number from another network.

Budget the range, not just the batch

The real budget is width × concurrency × the provider's rate limit, and the third term is usually enforced over a rolling window rather than per request. Alchemy evaluates limits over a 10-second rolling window with a token bucket and strict refill rates, so a 300 CU/s account can spend up to 3,000 CUs in any 10-second period [10]. A wide range at high concurrency spends that allowance faster than a narrow range at the same concurrency, and the failure arrives as a 429 rather than a range error.

Infura's 429 body carries event -33200 and the message Too Many Requests, with details telling you to reduce requests per second or upgrade [11]. Shrinking the range does nothing for it. Backoff and jitter in an RPC client is its own problem, and so is which JSON-RPC errors are retryable at all; both are covered in RPC retry and backoff.

Retries are spend, not free. A backfill that spends a third of its allowance re-issuing a range that was never going to fit has a smaller effective budget than one that shrinks on the first boundary error. Classify the failure before reacting to it:

Infura also caps query duration at 10 seconds [5]: shrinking the range fixes that one, retrying does not.

The filter is the other lever on the result cap. A range that returns too many results can often be split by address or by topic instead of by block, and building topic0 filters to narrow a range keeps the block span you already measured.

Checkpoint so a restart is cheap

Persist the highest contiguously committed height, and resume from that, not from the tip and not from the last range you started. The distinction matters because ranges finish out of order: with concurrency above one, the range at height 12,000,000 can commit while 11,900,000 is still in flight. A checkpoint that records the highest committed height will skip the gap.

Two records do the work. A per-range completion marker, so a partially committed batch is visible rather than inferred. And the highest contiguous height, which is the only number a restart can trust.

A backfill without a checkpoint does not fail once. It fails every time it restarts, and each restart re-spends the allowance on ranges that already succeeded. A range error is deterministic — the same request at the same boundary fails the same way — so a checkpoint plus the recorded boundary is enough to resume without re-probing the ceiling.

The checkpoint is also what makes a completeness check possible. Compare the committed range list against the planned one, and confirm the highest contiguous height reaches the tip you meant to cover. Without the markers there is nothing to compare, and a backfill that reports success is reporting on its own bookkeeping.

Reorg handling is a separate problem, and it belongs with idempotent writes and gap detection rather than with range budgeting.

Historical state is a different problem

A pruned full node serves recent blocks and then refuses an old state read. geth reports it as missing trie node <hash> (path <path>) [13], over the underlying condition ErrStateUnavailable, state not available, returned either because the state was never stored or because it has been pruned [14]. Our /errors page records the same failure as missing trie node (state at a past block is pruned) and notes that geth-family clients report the gap that way because the trie root for that block is gone [15].

The fallback is bounded. geth will walk back at most 128 ancestor blocks on hash-scheme nodes when reconstructing missing historical state, before giving up [16]. That is far short of a backfill, and it is the concrete reason a pruned node cannot be talked into serving an old height. Routing historical state reads to an archive endpoint is its own decision, covered in full node vs archive node; retrying them is not a decision at all.

Log queries are not immune. eth_getLogs returns PrunedHistoryError — pruned history unavailable, JSON-RPC code 4444 [12] — when fromBlock sits below the node's history-pruning cutoff, a check geth applies in eth/filters/api.go. The execution-apis spec declares 4444 for eth_getLogs [6]. No width fixes that one.

What this does not fix

Range budgeting does not recover data the provider no longer serves. The pruned-history errors are the concrete case: below the cutoff, the answer is a different endpoint, not a smaller range [12][14].

It does not resume a dropped subscription from the right height. Filling the gap after a WebSocket drop is a separate problem with its own bookkeeping, covered in WebSocket reconnection without losing events.

It does not make a replay safe. Idempotent writes are what stop a re-run from double-counting, and that is a property of your write path, not of your range width.

And it does not tell you a range is complete. Completeness is a check against the checkpoint, not a consequence of the budget.

Where this leaves you

The order of work is fixed by what each step depends on. Measure the ceiling per chain, because it is the only input you cannot derive. Set width and concurrency against it, with retries counted as spend. Checkpoint before the first range commits, so the first restart is cheap. Then run.

A backfill started before the ceiling is known is a backfill you will restart. The measurement is an hour; the restart is the afternoon.

Read the rate-limit guide

Sources

1. go-ethereum eth/filters/filter_system.go — Config.RangeLimit default (master branch, retrieved 2026-11-09) — ethereum/go-ethereum (GitHub), 2026-11-09
2. go-ethereum eth/filters/filter.go — Filter.Logs range-limit check (master branch, retrieved 2026-11-09) — ethereum/go-ethereum (GitHub), 2026-11-09
3. Chainstack docs — Throughput guidelines: EVM Range limits — Chainstack Developer Portal, 2026-11-09
4. Alchemy docs — eth_getLogs: block ranges by plan tier, 150MB response cap — Alchemy Docs, 2026-11-09
5. Infura docs — Ethereum eth_getLogs: Constraints and error bodies — Infura Documentation, 2026-09-24
6. execution-apis src/eth/filter.yaml — eth_getLogs declares error 4444 Pruned history unavailable — ethereum/execution-apis (GitHub), 2026-11-09
7. Chainstack docs — Understanding eth_getLogs limitations — Chainstack Developer Portal, 2026-11-09
8. SwiftNodes /errors — eth_getLogs block range too large: the exact messages — SwiftNodes, 2026-09-24
9. Chainstack docs — eth_getLogs | Ethereum (block-range cap by plan) — Chainstack Developer Portal, 2026-11-09
10. Alchemy docs — Throughput: CU/s limits over a 10-second rolling window — Alchemy Docs, 2026-11-09
11. Infura docs — Avoid rate limiting: the 429 error body — Infura Documentation, 2026-09-24
12. go-ethereum core/history/historymode.go — PrunedHistoryError text and code 4444 (master, retrieved 2026-11-09) — ethereum/go-ethereum (GitHub), 2026-11-09
13. go-ethereum trie/errors.go — MissingNodeError message format (master, retrieved 2026-11-09) — ethereum/go-ethereum (GitHub), 2026-11-09
14. go-ethereum triedb/database/database.go — ErrStateUnavailable "state not available" (master, retrieved 2026-11-09) — ethereum/go-ethereum (GitHub), 2026-11-09
15. SwiftNodes /errors — missing trie node: historical state pruned — SwiftNodes, 2026-09-23
16. go-ethereum eth/state_accessor.go — reexecLimit = 128 ancestor blocks for missing historical state — ethereum/go-ethereum (GitHub), 2026-11-09

This article was researched, written and edited by AI under human supervision, and checked against Rankwrite's public editorial rubric.