GraphRAG Global Search Fan-Out

Global search is not a search. It does not look for anything. It reads a community report for every entity in your corpus, splits the pile into context-window-sized batches, and spends one model call on each — then one more to merge the answers. Ask a narrower question and it does exactly the same work.

This is the design, not a bug. The GraphRAG paper frames global questions as query-focused summarisation, not retrieval: "each community summary is used to generate a partial response, before all partial responses are again summarized in a final response to the user." Summarising a corpus means reading the corpus. The consequence people do not price in is that the unit of work is the corpus, so cost and latency are a property of your index and are the same for every question you will ever ask it.

The panel below builds the Leiden hierarchy, rolls it up to the level you ask for, packs the reports into batches and counts the calls. Move the question radio first. Watch every number stay still.

Now move corpus documents from 20 to 4,000 with the question left alone. The call count climbs the whole way. The question never entered the calculation: batching happens in build_community_context, which receives the report list and the token budget and nothing else. There is no similarity step, no top-k, no filter — the reports are shuffled with a fixed seed and packed greedily until the budget is hit. This is what separates it from an ordinary vector retriever, which answers every question with the same one call no matter how large the corpus grows.

The roll-up covers the corpus, not the query

Ask for --community-level 2 and it is tempting to read that as "give me the level-2 summaries" — a bounded, shallow set. It is not a filter in that sense. The loader's own docstring says what it does: "select reports with the max community level that an entity belongs to." It walks every entity, finds the deepest community that entity sits in at or below your level, and keeps that community's report. Union those and you have a covering set: every entity is represented exactly once.

So the report count is not the number of communities at level 2. It is the number of leaves of the hierarchy truncated at level 2 — which also picks up small communities that were never split at all, because they were already under max_cluster_size and had nowhere to go. Move the level slider across its range with everything else fixed: at 400 documents the same index costs 4 calls at level 0 and 31 at level 3, an eightfold spread on a flag most people set once and forget. The CLI default is 2.

Now try max_cluster_size, and watch the thing that does not happen. Take it from 5 to 40 and the report count falls by more than half — but the token total and the call count barely move. Report length tracks community size, so coarser communities mean fewer, longer reports and roughly the same number of bytes. The parameter that looks like a cost knob is not one, and the parameter documented as a clustering knob (--community-level) is.

Level 0 buys its cheapness by throwing information away rather than by reading less of the corpus. A level-0 community may hold several hundred entities, and its report is still capped at 2,000 tokens by community_reports.max_length. You are not summarising less; you are compressing far more into the same space, and the answer thins out accordingly. Cheap and shallow, or expensive and specific — that is the actual trade the level flag controls, and the token counter is the only place it is visible.

The reduce step throws away most of what you paid for

Each map call returns a JSON list of points, each with a description and an importance score. The reduce step collects points from every batch, drops anything scored zero, sorts the rest by score descending, and then packs them into the reduce prompt until data_max_tokens — also 12,000 by default — is exhausted. The loop ends with a plain break. Everything after the break is discarded.

Watch the map output discarded readout as you grow the corpus. At a few hundred documents almost every point survives. At a few thousand, most of the map stage is bought and thrown away, and the warm bars in the batch chart are calls that contributed literally nothing to the answer. The reduce budget is a constant; the map stage is linear in corpus size; the gap between them is pure waste, and it widens with every document you index.

The ranking that decides what survives is also weaker than it looks. Scores are assigned independently inside each batch, by a model that can see only that batch. Batch 3's "score 90" and batch 61's "score 90" were produced by different contexts with no shared scale — the reduce step sorts them against each other anyway. This is the same context-window arithmetic that governs how much you can hold in one prompt at all: the constraint is real, the workaround is a heuristic, and the heuristic degrades quietly as the corpus grows.

There is a failure mode worth knowing here. If a map batch returns malformed JSON, the code logs Warning: Error parsing search response json - skipping this batch and moves on with an empty result. A batch you paid full price for vanishes from the answer, and nothing in the response tells you it happened.

Dynamic community selection, and the question it cannot save

Enable dynamic_community_selection and the query finally enters the calculation. A breadth-first walk starts at the level-0 communities, asks the model to rate each report's relevance, and descends into the children of anything scoring at or above dynamic_search_threshold. Each community it keeps discards its parent from the set, so the walk converges on the most specific report that still cleared the bar.

Push the corpus to 4,000 documents and select "Who is Ada Okafor?". The reports read collapse from about 1,500 to a handful — and the call count only falls from roughly 140 to 100. Rating is itself a model call per community, so what you save on the map stage you spend on the walk down. Dynamic selection is not a retrieval index; it is a cheaper linear scan.

Now switch the question back to "What are the main themes?" with dynamic selection still on, and read the log. Almost every community rates as relevant, because for a genuine sensemaking question almost every community is relevant. The walk descends everywhere: the reports read drop by about 5%, the call count goes up roughly fourteenfold, and the log says so in one line. The pruning step made the exact query type GraphRAG was built for far more expensive, not less.

Raise dynamic_search_threshold to 4 or 5 on the needle question and you find the other edge. When nothing at level 0 clears the bar and the queue empties with no relevant community found, the algorithm does not give up — it enqueues every community at the next level and rates them all, up to dynamic_search_max_level. A stricter threshold, set to save money, triggers a fan-out that costs more than not pruning at all. The log prints the moment it happens.

When the plain retriever wins

A conventional retriever answers with one model call and one context window, forever, regardless of corpus size. Global search answers with O(corpus / context window) calls. That ratio is the entire decision, and it resolves the same way most of the time:

• The question names an entity, a document or a span → the graph buys you nothing that a good retriever does not. Use local search, or skip the graph entirely.
• The question has an answer that exists in three chunks scattered across the corpus → this is a ranking problem, not a summarisation problem.
• The question is genuinely about the shape of the whole corpus — themes, coverage, what is missing, how positions changed — and no single chunk contains the answer → global search is doing something a retriever structurally cannot, and the fan-out is the price of it.

The honest framing: global search is a batch analytics job with a chat interface. Price it per run, cache it, schedule it. Treating it as a query endpoint is what produces the "why is my GraphRAG bill like this" thread. And note that all of the above is the query side only — building the index is a separate bill that scales with chunks and extraction passes, and it is usually the larger of the two.

Checking it yourself

Before you run anything, count your report set. Load community_reports.parquet, filter to level at or below the level you intend to query, and look at the row count. That number, times your average report length, divided by max_context_tokens, is your map call count. It is knowable in advance and almost nobody computes it.

At runtime, the result object carries the accounting: a GlobalSearchResult exposes llm_calls, prompt_tokens and output_tokens, and each map batch reports llm_calls=1. Sum them and compare with the estimate above. With dynamic selection on, turn the log level up to DEBUG and look for the line beginning dynamic community selection (took: …) — it prints the rating distribution and "N out of M community reports are relevant". If N is close to M, dynamic selection is costing you and saving nothing; turn it off.

If the answer comes back as a canned refusal, look for the warning All map responses have score 0. That is not a retrieval miss in the usual sense — every report in your corpus was read and every one was judged irrelevant, which almost always means the question was a lookup and should never have gone through global search. The same symptom from a broken graph is a different problem entirely; see what happens when the same entity became three nodes.

Next: the mechanism that decides how many communities you get in the first place, hierarchical Leiden and max_cluster_size, and the failure that makes all of this fan-out read a graph that is quietly wrong, entity resolution.