Skip to main content
Waterline uses language models for three distinct tasks that have very different cost and quality requirements. Rather than locking you into a single model for everything, Waterline lets you route each task tier to its own provider and model. This page explains how those tiers work, how to configure each provider, and how to get the best value out of your setup.

​
The three LLM tiers

TaskEnv varVolumeRecommended model
Semantic diff — summarizes what changed in a commit or PRLLM_PROVIDERLow — once per commitClaude Sonnet or GPT-4o
Ticket analysis — relevance scoring, criteria mappingANALYSIS_LLM_PROVIDERMedium — once per analysis runGPT-4o-mini or Claude Haiku
Symbol summarization — one call per function/class at index timeSYMBOL_LLM_PROVIDERHigh — thousands of calls for a typical repoClaude Haiku or GPT-4o-mini
Each tier falls back to the one above it if unset: SYMBOL_LLM_PROVIDER falls back to ANALYSIS_LLM_PROVIDER, which falls back to LLM_PROVIDER. You can start with just LLM_PROVIDER to get everything working, then add the cheaper tiers later to reduce costs.

​
Provider setup

Set up Waterline to use Claude for all three tiers. Using Claude Haiku for symbol summarization dramatically reduces your indexing cost compared to Sonnet.
LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_MODEL=claude-3-7-sonnet-latest

ANALYSIS_LLM_PROVIDER=anthropic
ANALYSIS_ANTHROPIC_MODEL=claude-haiku-4-5-20251001

SYMBOL_LLM_PROVIDER=anthropic
SYMBOL_ANTHROPIC_MODEL=claude-haiku-4-5-20251001
Anthropic does not provide an embedding API. Even when you use Claude for all LLM tasks, you must also set OPENAI_API_KEY and configure the embedding variables:
EMBEDDING_PROVIDER=openai
EMBEDDING_MODEL=text-embedding-3-small
OPENAI_API_KEY=sk-...
This is the configuration used in production for the Waterline hosted version. Claude Haiku handles the high-volume symbol summarization at low cost, GPT-4o-mini handles analysis tasks, and Claude Sonnet handles the low-volume semantic diff work where quality matters most. 
# Embeddings — OpenAI only (Anthropic doesn't provide an embedding API)
EMBEDDING_PROVIDER=openai
EMBEDDING_MODEL=text-embedding-3-small
OPENAI_API_KEY=sk-...

# Symbol indexing — Claude Haiku (~30x cheaper than Sonnet for bulk calls)
SYMBOL_LLM_PROVIDER=anthropic
SYMBOL_ANTHROPIC_MODEL=claude-haiku-4-5-20251001
ANTHROPIC_API_KEY=sk-ant-...

# Ticket analysis — GPT-4o-mini (fast and reliable for structured output)
ANALYSIS_LLM_PROVIDER=openai
ANALYSIS_OPENAI_MODEL=gpt-4o-mini

# Semantic diff / general — Claude Sonnet (best quality for low-volume tasks)
LLM_PROVIDER=anthropic
ANTHROPIC_MODEL=claude-3-7-sonnet-latest

​
Cost optimization tips

  • SYMBOL_LLM_PROVIDER drives the most LLM spending during a first-time index. Always point it at the cheapest capable model — Claude Haiku and GPT-4o-mini both work well.
  • LLM_PROVIDER (semantic diff) runs infrequently. This is the right place to use a higher-quality model without worrying about cost.
  • You can skip the analysis and symbol tier variables entirely when starting out. Set only LLM_PROVIDER and come back to split the tiers once your bill gives you a reason to.
  • For REPO_MAX_FILES and REPO_MAX_SYMBOLS, see the environment variables reference — these limits cap how many LLM calls a single repo index can trigger.