SHIPIT

LLM providers

shipit_agent runs the SAME agent code on OpenAI, Anthropic, AWS Bedrock, Google Vertex, Gemini, Groq, Together, Ollama, and any LiteLLM-reachable model. Pick a provider two ways — a direct adapter class, or build_llm_from_env() driven by SHIPIT_LLM_PROVIDER — and swap providers in one line.

3 min read
12 sections
Edit this page

shipit_agent is provider-agnostic. The Agent, DeepAgent, tools, streaming, RAG, caching, and tracing layers never care which model is behind the LLM — they only talk to the small LLM interface. That means you write your agent once and run it on whichever provider you want.

There are two ways to get an LLM:

  1. Direct adapter class — import the adapter and construct it. Explicit, great for code that pins one provider.
  2. build_llm_from_env() — set SHIPIT_LLM_PROVIDER (plus that provider's credentials) in your .env and let the factory pick the adapter. Great for "swap providers without touching code."

Both come from shipit_agent.llms:

python
from shipit_agent.llms import (
    build_llm_from_env,
    OpenAIChatLLM,
    AnthropicChatLLM,
    BedrockChatLLM,
    VertexAIChatLLM,
    GeminiChatLLM,
    GroqChatLLM,
    TogetherChatLLM,
    OllamaChatLLM,
    LiteLLMChatLLM,
    LiteLLMProxyChatLLM,
)

Swap providers in one line

The rest of your agent code is identical regardless of provider. Only the one line that builds the LLM changes:

python
from shipit_agent import Agent
from shipit_agent.llms import build_llm_from_env

llm = build_llm_from_env("openai")     # ← change "openai" to any provider below
# everything from here down never changes:
agent = Agent.with_builtins(llm=llm)
print(agent.run("Find today's Bitcoin price in USD.").output)

Or change nothing in code at all — set SHIPIT_LLM_PROVIDER in .env and call build_llm_from_env() with no argument:

bash
# .env
SHIPIT_LLM_PROVIDER=anthropic     # bedrock|openai|anthropic|gemini|vertex|groq|together|ollama|litellm
python
llm = build_llm_from_env()             # reads SHIPIT_LLM_PROVIDER from .env

Default provider

If SHIPIT_LLM_PROVIDER is unset, the factory defaults to bedrock. Pass an explicit provider (build_llm_from_env("openai")) to override .env for a single call.

Every supported provider

ProviderSHIPIT_LLM_PROVIDERAdapter classRequired env varsDefault model
OpenAIopenaiOpenAIChatLLMOPENAI_API_KEYgpt-4o-mini
AnthropicanthropicAnthropicChatLLMANTHROPIC_API_KEYclaude-3-5-sonnet-latest
AWS BedrockbedrockBedrockChatLLMAWS_REGION_NAME (or AWS_DEFAULT_REGION) or AWS_PROFILEbedrock/openai.gpt-oss-120b-1:0
Google VertexvertexVertexAIChatLLMcreds file + VERTEXAI_PROJECT + VERTEXAI_LOCATIONvertex_ai/gemini-1.5-pro
Google GeminigeminiGeminiChatLLMGEMINI_API_KEY or GOOGLE_API_KEYgemini/gemini-1.5-pro
GroqgroqGroqChatLLMGROQ_API_KEYgroq/llama-3.3-70b-versatile
TogethertogetherTogetherChatLLMTOGETHERAI_API_KEY or TOGETHER_API_KEYtogether_ai/meta-llama/Llama-3.1-70B-Instruct-Turbo
Ollama (local)ollamaOllamaChatLLMnone — needs a local Ollama serverollama/llama3.1
LiteLLM (proxy to ~everything)litellmLiteLLMChatLLM / LiteLLMProxyChatLLMSHIPIT_LITELLM_MODEL (+ provider creds / SHIPIT_LITELLM_API_BASE)none — SHIPIT_LITELLM_MODEL required

Per-provider init snippets follow. Each shows both the direct adapter class and the matching build_llm_from_env() env-var setup.

OpenAI

python
from shipit_agent.llms import OpenAIChatLLM, build_llm_from_env

# Direct:
llm = OpenAIChatLLM(model="gpt-4o-mini")
# Force at least one tool call per turn (lazy models):
llm = OpenAIChatLLM(model="gpt-4o-mini", tool_choice="required")

# From env:
llm = build_llm_from_env("openai")
bash
SHIPIT_LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
SHIPIT_OPENAI_MODEL=gpt-4o-mini          # optional
SHIPIT_OPENAI_TOOL_CHOICE=               # optional — "required" forces a tool call

OpenAI does automatic prompt caching; shipit surfaces the cache reads. See Prompt caching.

Anthropic

python
from shipit_agent.llms import AnthropicChatLLM, build_llm_from_env

# Direct (prompt caching on by default for Claude):
llm = AnthropicChatLLM(model="claude-sonnet-4-20250514")
# Extended thinking:
llm = AnthropicChatLLM(model="claude-opus-4-1", thinking_budget_tokens=2048)

# From env:
llm = build_llm_from_env("anthropic")
bash
SHIPIT_LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-...
SHIPIT_ANTHROPIC_MODEL=claude-3-5-sonnet-latest    # optional

AWS Bedrock

Bedrock authenticates with AWS credentials, not an API key — a region plus your standard AWS access keys, an AWS_PROFILE, or an instance role.

python
from shipit_agent.llms import BedrockChatLLM, build_llm_from_env

# Direct:
llm = BedrockChatLLM(model="bedrock/openai.gpt-oss-120b-1:0")
# Claude on Bedrock (tool-pairing handled automatically):
llm = BedrockChatLLM(model="bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")

# From env:
llm = build_llm_from_env("bedrock")
bash
SHIPIT_LLM_PROVIDER=bedrock
AWS_REGION_NAME=us-east-1                 # or AWS_DEFAULT_REGION, or set AWS_PROFILE
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_SESSION_TOKEN=                        # optional
SHIPIT_BEDROCK_MODEL=bedrock/openai.gpt-oss-120b-1:0    # optional

Google Vertex AI

Vertex needs a service-account JSON file plus a project and a location.

python
from shipit_agent.llms import VertexAIChatLLM, build_llm_from_env

# Direct:
llm = VertexAIChatLLM(
    model="vertex_ai/gemini-1.5-pro",
    service_account_file="/path/to/sa.json",
    project_id="my-gcp-project",
    location="us-central1",
)

# From env:
llm = build_llm_from_env("vertex")
bash
SHIPIT_LLM_PROVIDER=vertex
GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json   # or SHIPIT_VERTEX_CREDENTIALS_FILE
VERTEXAI_PROJECT=my-gcp-project                    # or GOOGLE_CLOUD_PROJECT
VERTEXAI_LOCATION=us-central1                      # or VERTEX_LOCATION / GOOGLE_CLOUD_LOCATION
SHIPIT_VERTEX_MODEL=vertex_ai/gemini-1.5-pro       # optional

The Vertex adapter handles the full Vertex catalog — Gemini, Claude-on-Vertex, Llama-on-Vertex, etc. Claude-on-Vertex gets cache_control prompt caching too.

Google Gemini

python
from shipit_agent.llms import GeminiChatLLM, build_llm_from_env

# Direct:
llm = GeminiChatLLM(model="gemini/gemini-1.5-pro")

# From env:
llm = build_llm_from_env("gemini")
bash
SHIPIT_LLM_PROVIDER=gemini
GEMINI_API_KEY=...                        # or GOOGLE_API_KEY
SHIPIT_GEMINI_MODEL=gemini/gemini-1.5-pro    # optional

Groq

python
from shipit_agent.llms import GroqChatLLM, build_llm_from_env

# Direct:
llm = GroqChatLLM(model="groq/llama-3.3-70b-versatile")

# From env:
llm = build_llm_from_env("groq")
bash
SHIPIT_LLM_PROVIDER=groq
GROQ_API_KEY=gsk_...
SHIPIT_GROQ_MODEL=groq/llama-3.3-70b-versatile    # optional

Together AI

python
from shipit_agent.llms import TogetherChatLLM, build_llm_from_env

# Direct:
llm = TogetherChatLLM(
    model="together_ai/meta-llama/Llama-3.1-70B-Instruct-Turbo"
)

# From env:
llm = build_llm_from_env("together")
bash
SHIPIT_LLM_PROVIDER=together
TOGETHERAI_API_KEY=...                    # or TOGETHER_API_KEY
SHIPIT_TOGETHER_MODEL=together_ai/meta-llama/Llama-3.1-70B-Instruct-Turbo    # optional

Ollama (local models)

No API key — just a running local Ollama server.

python
from shipit_agent.llms import OllamaChatLLM, build_llm_from_env

# Direct:
llm = OllamaChatLLM(model="ollama/llama3.1")

# From env:
llm = build_llm_from_env("ollama")
bash
SHIPIT_LLM_PROVIDER=ollama
SHIPIT_OLLAMA_MODEL=ollama/llama3.1       # optional

LiteLLM (proxy to ~everything)

LiteLLM routes to ~100 providers behind one interface. Use the direct LiteLLMChatLLM for any LiteLLM-format model string, or LiteLLMProxyChatLLM to point at a self-hosted LiteLLM proxy gateway. With build_llm_from_env, the factory picks LiteLLMProxyChatLLM automatically when SHIPIT_LITELLM_API_BASE is set, otherwise LiteLLMChatLLM.

python
from shipit_agent.llms import (
    LiteLLMChatLLM,
    LiteLLMProxyChatLLM,
    build_llm_from_env,
)

# Direct — any LiteLLM model string:
llm = LiteLLMChatLLM(model="anthropic/claude-sonnet-4-20250514")

# Direct — self-hosted LiteLLM proxy gateway:
llm = LiteLLMProxyChatLLM(
    model="gpt-4o-mini",                            # whatever the proxy routes to
    api_base="https://litellm.my-company.internal",
    api_key="sk-proxy-token",
    custom_llm_provider="openai",                   # proxy speaks OpenAI format
)

# From env (LiteLLMChatLLM, or LiteLLMProxyChatLLM if SHIPIT_LITELLM_API_BASE set):
llm = build_llm_from_env("litellm")
bash
SHIPIT_LLM_PROVIDER=litellm
SHIPIT_LITELLM_MODEL=anthropic/claude-sonnet-4-20250514    # required
SHIPIT_LITELLM_API_KEY=                                    # optional
SHIPIT_LITELLM_API_BASE=                                   # set → LiteLLMProxyChatLLM
SHIPIT_LITELLM_CUSTOM_PROVIDER=                            # optional

LiteLLM forwards both cache-token shapes (Anthropic cache_read_input_tokens and OpenAI prompt_tokens_details.cached_tokens), so cross-provider cost accounting keeps working through the proxy.

See also

  • Environment setup — .env discovery, credential requirements, and switching providers without restarting.
  • Quick start — get an agent running in five minutes on any provider.
  • Prompt caching — caching works across Anthropic, Bedrock, Vertex (cache_control), and OpenAI (automatic); LiteLLM forwards both.