Skip to main content

​
Mission

ai_sdk aims to minimise boilerplate when working with modern LLMs while staying provider-agnostic and fully typed.
  1. One-line helpers (generate_text, embed_many, …) do 90 % of common work.
  2. Abstract base classes (LanguageModel, EmbeddingModel) keep providers pluggable.
  3. Pydantic models enforce type-safety for structured output & internal messages.
  4. Everything is sync-first for notebooks/scripts but exposes async streaming where it matters.

​
High-level flow

1

Factory helper returns a provider wrapper

model = openai("gpt-4.1-mini")
2

Helper builds a provider-agnostic request

The SDK translates prompt/system/messages/tools into the provider’s native schema.
3

Provider SDK call & response normalisation

Each provider implementation maps the raw response back into a lightweight dict containing text, finish_reason, usage, raw_response, …
4

Result objects give you typed access

GenerateTextResult
├─ text: str
├─ usage: TokenUsage
└─ tool_calls / tool_results (optional)

​
Where does retry logic live?

  • Embeddings - batching & retries happen in the helper (embed_many).
  • Text generation - delegate retries to the provider’s SDK since most include back-off handlers.

​
Extending the SDK

  1. Implement LanguageModel or EmbeddingModel.
  2. Expose a public factory (e.g. mycloud(model="x")).
  3. Users instantly get the full helper surface - no changes elsewhere.
Keeping the abstraction layer razor-thin (less than 100 LOC per provider) makes it easy for the community to add new backends.

​
Roadmap

  • Built-in vector-store utilities
  • Native image generation helpers
  • Automatic schema extraction for generate_object
Open to contributions! Check the GitHub issues for “good first issue” labels.