The shape of an LLM API call
Every provider (OpenAI, Anthropic, Google) exposes a similar chat completions interface. You send a list of messages with roles — system (stable instructions/persona), user (the input), assistant (prior model turns) — and get back a completion. Multi-turn conversation = you resend the growing message history each call (the API is stateless; you hold the conversation state).
The parameters that matter
- model — which model (and version). Pin it; behavior shifts across versions.
- temperature (0–1+) — randomness. Low (0–0.3) for deterministic/structured tasks (extraction, classification, code); higher for creative work.
- max_tokens — cap the output length (cost + runaway control).
- top_p — nucleus sampling (usually leave default; tune temperature or top_p, not both).
- stop sequences — end generation at a marker.
- system prompt — the durable behavior spec (Prompt Engineering course).
Use the SDK, not raw HTTP
Providers ship official SDKs (Python, TypeScript) that handle auth, retries with backoff, streaming, and typed responses. Use them — and keep the API key server-side only (never in the client; all model calls go through your backend, per Foundations/AI Security). The response includes the content plus usage (input/output token counts) — log it for cost tracking (Cost course).
Token limits & context windows
Each model has a context window (max input+output tokens). Exceed it and the call errors. Budget your prompt: system + history + retrieved context + expected output must fit. Long conversations need history management (summarize/truncate — Cost course).
Reasoning about the response
The model returns a finish reason (stop = done, length = hit max_tokens and got cut off, tool_use = wants to call a tool). Check it — a cutoff means your answer is truncated and you need a higher max_tokens or a shorter task.