Most teams start LLM integration the same way: a direct OpenAI API call in a route handler, no retry logic, no cost cap, no evaluation. This works fine in demos and breaks predictably in production. Here are the patterns we have settled on after shipping more than 60 AI-powered features.
Structured outputs over raw completions
Use response_format: { type: 'json_schema' } on every call that produces data your application consumes. Parsing free-form completions with regex is a maintenance nightmare. Structured outputs give you a contract the model is penalised for violating.
RAG over fine-tuning for domain knowledge
Unless you are changing model behaviour (tone, reasoning style, task specialisation), RAG is almost always faster to ship and cheaper to maintain than fine-tuning. A well-built retrieval pipeline with chunking, embedding, and reranking will outperform a fine-tuned model on factual Q&A tasks while staying current as your data changes.
Cost and latency guardrails
Set per-user token budgets enforced server-side. Cache embeddings aggressively — recomputing the same 1,000-token chunk repeatedly is waste. Use streaming for any response longer than two sentences to avoid perceived latency.