Skip to main content

OpenAI

Use OpenAI when the app needs chat, summarization, generation, classification, extraction, or workflow assistance.

Example app: document chatbot​

A RAG chatbot is a useful OpenAI pattern for knowledge bases, customer support, internal research, education, and document review.

Build a document chatbot.
Users can upload PDFs, view document status, and ask questions.
Extract text from PDFs, split it into chunks, create embeddings, and store vectors.
Use OpenAI for embeddings and a configurable chat model for answers.
Return answers with cited source chunks and a clear "not found" state.

Plan the AI feature​

Before implementation, define:

  • The exact user action that calls AI.
  • Inputs sent to the model.
  • Expected output format.
  • Model budget and maximum response length.
  • Whether outputs must be reviewed before saving.
  • Data that must never be sent to the model.
OpenAI API key management page
Create a dedicated key for development and keep it out of browser code.
OpenAI API key creation page
Name keys clearly so they can be rotated by app or environment.
OpenAI API key safe storage reminder
Save generated keys only in approved secret storage.
OPENAI_API_KEY=
OPENAI_MODEL=
OPENAI_DAILY_BUDGET_USD=

For RAG workflows, also ask for variables such as:

OPENAI_EMBEDDING_MODEL=
MAX_UPLOAD_MB=
RAG_TOP_K=

Prompt Cocoding AI​

Add an AI support reply draft feature.
Use OPENAI_API_KEY on the server only.
Use OPENAI_MODEL from environment configuration.
Return a short draft, confidence score, and escalation reason.
Do not send payment data or private notes to the model.

Test the AI feature​

  1. Use sample data only.
  2. Confirm the request is made from the backend.
  3. Confirm the response follows the expected format.
  4. Add rate limiting or budget checks for repeated use.
  5. Add an error state for missing API keys or provider failures.
Generated document chat app answering a question
A document chatbot should show the user's question, the AI answer, and enough context to verify the response.

RAG workflow checklist​

  1. Upload a test PDF.
  2. Extract text.
  3. Split text into chunks.
  4. Generate embeddings.
  5. Store embeddings in the database.
  6. Convert the user question into an embedding.
  7. Retrieve the most relevant chunks.
  8. Generate an answer from those chunks.
  9. Show sources or a "not enough information" response.

Cost controls​

  • Set model and embedding model explicitly.
  • Add upload size limits.
  • Cache document chunks and embeddings.
  • Add per-user request limits.
  • Log token usage without storing private prompts unnecessarily.

Troubleshooting​

SymptomLikely causeWhat to check
API key errorMissing or invalid keyStore OPENAI_API_KEY server-side and restart.
High usageNo budget or rate limitAdd a daily budget and per-user throttling.
Bad output shapePrompt missing format rulesAsk Cocoding AI to enforce a JSON schema or validation step.
Sensitive data riskToo much context sentReduce input fields and add redaction before the AI call.