Making Your AI Chatbot Smarter with OpenAI API & Machine Learning

This tutorial upgrades your basic Python chatbot into a smarter assistant by integrating the OpenAI Chat API, using embeddings + FAISS for context retrieval (RAG), and showing optional local-model fallbacks. Code examples are beginner-friendly and ready for experimentation.

Prerequisites

Python 3.8+
An OpenAI API key (set in OPENAI_API_KEY environment variable)
Basic Python knowledge and prior chatbot from Part 1
Optional: faiss-cpu and flask if you follow RAG + deployment

Note: Don’t hard-code API keys into your code. Use environment variables or a secure secrets manager.

Install required packages

pip install openai flask python-dotenv faiss-cpu tiktoken

For a local-model fallback (optional):

pip install transformers sentence-transformers torch

1. Simple Chat via OpenAI Chat API (Python)

Below is a minimal example showing how to call the Chat Completions endpoint (Chat API). This uses the OpenAI Python client. Replace environment key appropriately.

import os
import openai

# Set API key in environment: export OPENAI_API_KEY="sk-..."
openai.api_key = os.getenv("OPENAI_API_KEY")

def ask_openai_system(user_message, system_prompt="You are a helpful assistant."):
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_message}
    ]
    resp = openai.ChatCompletion.create(
        model="gpt-4o-mini",    # replace with available model like "gpt-4o-mini" or "gpt-4o" or "gpt-4"
        messages=messages,
        max_tokens=512,
        temperature=0.2
    )
    return resp.choices[0].message["content"].strip()

# Example:
print(ask_openai_system("Explain recursion in simple terms."))

Tips: Use a concise system prompt to control style, and tune temperature for creativity vs determinism.

2. Retrieval-Augmented Generation (RAG) — Why and How

RAG boosts your chatbot’s factual accuracy by retrieving relevant documents (user manuals, knowledge base, FAQs) and providing them as context to the model. We’ll:

Create embeddings for knowledge documents using OpenAI embeddings
Index them with FAISS
On each user query, find nearest documents and include them in the prompt

2.1 Create embeddings & index with FAISS (sample)

import os
import openai
import faiss
import numpy as np
from typing import List

openai.api_key = os.getenv("OPENAI_API_KEY")

def get_embedding(text: str, model: str = "text-embedding-3-small"):
    resp = openai.Embedding.create(model=model, input=text)
    return np.array(resp.data[0].embedding, dtype=np.float32)

# Example: build index from documents
docs = [
    {"id": "doc1", "text": "Techverse is a blog about coding and AI by Vijay."},
    {"id": "doc2", "text": "To install Python, visit python.org and download the installer."},
    # add more docs...
]

# Build embeddings matrix
embeddings = [get_embedding(d["text"]) for d in docs]
dim = len(embeddings[0])
index = faiss.IndexFlatL2(dim)
index.add(np.vstack(embeddings))

# Store a parallel list of doc ids/content for retrieval mapping
doc_texts = [d["text"] for d in docs]

2.2 Querying the index & composing the prompt

def retrieve_docs(query: str, k: int = 3):
    q_emb = get_embedding(query)
    D, I = index.search(np.array([q_emb]), k)
    hits = [doc_texts[i] for i in I[0]]
    return hits

def ask_rag(query: str):
    docs = retrieve_docs(query, k=3)
    context = "\n\n---\n\n".join(docs)
    system_prompt = "You are an assistant that can use the provided context to answer questions. If the answer is not in the context, say you don't know."
    user_message = f"Context:\n{context}\n\nQuestion: {query}"
    return ask_openai_system(user_message, system_prompt)

# Example:
print(ask_rag("How do I install Python on Windows?"))

This approach keeps answers grounded to your documents and reduces hallucinations.

3. Full Example: Flask-based Chat API (Webhook)

Expose a simple HTTP endpoint that your UI (web, Telegram bot) can call. This example demonstrates a RAG-backed chat endpoint.

from flask import Flask, request, jsonify
import os

app = Flask(__name__)

@app.route("/chat", methods=["POST"])
def chat():
    data = request.json
    user_text = data.get("message", "")
    # Use ask_rag from earlier (ensure index built at app startup)
    answer = ask_rag(user_text)
    return jsonify({"answer": answer})

if __name__ == "__main__":
    # Build FAISS index here once at startup...
    app.run(host="0.0.0.0", port=5000)

In production, secure the endpoint (API key or auth token), add rate limiting, and deploy to a hosted service like Render, Heroku, or a VPS.

4. Optional: Use a Local Model (Hugging Face) as Fallback

If you want on-device inference or want to avoid sending every prompt to OpenAI, you can run a local smaller model as fallback. This is slower and resource-heavy but possible.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "tiiuae/falcon-7b-instruct"  # example — check model size and resources
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")

def ask_local(prompt):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=256)
    return tokenizer.decode(out[0], skip_special_tokens=True)

Note: Many high-quality models require GPUs and substantial RAM. For small experimentation, consider tiny models from Hugging Face or use hosted inferencing.

5. Testing, Safety & Best Practices

Limit tokens: Use sensible max_tokens and an appropriate context window.
System prompt: Use a clear system prompt to set assistant behavior.
Sanitize user input: Prevent prompt injection by validating and escaping user content when concatenating with system prompts.
Cost control: Cache repeated answers and use embeddings-only for cheap retrieval where possible.
Privacy: Don’t send sensitive user data to external APIs without consent.
Monitoring: Log unusual responses and rate-limit usage.

6. Advanced ideas to iterate on

Fine-tuning / Instruction tuning: Train a specialized model with your dataset for higher domain accuracy (requires more infra).
Tools & function-calling: Use function-calling features to let the model trigger controlled code paths (for calculators, DB queries, or file access).
Conversational memory: Keep a short summary of the conversation as context rather than full history to save tokens.
Multimodal inputs: Add images or files for richer interactions (where supported by model).

7. Common Errors & Troubleshooting

401 / Invalid API Key: Check environment variables and key permissions.
Rate limit errors: Back off and implement retry with exponential backoff.
High latency: Cache embeddings, reduce model size, or use streaming where available.
Hallucinations: Use RAG and clearer system prompts; verify critical facts with external sources.

8. Costs & Privacy Reminder

Using hosted LLMs costs money. Monitor usage and set budgets. If your chatbot will handle user data, ensure compliance with privacy regulations and inform users how their data is used.

Quick checklist before deploying:

Use environment variables for keys
Sanitize inputs and limit length
Add authentication to endpoints
Log and monitor usage and errors
Inform users about data handling

Conclusion

You’ve gone from a rule-based Python chatbot to a smarter assistant powered by the OpenAI Chat API and a retrieval layer (RAG) using embeddings and FAISS. This architecture is practical, scalable, and greatly reduces hallucinations by grounding answers in real documents.

If you want, I can now:

Produce a ready-to-paste Flask + HTML chatbot UI that calls your /chat endpoint
Create a step-by-step walkthrough of deploying this bot to a cloud service (Render / Vercel / Heroku)
Build a Part 3 showing how to connect the bot to Telegram or Discord

Which of these would you like next?

Search This Blog

TechVerse by Vijay

Making the Chatbot Smarter with OpenAI API and Machine Learning

Techverse by Vijay