How do I call a large language model from my backend?

To call a large language model from your backend, you usually:

1. Pick a model provider

   • OpenAI, Anthropic, Google, Azure OpenAI, etc.
   • Or self-host an open-source model.

2. Store your API key securely

   • Put it in backend environment variables.
   • Never expose it in frontend code.

3. Make an HTTP request from your server

   • Send the user’s prompt to the model API.
   • Receive the generated text as a response.

4. Return the result to your frontend

   • Your backend acts as a proxy between the user and the model.

Example: Node.js backend with OpenAI

import express from "express";
import OpenAI from "openai";

const app = express();
app.use(express.json());

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

app.post("/chat", async (req, res) => {
  try {
    const { message } = req.body;

    const response = await client.responses.create({
      model: "gpt-4.1-mini",
      input: message,
    });

    res.json({
      reply: response.output_text,
    });
  } catch (err) {
    console.error(err);
    res.status(500).json({ error: "Failed to get model response" });
  }
});

app.listen(3000, () => console.log("Server running on port 3000"));

Example: Python backend with OpenAI

from openai import OpenAI
from flask import Flask, request, jsonify

app = Flask(__name__)
client = OpenAI()

@app.post("/chat")
def chat():
    try:
        message = request.json["message"]
        response = client.responses.create(
            model="gpt-4.1-mini",
            input=message
        )
        return jsonify({"reply": response.output_text})
    except Exception as e:
        print(e)
        return jsonify({"error": "Failed to get model response"}), 500

if __name__ == "__main__":
    app.run(port=3000)

Important backend best practices

• Keep API keys server-side only
• Validate and sanitize user input
• Add rate limiting to avoid abuse
• Log requests carefully, avoiding sensitive data
• Set timeouts and retries for reliability
• Stream responses if you want faster perceived latency
• Cache repeated queries if appropriate

Typical architecture

Frontend → Your backend → LLM provider API → Your backend → Frontend

This setup lets you:

• protect your API key
• enforce auth and usage limits
• add business logic before/after the model call

If you want, I can also show:

• a streaming version
• how to do this with FastAPI, Express, Django, or Next.js
• how to call Anthropic or Azure OpenAI instead