How to Install an LLM in Python: Local or API

Two meanings of installing an LLM

When people ask how to install llm in python, they usually mean one of two things. The first is running a model locally: downloading weights and loading them with a library on your own CPU or GPU. The second is installing a client library so your Python code can talk to a model that runs somewhere else. Both are valid, and the right one depends on your hardware and what you want to build.

The API route is much shorter. You run pip install openai, point the client at a base URL, and call the model. No weights, no GPU drivers, no memory planning. The rest of this page gives the steps and the code for that route, with a short section on the local one so you can compare.

Local install or hosted API: how to choose

Question	Local install	Hosted API
What you install	An inference library plus model weights	The openai package
Hardware	Memory and usually a GPU sized for the model	Any machine that can make HTTPS calls
Cost	Your hardware and electricity	Per token, from prepaid credit
Works offline	Yes	No
Time to first reply	Download, setup, tuning	A few minutes

Local makes sense if you must keep everything on one machine, need to work offline, or want to experiment with the weights themselves. Libraries such as llama.cpp bindings or Hugging Face transformers are the usual starting points; check their own documentation for hardware requirements, since those depend on the model size you pick.

The API makes sense if you want a working result quickly, your laptop cannot hold a large model, or your app needs to serve many users. You pay per token instead of buying hardware. For this service that is $0.15 per 1M input tokens and $0.50 per 1M output tokens, with no subscription. The pricing page has the details.

Step 1: create a virtual environment and install the package

Use a virtual environment so the install stays clean. Any recent Python 3 release works with the openai package.

Copypython3 -m venv .venv
source .venv/bin/activate
pip install openai

Next you need a key. Sign up on the API key page with Google or with email and password. New users can claim $0.50 of trial credit there with one click, valid for 7 days and with no card. Store the key in an environment variable instead of in your source file:

Copyexport API_KEY="paste-your-key-here"

Step 2: make your first call

The service is OpenAI-compatible, so the only changes from a stock OpenAI script are the base URL and the model ID. The model ID is uncensored.

Copyimport os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.getaiapikey.com/v1",
    api_key=os.environ["API_KEY"],
)

resp = client.chat.completions.create(
    model="uncensored",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Explain what a context window is in two sentences."},
    ],
    temperature=0.7,
    max_tokens=200,
)

print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

If this prints an answer and two token counts, the install is done. To check that your key works and see the model ID, you can also call GET /v1/models:

Copyfor m in client.models.list():
    print(m.id)

The same request from the shell, with no Python at all, is a single curl call to https://api.getaiapikey.com/v1/chat/completions with the header Authorization: Bearer followed by your key. The docs quickstart shows it in full.

Step 3: stream tokens as they arrive

Waiting for a long answer to finish before printing anything feels slow. Set stream=True and the server sends tokens over server-sent events. You can ask for a usage summary in the final chunk with stream_options.

Copystream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Write a short scene set in a lighthouse."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage.total_tokens, "tokens")

One detail matters for production: a streamed request holds its concurrency slot until the stream ends. A full key allows 8 concurrent requests, so if your app opens many long streams, close them when the user disconnects.

Step 4: JSON mode and tool calling

For structured output, set response_format to a JSON object. Also tell the model in the prompt to answer in JSON, because the setting guarantees valid JSON syntax but not the field names you want.

Copyimport json

resp = client.chat.completions.create(
    model="uncensored",
    messages=[
        {"role": "system", "content": "Reply only with JSON: {\"title\": string, \"tags\": [string]}"},
        {"role": "user", "content": "Give me a title and tags for a post about slow travel."},
    ],
    response_format={"type": "json_object"},
)
data = json.loads(resp.choices[0].message.content)
print(data["title"], data["tags"])

Tool calling follows the standard pattern. You describe a function, the model may ask to call it, you run it yourself and send the result back with role: "tool".

Copytools = [{
    "type": "function",
    "function": {
        "name": "get_time",
        "description": "Return the current time for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What time is it in Lisbon?"}]
first = client.chat.completions.create(
    model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)

msg = first.choices[0].message
if msg.tool_calls:
    call = msg.tool_calls[0]
    messages.append(msg)
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps({"time": "14:05"}),
    })
    final = client.chat.completions.create(
        model="uncensored", messages=messages, tools=tools
    )
    print(final.choices[0].message.content)

Test tool calls on a few prompts before you rely on them. Any model can pick the wrong function or fill an argument badly, so validate arguments in your own code before running anything with side effects.

Step 5: handle limits and errors

Full keys allow 8 concurrent requests and 300 requests per minute. Trial keys allow 2 and 60. Past those limits you get HTTP 429. The error code rate_limited means too many requests per minute, and concurrency means too many at once. On temporary overload the API also returns 429 with a Retry-After: 5 header.

The OpenAI Python client retries some failures on its own, but an explicit wrapper keeps behaviour obvious:

Copyimport time
from openai import RateLimitError

def ask(messages, tries=4):
    for attempt in range(tries):
        try:
            return client.chat.completions.create(model="uncensored", messages=messages)
        except RateLimitError:
            time.sleep(5 * (attempt + 1))
    raise RuntimeError("still rate limited after retries")

Each response carries X-Request-Id, X-Balance-USD and rate-limit headers, which you can log. Include the request ID if you contact support. Failed or refused requests are not charged, so retrying does not cost extra.

Also plan for the size limits. The context window is 100,000 tokens, prompt plus completion, and the maximum request body is 8 MB. Long chat histories should be trimmed or summarised before they reach those numbers.

When the call does not work

Most first-run problems come from a handful of mistakes. Check these in order before you suspect anything else.

• Base URL without the version path. The value must end in /v1. The client adds /chat/completions to it, so a missing /v1 sends the request to the wrong route.
• Wrong or missing key. The key goes in the Authorization: Bearer header, which the client builds from api_key. If the environment variable is empty, the client either raises an error at start or sends a blank key. Print bool(os.environ.get("API_KEY")) to check.
• A model name from another service. The only model ID here is uncensored. Names like the ones in other tutorials will not match.
• No balance. Look at the X-Balance-USD header. A trial that passed its 7 days, or a spent balance, needs a top-up from $20.
• Old package. If an example uses OpenAI( and Python says it cannot import that name, you have the pre-1.0 package. Run pip install --upgrade openai.

To see the raw response including headers, use the client's raw-response wrapper:

Copyraw = client.chat.completions.with_raw_response.create(
    model="uncensored",
    messages=[{"role": "user", "content": "ping"}],
    max_tokens=5,
)
print(raw.headers.get("x-request-id"), raw.headers.get("x-balance-usd"))
completion = raw.parse()
print(completion.choices[0].message.content)

That prints the request ID and your remaining balance next to the answer, which is a quick health check you can run from a cron job or a startup script.

Using it from other tools

Because the endpoint follows the OpenAI format, other Python libraries can use it with the same two settings. Frameworks that accept a custom OpenAI base URL only need base_url and api_key. For Node, the official openai package takes the same fields:

Copyimport OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.getaiapikey.com/v1",
  apiKey: process.env.API_KEY,
});

const r = await client.chat.completions.create({
  model: "uncensored",
  messages: [{ role: "user", content: "Say hello in five words." }],
});
console.log(r.choices[0].message.content);

This is a text-only service: no image or audio generation, no embeddings and no fine-tuning. If your project needs those, pair this API with another tool for that part. The model is an open-weight one run by an independent provider and tuned to answer without content refusals for lawful adult use. Sexual content involving minors is never allowed.

More guides

• LLM Free API Key: What the Trial Credit Covers
• LLM API Key for a Discord Bot: Setup Guide
• AI API Checklist