Call an uncensored LLM API from Python

What you need

• Python 3.10 or newer, which the current openai package requires. The examples were run on Python 3.12.
• An API key from a provider, or a local OpenAI-compatible server such as llama.cpp’s llama-server, vLLM or Ollama. A local server does not need a real key, but the SDK requires a non-empty string.
• The model id the provider expects. It is not always the Hugging Face repo name: compare the ids on our provider pages.

Install the SDK in a virtual environment:

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
python -m pip install openai

Keep the key out of the code

Put the three values in the environment, not in the script. That way the same file runs against any provider and never leaks a key into version control.

export LLM_BASE_URL="https://api.featherless.ai/v1"
export LLM_API_KEY="your-key-here"
export LLM_MODEL="huihui-ai/Huihui-Qwen3.8-27B-abliterated"

On Windows PowerShell use $env:LLM_API_KEY = "...". In production, use your platform’s secret store. The base URL and model above are one example; the API overview lists the documented base URL for each provider.

First request

"""One chat completion against any OpenAI-compatible API.

Set LLM_BASE_URL, LLM_API_KEY and LLM_MODEL in the environment first.
"""
import os
import sys

from openai import OpenAI, APIConnectionError, APIStatusError, APITimeoutError

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
    timeout=60.0,      # seconds for the whole request
    max_retries=2,     # the SDK retries 408, 409, 429 and 5xx with backoff
)

try:
    reply = client.chat.completions.create(
        model=os.environ["LLM_MODEL"],
        messages=[
            {"role": "system", "content": "You are a concise assistant for developers."},
            {"role": "user", "content": "Explain what a race condition is in two sentences."},
        ],
        max_tokens=300,
    )
except APITimeoutError:
    sys.exit("The request timed out. Try a smaller max_tokens or a longer timeout.")
except APIConnectionError as e:
    sys.exit(f"Could not reach {os.environ['LLM_BASE_URL']}: {e}")
except APIStatusError as e:
    # 401 bad key, 404 wrong base URL or model, 413 prompt too long, 429 rate limit
    sys.exit(f"The API returned {e.status_code}: {e.message}")

choice = reply.choices[0]
print(choice.message.content)
print(f"finish_reason={choice.finish_reason}")
if reply.usage:
    print(f"tokens: {reply.usage.prompt_tokens} in, {reply.usage.completion_tokens} out")

The request has the usual chat format: a list of messages with roles system, user and assistant. The SDK sends Authorization: Bearer <key> and posts to <base URL>/chat/completions.

Three details are worth knowing:

• finish_reason tells you why the reply ended: stop for a natural end, length when it hit max_tokens or the provider’s output cap. A length reply is cut off; do not treat it as complete.
• usage reports the tokens billed. Use real numbers from here in the token cost calculator instead of guessing.
• Retries. max_retries=2 lets the SDK retry rate limits (429) and server errors with backoff. In our test the mock server returned 429 twice and the third attempt succeeded without any code of ours.

Streaming

For anything longer than a sentence or two, stream the reply so the user sees text as it is generated. Thinking models in particular can spend many seconds reasoning before the first visible token.

"""Stream a reply token by token, then print the usage if the server sends it."""
import os

from openai import OpenAI

client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"], timeout=60.0)

stream = client.chat.completions.create(
    model=os.environ["LLM_MODEL"],
    messages=[{"role": "user", "content": "Write a haiku about version control."}],
    max_tokens=200,
    stream=True,
    stream_options={"include_usage": True},  # some servers ignore this; usage may be absent
)

parts = []
usage = None
try:
    for chunk in stream:
        if chunk.usage:
            usage = chunk.usage
        if not chunk.choices:
            continue
        delta = chunk.choices[0].delta
        if delta.content:
            parts.append(delta.content)
            print(delta.content, end="", flush=True)
finally:
    stream.close()  # releases the connection even if you stop early

print()
if usage:
    print(f"tokens: {usage.prompt_tokens} in, {usage.completion_tokens} out")
print(f"characters received: {len(''.join(parts))}")

stream_options={"include_usage": True} asks for a final chunk with token counts. Not every server honours it: some always send usage, some ignore the option, and the chunk carrying usage has an empty choices list, which is why the loop checks chunk.choices first. The streaming guide explains the event format and how to cancel a stream.

Errors you will meet

Some providers return their own error types inside the JSON body. The SDK’s APIStatusError exposes status_code and message, which is enough to log and decide.

Switching providers

Only the environment changes. These are documented base URLs; the model ids are examples from our data:

The parameters each provider supports differ even when the endpoint is the same. Before relying on response_format, seed or stop, read the OpenAI compatibility guide, which lists what each provider documents as supported or ignored.

Troubleshooting

• The reply is empty and finish_reason is length. A thinking model used the whole max_tokens budget on reasoning. Raise max_tokens, or lower the reasoning effort if the model supports it.
• The answer contains <think> text. Some servers return reasoning inside the content, others in a separate field. Check the model card and the provider’s docs for how reasoning is returned.
• Requests hang. Set timeout on the client, as in the first example, and stream long replies.