Local vs hosted uncensored LLMs

The short answer

Run a refusal-reduced model on your own hardware when prompts must stay on machines you control, when you need one exact file, or when your volume is high and steady. Use a hosted pay-per-token API when volume is low or irregular, when the model will not fit your hardware, or when operations time is the scarce resource. A rented GPU sits between the two.

The rest of this page explains the tradeoffs, then works through a cost comparison in which every input is an assumption you should replace.

Privacy: what local gives you, and what it does not

Running the weights yourself means inference happens on hardware you control. llama.cpp’s server, for example, listens on 127.0.0.1 by default, so nothing outside the machine can reach it until you change --host. It also starts with no API key (--api-key is “default: none”), so set one before you open it to a network (server README).

Local is not private by itself:

• Logs, chat histories and caches that your tools write to disk are yours to protect and delete.
• Binding the server to all interfaces, or putting it behind a tunnel, lets anyone who can reach the port use the model.
• The model file is a third-party artifact. Know who uploaded it and which base model it names.
• The license travels with the weights. The Llama 3.3 Acceptable Use Policy begins “You agree you will not use, or allow others to use, Llama 3.3 to:”, and that applies on your desk as much as in a data center. Our license comparison covers the other base models.

What “private” means for a hosted API

A hosted API sends your prompts over the network to someone else’s machines, where they are processed and may be logged. Logging policies vary between providers, and sometimes between plans of the same provider. “Private”, “no-log” or “zero retention” on a landing page is a summary. The privacy policy, the terms of service and any data processing agreement are what you can hold the provider to.

Before you send sensitive data, answer these questions from the provider’s own documents:

1. Are prompts and outputs stored? For how long, and in which country?
2. Who can read them: staff reviewing abuse, support, subprocessors?
3. Are they used to train or evaluate models, and is that opt-in, opt-out or excluded by contract?
4. Does the provider run the model itself, or forward requests to other companies?
5. Can you delete your data, and does the provider confirm deletion?
6. Which model serves the request? Some hosted providers list specific abliterated Hugging Face repositories by name. Others serve their own builds without saying how they were made.

Keep a dated copy of the policy you relied on, because policies change. Our provider directory records what each provider states about its models and data handling.

Latency and throughput

Two numbers describe how a model feels in use: time to first token, which grows with prompt length, and generation speed in tokens per second.

Locally there is no network round trip, but both numbers depend on your hardware and on how much of the model fits in GPU memory. llama.cpp’s server lets you choose how many layers to store in VRAM (-ngl), so a model larger than your GPU can still run, more slowly, with the rest in system memory.

A hosted API adds a network round trip and, on shared capacity, possible queueing. Providers may also cap requests per minute. Neither side wins in general, so measure both with your own prompts and context lengths.

Will the model fit? Memory comes first

On local hardware, memory is the first constraint. Two Q4_K_M file sizes from Hugging Face, checked 14 November 2026:

• Llama 3.3 70B abliterated: 42.52 GB.
• GLM-4.7-Flash abliterated: 18.13 GB.

The file is not the whole budget. The KV cache grows with context length. The Llama 3.3 70B Instruct config lists 80 layers, 8 key-value heads and a head dimension of 128, so a 16-bit cache needs:

2 (keys and values) × 80 layers × 8 heads × 128 dims × 2 bytes = 327,680 bytes per token
327,680 × 32,768 tokens  = 10 GiB
327,680 × 131,072 tokens = 40 GiB (the configured maximum context)

Compute buffers come on top. The VRAM estimator runs this arithmetic for other models and settings, and GGUF quantization explained shows how the quant type changes file size.

Upkeep is part of the cost

With your own hardware or a rented GPU, you are the operator. The recurring tasks:

• updating the inference runtime and GPU drivers, then retesting;
• downloading new files when a variant is revised, and keeping track of which file you tested;
• fixing chat templates; llama.cpp takes the template from the model’s metadata by default and offers --chat-template-file to override it;
• securing the endpoint: bind address, API keys, TLS and firewall rules;
• watching memory, disk space and crashes.

A hosted API moves most of this to the provider and adds a different kind of upkeep: models get retired, APIs change and terms are revised.

A worked cost comparison

Every number in this section is an assumption chosen to keep the arithmetic easy to follow. None of them is a market price, a quote or a measurement. Replace each one with your own figures. The token cost calculator handles the hosted side, and the API pricing table lists current per-token prices.

Inputs

Arithmetic

Busy time, from A1 and A5:

• Prompt processing: 30,000,000 ÷ 1,000 = 30,000 s, about 8.3 h.
• Generation: 6,000,000 ÷ 40 = 150,000 s, about 41.7 h.
• Total: about 50 busy hours per month.

What changes the answer

With these inputs, the hosted API is cheapest by a wide margin, and the biggest local cost is neither hardware nor power but upkeep time. Change the inputs and the order changes.

• Volume. Hosted cost grows with tokens; local cost is mostly fixed. Multiply the workload by a factor k: hosted costs 24k + 30 dollars, own hardware 277.93 + 4k (power rises by $4 per extra 50 busy hours). They meet at k ≈ 12.4, about 370 million input and 74 million output tokens a month. That needs about 620 busy hours, close to the 730 hours a single stream has.
• Hardware you already own. Drop A4 and own hardware costs 194.60 + 4k, which meets the hosted line at k ≈ 8.2.
• Concurrency. llama.cpp’s server supports parallel decoding and continuous batching. Serving several requests at once can raise total throughput above the single-stream speed in A5, which lowers the cost per token of owned or rented hardware at high volume.
• Your time. If upkeep is already part of your job, A8 matters less. In this example it is the largest local cost.
• Requirements that are not costs. If a policy says data must not leave your network, the hosted row is not an option at any price.

When each option fits

Your own hardware fits when prompts must stay on machines you control, when you need one exact file such as a specific variant and quant, when you work offline, or when volume is high and steady enough to keep the hardware busy. Our list of models that run locally is a starting point.

A rented GPU fits batch jobs and models that do not fit your own hardware, when you still want to pick the exact weights. Start it for the job and stop it afterwards, because idle hours are where the cost goes.

A hosted API fits prototypes, low or irregular volume, and teams without time for operations, provided the provider’s data policy is acceptable for what you send.

You do not have to choose once. llama.cpp’s server offers OpenAI-compatible chat completion routes, and vLLM provides “an HTTP server that is compatible with many interfaces”, including OpenAI’s. Code written against that interface can point at a local server or a hosted provider by changing the base URL, key and model name; the OpenAI-compatible API guide shows how.