[{"data":1,"prerenderedAt":114},["ShallowReactive",2],{"learn-local-vs-cloud-llm":3},{"collection":4,"slug":5,"path":6,"title":7,"seoTitle":8,"description":9,"summary":10,"primaryKeyword":11,"cluster":12,"published":13,"updated":13,"reviewed":13,"readingMinutes":14,"html":15,"toc":16,"wordCount":52,"sources":53,"faq":78,"related":91,"tested":108,"internalLinks":109},"learn","local-vs-cloud-llm","\u002Flearn\u002Flocal-vs-cloud-llm\u002F","Local vs hosted uncensored LLMs","Local vs Hosted Uncensored LLM: Privacy, Cost, Upkeep","Run a refusal-reduced model yourself or call a hosted API? Privacy, latency, upkeep and a worked cost comparison in which every input is a stated assumption.","Run the model locally when prompts must stay on hardware you control, your volume is high and steady, or you need one exact variant file, and you can absorb the upkeep. Use a hosted API when volume is low or irregular, the model does not fit your hardware, or your time costs more than the tokens. A hosted API is only as private as its operator's policy, so read it.","local vs hosted uncensored llm","deployment","2026-11-14",7,"\u003Ch2 id=\"the-short-answer\">The short answer\u003C\u002Fh2>\n\u003Cp>Run a refusal-reduced model on your own hardware when prompts must stay on machines you control, when you need one exact file, or when your volume is high and steady. Use a hosted pay-per-token API when volume is low or irregular, when the model will not fit your hardware, or when operations time is the scarce resource. A rented GPU sits between the two.\u003C\u002Fp>\n\u003Cp>The rest of this page explains the tradeoffs, then works through a cost comparison in which every input is an assumption you should replace.\u003C\u002Fp>\n\u003Cdiv class=\"table-wrap\" role=\"region\" aria-label=\"Scrollable table\" tabindex=\"0\">\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Factor\u003C\u002Fth>\n\u003Cth>Your own hardware\u003C\u002Fth>\n\u003Cth>Rented GPU, your software\u003C\u002Fth>\n\u003Cth>Hosted API, pay per token\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>Where prompts go\u003C\u002Ftd>\n\u003Ctd>Stay on the machine unless you expose the server\u003C\u002Ftd>\n\u003Ctd>To a machine you rent; the host runs the hardware\u003C\u002Ftd>\n\u003Ctd>To the provider; its policy decides retention and access\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Model choice\u003C\u002Ftd>\n\u003Ctd>Any file you can download and fit in memory\u003C\u002Ftd>\n\u003Ctd>The same, with more memory available\u003C\u002Ftd>\n\u003Ctd>What the provider serves, which may or may not be a named public repo\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Latency\u003C\u002Ftd>\n\u003Ctd>No network hop; speed set by your hardware\u003C\u002Ftd>\n\u003Ctd>A network hop to dedicated hardware\u003C\u002Ftd>\n\u003Ctd>A network hop, plus possible queueing and rate limits\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Upkeep\u003C\u002Ftd>\n\u003Ctd>Yours: drivers, runtime, model files, security\u003C\u002Ftd>\n\u003Ctd>Yours, plus starting, stopping and storage\u003C\u002Ftd>\n\u003Ctd>Mostly the provider’s; you track model and API changes\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Cost shape\u003C\u002Ftd>\n\u003Ctd>Up front, then power and time\u003C\u002Ftd>\n\u003Ctd>Per hour while running\u003C\u002Ftd>\n\u003Ctd>Per token\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\u003Ch2 id=\"privacy-what-local-gives-you-and-what-it-does-not\">Privacy: what local gives you, and what it does not\u003C\u002Fh2>\n\u003Cp>Running the weights yourself means inference happens on hardware you control. llama.cpp’s server, for example, listens on \u003Ccode>127.0.0.1\u003C\u002Fcode> by default, so nothing outside the machine can reach it until you change \u003Ccode>--host\u003C\u002Fcode>. It also starts with no API key (\u003Ccode>--api-key\u003C\u002Fcode> is “default: none”), so set one before you open it to a network (\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fblob\u002Fmaster\u002Ftools\u002Fserver\u002FREADME.md\" rel=\"noopener\" class=\"ext\">server README\u003C\u002Fa>).\u003C\u002Fp>\n\u003Cp>Local is not private by itself:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Logs, chat histories and caches that your tools write to disk are yours to protect and delete.\u003C\u002Fli>\n\u003Cli>Binding the server to all interfaces, or putting it behind a tunnel, lets anyone who can reach the port use the model.\u003C\u002Fli>\n\u003Cli>The model file is a third-party artifact. Know who uploaded it and which base model it names.\u003C\u002Fli>\n\u003Cli>The license travels with the weights. The \u003Ca href=\"https:\u002F\u002Fwww.llama.com\u002Fllama3_3\u002Fuse-policy\u002F\" rel=\"noopener\" class=\"ext\">Llama 3.3 Acceptable Use Policy\u003C\u002Fa> begins “You agree you will not use, or allow others to use, Llama 3.3 to:”, and that applies on your desk as much as in a data center. \u003Ca href=\"\u002Flearn\u002Fllm-model-licenses\u002F\">Our license comparison\u003C\u002Fa> covers the other base models.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2 id=\"what-private-means-for-a-hosted-api\">What “private” means for a hosted API\u003C\u002Fh2>\n\u003Cp>A hosted API sends your prompts over the network to someone else’s machines, where they are processed and may be logged. Logging policies vary between providers, and sometimes between plans of the same provider. “Private”, “no-log” or “zero retention” on a landing page is a summary. The privacy policy, the terms of service and any data processing agreement are what you can hold the provider to.\u003C\u002Fp>\n\u003Cp>Before you send sensitive data, answer these questions from the provider’s own documents:\u003C\u002Fp>\n\u003Col>\n\u003Cli>Are prompts and outputs stored? For how long, and in which country?\u003C\u002Fli>\n\u003Cli>Who can read them: staff reviewing abuse, support, subprocessors?\u003C\u002Fli>\n\u003Cli>Are they used to train or evaluate models, and is that opt-in, opt-out or excluded by contract?\u003C\u002Fli>\n\u003Cli>Does the provider run the model itself, or forward requests to other companies?\u003C\u002Fli>\n\u003Cli>Can you delete your data, and does the provider confirm deletion?\u003C\u002Fli>\n\u003Cli>Which model serves the request? Some hosted providers list specific abliterated Hugging Face repositories by name. Others serve their own builds without saying how they were made.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>Keep a dated copy of the policy you relied on, because policies change. Our \u003Ca href=\"\u002Fapi\u002Fproviders\u002F\">provider directory\u003C\u002Fa> records what each provider states about its models and data handling.\u003C\u002Fp>\n\u003Ch2 id=\"latency-and-throughput\">Latency and throughput\u003C\u002Fh2>\n\u003Cp>Two numbers describe how a model feels in use: time to first token, which grows with prompt length, and generation speed in tokens per second.\u003C\u002Fp>\n\u003Cp>Locally there is no network round trip, but both numbers depend on your hardware and on how much of the model fits in GPU memory. llama.cpp’s server lets you choose how many layers to store in VRAM (\u003Ccode>-ngl\u003C\u002Fcode>), so a model larger than your GPU can still run, more slowly, with the rest in system memory.\u003C\u002Fp>\n\u003Cp>A hosted API adds a network round trip and, on shared capacity, possible queueing. Providers may also cap requests per minute. Neither side wins in general, so measure both with your own prompts and context lengths.\u003C\u002Fp>\n\u003Ch2 id=\"will-the-model-fit-memory-comes-first\">Will the model fit? Memory comes first\u003C\u002Fh2>\n\u003Cp>On local hardware, memory is the first constraint. Two Q4_K_M file sizes from Hugging Face, checked 14 November 2026:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fbartowski\u002FLlama-3.3-70B-Instruct-abliterated-GGUF\" rel=\"noopener\" class=\"ext\">Llama 3.3 70B abliterated\u003C\u002Fa>: 42.52 GB.\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fmradermacher\u002FHuihui-GLM-4.7-Flash-abliterated-GGUF\" rel=\"noopener\" class=\"ext\">GLM-4.7-Flash abliterated\u003C\u002Fa>: 18.13 GB.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>The file is not the whole budget. The KV cache grows with context length. The \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FLlama-3.3-70B-Instruct\" rel=\"noopener\" class=\"ext\">Llama 3.3 70B Instruct\u003C\u002Fa> config lists 80 layers, 8 key-value heads and a head dimension of 128, so a 16-bit cache needs:\u003C\u002Fp>\n\u003Cdiv class=\"code-block\" data-lang=\"text\">\u003Cspan class=\"sr-only\">text code\u003C\u002Fspan>\u003Cpre tabindex=\"0\">\u003Ccode class=\"language-text\">2 (keys and values) × 80 layers × 8 heads × 128 dims × 2 bytes = 327,680 bytes per token\n327,680 × 32,768 tokens  = 10 GiB\n327,680 × 131,072 tokens = 40 GiB (the configured maximum context)\n\u003C\u002Fcode>\u003C\u002Fpre>\u003C\u002Fdiv>\n\u003Cp>Compute buffers come on top. The \u003Ca href=\"\u002Ftools\u002Fvram-estimator\u002F\">VRAM estimator\u003C\u002Fa> runs this arithmetic for other models and settings, and \u003Ca href=\"\u002Flearn\u002Fgguf-quantization\u002F\">GGUF quantization explained\u003C\u002Fa> shows how the quant type changes file size.\u003C\u002Fp>\n\u003Ch2 id=\"upkeep-is-part-of-the-cost\">Upkeep is part of the cost\u003C\u002Fh2>\n\u003Cp>With your own hardware or a rented GPU, you are the operator. The recurring tasks:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>updating the inference runtime and GPU drivers, then retesting;\u003C\u002Fli>\n\u003Cli>downloading new files when a variant is revised, and keeping track of which file you tested;\u003C\u002Fli>\n\u003Cli>fixing chat templates; llama.cpp takes the template from the model’s metadata by default and offers \u003Ccode>--chat-template-file\u003C\u002Fcode> to override it;\u003C\u002Fli>\n\u003Cli>securing the endpoint: bind address, API keys, TLS and firewall rules;\u003C\u002Fli>\n\u003Cli>watching memory, disk space and crashes.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>A hosted API moves most of this to the provider and adds a different kind of upkeep: models get retired, APIs change and terms are revised.\u003C\u002Fp>\n\u003Ch2 id=\"a-worked-cost-comparison\">A worked cost comparison\u003C\u002Fh2>\n\u003Cp>Every number in this section is an assumption chosen to keep the arithmetic easy to follow. None of them is a market price, a quote or a measurement. Replace each one with your own figures. The \u003Ca href=\"\u002Ftools\u002Ftoken-cost-calculator\u002F\">token cost calculator\u003C\u002Fa> handles the hosted side, and the \u003Ca href=\"\u002Fapi\u002Fpricing\u002F\">API pricing table\u003C\u002Fa> lists current per-token prices.\u003C\u002Fp>\n\u003Ch3 id=\"inputs\">Inputs\u003C\u002Fh3>\n\u003Cdiv class=\"table-wrap\" role=\"region\" aria-label=\"Scrollable table\" tabindex=\"0\">\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>#\u003C\u002Fth>\n\u003Cth>Input\u003C\u002Fth>\n\u003Cth>Assumed value\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>A1\u003C\u002Ftd>\n\u003Ctd>Monthly workload\u003C\u002Ftd>\n\u003Ctd>30 million input tokens, 6 million output tokens\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A2\u003C\u002Ftd>\n\u003Ctd>Hosted API price (placeholder)\u003C\u002Ftd>\n\u003Ctd>$0.50 per million input tokens, $1.50 per million output tokens\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A3\u003C\u002Ftd>\n\u003Ctd>Rented GPU price (placeholder)\u003C\u002Ftd>\n\u003Ctd>$2.00 per hour\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A4\u003C\u002Ftd>\n\u003Ctd>Your own hardware\u003C\u002Ftd>\n\u003Ctd>$3,000, written off over 36 months\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A5\u003C\u002Ftd>\n\u003Ctd>Speed of one request stream, own or rented\u003C\u002Ftd>\n\u003Ctd>1,000 tokens\u002Fs prompt processing, 40 tokens\u002Fs generation\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A6\u003C\u002Ftd>\n\u003Ctd>Power draw of your own hardware\u003C\u002Ftd>\n\u003Ctd>0.40 kW under load, 0.08 kW idle, switched on all month\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A7\u003C\u002Ftd>\n\u003Ctd>Electricity\u003C\u002Ftd>\n\u003Ctd>$0.25 per kWh\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A8\u003C\u002Ftd>\n\u003Ctd>Value of your time\u003C\u002Ftd>\n\u003Ctd>$60 per hour\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A9\u003C\u002Ftd>\n\u003Ctd>Upkeep per month\u003C\u002Ftd>\n\u003Ctd>3 hours for own or rented, 0.5 hours for hosted\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A10\u003C\u002Ftd>\n\u003Ctd>Extra rented hours for startup, loading and idle\u003C\u002Ftd>\n\u003Ctd>20 hours per month\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>A11\u003C\u002Ftd>\n\u003Ctd>Hours in a month\u003C\u002Ftd>\n\u003Ctd>730\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\u003Ch3 id=\"arithmetic\">Arithmetic\u003C\u002Fh3>\n\u003Cp>Busy time, from A1 and A5:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Prompt processing: 30,000,000 ÷ 1,000 = 30,000 s, about 8.3 h.\u003C\u002Fli>\n\u003Cli>Generation: 6,000,000 ÷ 40 = 150,000 s, about 41.7 h.\u003C\u002Fli>\n\u003Cli>Total: about 50 busy hours per month.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cdiv class=\"table-wrap\" role=\"region\" aria-label=\"Scrollable table\" tabindex=\"0\">\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Option\u003C\u002Fth>\n\u003Cth>Calculation\u003C\u002Fth>\n\u003Cth>Monthly total\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>Hosted API\u003C\u002Ftd>\n\u003Ctd>(30 × $0.50) + (6 × $1.50) = $24, plus 0.5 h × $60 = $30\u003C\u002Ftd>\n\u003Ctd>$54\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Rented GPU, started per job\u003C\u002Ftd>\n\u003Ctd>(50 + 20) h × $2.00 = $140, plus 3 h × $60 = $180\u003C\u002Ftd>\n\u003Ctd>$320\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Rented GPU, always on\u003C\u002Ftd>\n\u003Ctd>730 h × $2.00 = $1,460, plus $180\u003C\u002Ftd>\n\u003Ctd>$1,640\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Own hardware\u003C\u002Ftd>\n\u003Ctd>$3,000 ÷ 36 = $83.33; power (50 h × 0.40 kW + 680 h × 0.08 kW) = 74.4 kWh × $0.25 = $18.60; plus $180\u003C\u002Ftd>\n\u003Ctd>$281.93\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\u003Ch3 id=\"what-changes-the-answer\">What changes the answer\u003C\u002Fh3>\n\u003Cp>With these inputs, the hosted API is cheapest by a wide margin, and the biggest local cost is neither hardware nor power but upkeep time. Change the inputs and the order changes.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>Volume.\u003C\u002Fstrong> Hosted cost grows with tokens; local cost is mostly fixed. Multiply the workload by a factor k: hosted costs 24k + 30 dollars, own hardware 277.93 + 4k (power rises by $4 per extra 50 busy hours). They meet at k ≈ 12.4, about 370 million input and 74 million output tokens a month. That needs about 620 busy hours, close to the 730 hours a single stream has.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Hardware you already own.\u003C\u002Fstrong> Drop A4 and own hardware costs 194.60 + 4k, which meets the hosted line at k ≈ 8.2.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Concurrency.\u003C\u002Fstrong> llama.cpp’s server supports parallel decoding and continuous batching. Serving several requests at once can raise total throughput above the single-stream speed in A5, which lowers the cost per token of owned or rented hardware at high volume.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Your time.\u003C\u002Fstrong> If upkeep is already part of your job, A8 matters less. In this example it is the largest local cost.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Requirements that are not costs.\u003C\u002Fstrong> If a policy says data must not leave your network, the hosted row is not an option at any price.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2 id=\"when-each-option-fits\">When each option fits\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>Your own hardware\u003C\u002Fstrong> fits when prompts must stay on machines you control, when you need one exact file such as a specific variant and quant, when you work offline, or when volume is high and steady enough to keep the hardware busy. Our list of \u003Ca href=\"\u002Fmodels\u002Flocal\u002F\">models that run locally\u003C\u002Fa> is a starting point.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>A rented GPU\u003C\u002Fstrong> fits batch jobs and models that do not fit your own hardware, when you still want to pick the exact weights. Start it for the job and stop it afterwards, because idle hours are where the cost goes.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>A hosted API\u003C\u002Fstrong> fits prototypes, low or irregular volume, and teams without time for operations, provided the provider’s data policy is acceptable for what you send.\u003C\u002Fp>\n\u003Cp>You do not have to choose once. llama.cpp’s server offers OpenAI-compatible chat completion routes, and \u003Ca href=\"https:\u002F\u002Fdocs.vllm.ai\u002Fen\u002Flatest\u002Fserving\u002Fonline_serving\u002F\" rel=\"noopener\" class=\"ext\">vLLM\u003C\u002Fa> provides “an HTTP server that is compatible with many interfaces”, including OpenAI’s. Code written against that interface can point at a local server or a hosted provider by changing the base URL, key and model name; the \u003Ca href=\"\u002Fguides\u002Fopenai-compatible-api\u002F\">OpenAI-compatible API guide\u003C\u002Fa> shows how.\u003C\u002Fp>\n",[17,21,24,27,30,33,36,39,43,46,49],{"id":18,"text":19,"level":20},"the-short-answer","The short answer",2,{"id":22,"text":23,"level":20},"privacy-what-local-gives-you-and-what-it-does-not","Privacy: what local gives you, and what it does not",{"id":25,"text":26,"level":20},"what-private-means-for-a-hosted-api","What “private” means for a hosted API",{"id":28,"text":29,"level":20},"latency-and-throughput","Latency and throughput",{"id":31,"text":32,"level":20},"will-the-model-fit-memory-comes-first","Will the model fit? Memory comes first",{"id":34,"text":35,"level":20},"upkeep-is-part-of-the-cost","Upkeep is part of the cost",{"id":37,"text":38,"level":20},"a-worked-cost-comparison","A worked cost comparison",{"id":40,"text":41,"level":42},"inputs","Inputs",3,{"id":44,"text":45,"level":42},"arithmetic","Arithmetic",{"id":47,"text":48,"level":42},"what-changes-the-answer","What changes the answer",{"id":50,"text":51,"level":20},"when-each-option-fits","When each option fits",1628,[54,58,62,66,70,74],{"title":55,"publisher":56,"url":57},"LLaMA.cpp HTTP Server README","GitHub (ggml-org\u002Fllama.cpp)","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fblob\u002Fmaster\u002Ftools\u002Fserver\u002FREADME.md",{"title":59,"publisher":60,"url":61},"Online serving: OpenAI-Compatible Server","vLLM documentation","https:\u002F\u002Fdocs.vllm.ai\u002Fen\u002Flatest\u002Fserving\u002Fonline_serving\u002F",{"title":63,"publisher":64,"url":65},"Llama-3.3-70B-Instruct model card","Hugging Face (meta-llama)","https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FLlama-3.3-70B-Instruct",{"title":67,"publisher":68,"url":69},"Llama 3.3 Acceptable Use Policy","Meta","https:\u002F\u002Fwww.llama.com\u002Fllama3_3\u002Fuse-policy\u002F",{"title":71,"publisher":72,"url":73},"Llama-3.3-70B-Instruct-abliterated-GGUF","Hugging Face (bartowski)","https:\u002F\u002Fhuggingface.co\u002Fbartowski\u002FLlama-3.3-70B-Instruct-abliterated-GGUF",{"title":75,"publisher":76,"url":77},"Huihui-GLM-4.7-Flash-abliterated-GGUF","Hugging Face (mradermacher)","https:\u002F\u002Fhuggingface.co\u002Fmradermacher\u002FHuihui-GLM-4.7-Flash-abliterated-GGUF",[79,82,85,88],{"q":80,"a":81},"Is a hosted uncensored LLM API private?","Only as far as its operator's policy and contract say. Check what it stores, for how long, who can read it and whether requests are passed to other companies. A 'no-log' label on a landing page is not a guarantee.",{"q":83,"a":84},"Is running a model locally always cheaper?","No. In the worked example on this page, upkeep time costs more than hardware and power together, and local only breaks even at about 12 times the example workload. Run the comparison with your own numbers.",{"q":86,"a":87},"How much memory does Llama 3.3 70B need locally?","The Q4_K_M GGUF file alone is 42.52 GB, and a 16-bit KV cache for 32,768 tokens of context adds 10 GiB. Compute buffers come on top of both.",{"q":89,"a":90},"Can I switch between local and hosted later?","Yes, if your code talks to an OpenAI-compatible API. llama.cpp's server and vLLM both provide one, so moving to or from a hosted provider with the same interface is mostly a change of base URL, key and model name.",[92,96,100,104],{"path":93,"title":94,"description":95},"\u002Flearn\u002Fgguf-quantization\u002F","GGUF quantization explained","What GGUF is, what Q8_0, Q6_K, Q5_K_M, Q4_K_M and IQ4_XS mean, and how to estimate file size from bits per weight, checked against real Hugging Face files.",{"path":97,"title":98,"description":99},"\u002Fapi\u002Fproviders\u002F","API providers","Provider profiles with published features and limits.",{"path":101,"title":102,"description":103},"\u002Ftools\u002Fvram-estimator\u002F","VRAM estimator","Weights, KV cache and overhead for local inference.",{"path":105,"title":106,"description":107},"\u002Fguides\u002Fopenai-compatible-api\u002F","OpenAI-compatible LLM APIs, and where they differ","What OpenAI-compatible means for uncensored LLM APIs: base URLs, ignored parameters, reasoning fields, usage in streams, tool calls, plus a script to probe any endpoint.",null,[110,97,101,93,111,112,113,105],"\u002Flearn\u002Fllm-model-licenses\u002F","\u002Ftools\u002Ftoken-cost-calculator\u002F","\u002Fapi\u002Fpricing\u002F","\u002Fmodels\u002Flocal\u002F",1794622336927]