How to Host a Chatbot on Your Personal Computer

It’s truly remarkable how large language models powering AI assistants like ChatGPT and Gemini are transforming our lives. Many people now depend on them for tasks like coding, writing, summarizing, and internet searches. Regardless of where AI heads in the coming years, LLMs will undeniably be part of that evolution.

If you’re using ChatGPT, Claude, Perplexity, or any similar AI service, you are indeed utilizing an LLM. What may not be widely understood is that these LLMs can also run locally on your personal computer. This offers significant advantages, including offline access and enhanced privacy, as your data isn’t sent to the cloud for external analysis.

Additionally, you won’t have to pay any monthly fees to AI companies or deal with usage limits. Numerous LLMs are available for free download, including from renowned brands like Meta and Google. While they might not match the sophistication or speed of the subscription-based LLMs, they are capable enough for regular use, allowing you to choose as needed.

Running local models does involve more maintenance, and you sacrifice some of the convenience that comes with simply launching an app like ChatGPT. For instance, you’ll need to manage updates on your own. However, this setup provides a more tailored and private AI experience, and getting started isn’t overly complicated.

Requirements to Begin

You can operate local LLMs on Windows, macOS, and Linux, although macOS is often favored by AI enthusiasts due to its more uniform and consistent ecosystem. Since Macs are produced by a single company, ongoing compatibility issues are less of a concern. Furthermore, Apple Silicon chips integrate the CPU, GPU, and RAM, which is beneficial for AI models.

Regardless of your chosen platform, having ample RAM is essential. The minimum requirement is 8 GB; however, if that’s your only option, you may experience limitations in the performance and size of the LLMs you can run. A recommended 16 GB is better, and 32 GB or more will be necessary to utilize the largest and fastest models effectively. For optimal results, a dedicated GPU with substantial VRAM (anything over 8 GB significantly enhances performance) is necessary, as this memory type is optimized for the functions AI models perform.

An AI chatbot like ChatGPT relies on the cloud.Photograph: David Nield