How to Run an LLM (AI Chatbot) Locally on Your Computer

Here’s a simple and quick tutorial to install and run an open-source LLM AI Chatbot on your computer’s hard drive.

Note: these instructions are for MacOS and Linux (no Windows option at the time of writing)

First-time instructions:

1. Go to https://ollama.com/
2. Click Download
3. Open the Ollama .zip file
4. Open the Ollama app and follow the instructions to setup
5. Copy the Terminal prompt at the end of the dialogue
6. Paste the prompt into your computer’s terminal
7. Allow it to install
8. Once done, you officially have an LLM on your computer’s hard drive and you can start typing anything you want into your terminal to communicate with the LLM

To install other LLMs after completing the above steps:

1. Go to https://ollama.com/library and pick your next LLM
2. Copy the prompt on the LLM page
3. Paste the prompt into Terminal to install

To run your LLM:

1. Open Terminal
2. Enter ollama run llama2
   1. Or, if running a model beside llama2, enter that name instead
3. Hit enter and start your chat conversation with the LLM

What are the benefits of doing this compared to using an LLM on my browser?

1. It doesn’t need an internet connection since it’s on your computer’s hard drive
2. Given point 1, it’s faster
3. Given point 1, it’s more private
4. Given point 1, you have more control over your data
5. No limits on number of prompts
6. More flexibility and more customizable

Here are all the open-source LLMs available to use with Ollama along with their terminal prompts:

Model

Parameters

Size

Download

Llama 2

7B

3.8GB

Mistral

7B

4.1GB

Dolphin Phi

2.7B

1.6GB

Phi-2

2.7B

1.7GB

Neural Chat

7B

4.1GB

Starling

7B

4.1GB

Code Llama

7B

3.8GB

Llama 2 Uncensored

7B

3.8GB

Llama 2 13B

13B

7.3GB

Llama 2 70B

70B

39GB

Orca Mini

3B

1.9GB

Vicuna

7B

3.8GB

LLaVA

7B

4.5GB

That’s it. It’s super easy and only takes few minutes to get up and running with a local LLM.

Cheers!