Getting Started with Ollama

Get an LLM up and running on your local machine in 10 minutes.

Table of Contents

Quickstart

To get started, Download Ollama and run Llama3:

ollama run llama3

This will download the Llama3 model and run a new chat instance. If the model is already downloded, the terminal should immediately load a new chat.

Model Library

Ollama supports a variety of popular LLM models, the full list can be found here. Here is an example of models that can be downloaded:

Advanced

Import from GGUF

Ollama supports importing GGUF models in the Modelfile:

1. Create a file named Modelfile, with a FROM instruction with the local filepath to the model you want to import.

FROM ./vicuna-33b.Q4_0.gguf

2. Create the model in Ollama

ollama create example -f Modelfile

3. Run the model

ollama run example

CLI Reference

• Create a model: ollama create mymodel -f ./Modelfile
• Pull a model: ollama pull llama3
• Remove a model: ollama rm llama3
• Copy a model: ollama cp llama3 my-model
• List models on the computer: ollama list

Building Ollama

Check out the developer guide

1. Start the server: ./ollama serve
2. In a separate shell, run a model: ./ollama run llama3

REST API

Ollama has a REST API for running and managing models.

Generate a response

curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt":"Why is the sky blue?"
}'

Chat with an LLM

curl http://localhost:11434/api/chat -d '{
  "model": "llama3",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

Want more endpoints? Check out the API documentation.