Skip to content

Usage Guide

Complete guide to managing your Ollama environment and working with models.

Managing the Environment

Starting and Stopping Services

Start all services (Ollama + Chat UI): 

make up

Start Ollama only (no chat web UI): 

make up-core

Start with NVIDIA GPU acceleration (uses the docker-compose.gpu.yml override): 

make up-gpu

Stop all services: 

make down

Restart services: 

make restart

Note: After changing ports or other values in .env, use make down && make up (a plain restart does not re-read port mappings).

Check service status: 

docker compose ps

Monitoring and Logs

View real-time logs: 

make logs

# Or for specific service
docker compose logs -f ollama
docker compose logs -f chat

Access container shell: 

make shell

# Or directly
docker compose exec ollama bash

Cleanup

Remove all containers and volumes (deletes all models): 

make clean

Partial cleanup (keep volumes): 

make down

Working with Models

Listing Models

List all installed models: 

make list-models

# Or directly
docker compose exec ollama ollama list

This shows: - Model names - Size on disk - Model ID - Last modified date

Pulling Models

Via Web UI (Recommended): 1. Open http://localhost:8080 2. Click "Manage Models" 3. Enter model name (e.g., llama3.2, mistral:7b, phi3:mini) 4. Click "Pull Model" 5. Watch real-time progress with download speed and ETA

Via CLI: 

# Pull common base models
make pull-base

# Pull specific model
docker compose exec ollama ollama pull <model-name>

# Examples:
docker compose exec ollama ollama pull llama3.2:1b
docker compose exec ollama ollama pull mistral:7b
docker compose exec ollama ollama pull codellama:13b

Available models: See Ollama Library for all models.

Creating Custom Models

Custom models are defined using Modelfiles (similar to Dockerfiles).

Interactive creation (with menu selection): 

make create-model

This will: 1. Show a numbered list of available Modelfiles 2. Let you select one 3. Ask for a model name 4. Create the model

Direct creation: 

bash scripts/create-custom-model.sh <model-name> <modelfile-path>

# Examples:
bash scripts/create-custom-model.sh my-chatbot ./models/examples/chatbot/Modelfile
bash scripts/create-custom-model.sh code-helper ./models/custom/code-assistant/Modelfile

Creating your own Modelfile:

Create a file at ./models/custom/my-model/Modelfile:

# Base model (must be pulled first)
FROM llama3.2:1b

# Model parameters
PARAMETER temperature 0.7
PARAMETER num_ctx 4096
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1

# System prompt (defines behavior)
SYSTEM """
You are a helpful assistant specialized in [your use case].
[Define specific behavior, constraints, and personality here]
"""

# Optional: Few-shot examples
MESSAGE user "Example question?"
MESSAGE assistant "Example answer."

Then create the model: 

bash scripts/create-custom-model.sh my-model ./models/custom/my-model/Modelfile

See Modelfile Reference for complete syntax.

Chatting with Models

Interactive chat (via CLI): 

make chat
# Select from a numbered list of models

Or directly: 

docker compose exec ollama ollama run <model-name>

# Examples:
docker compose exec ollama ollama run llama3.2:1b
docker compose exec ollama ollama run my-chatbot

Single prompt (non-interactive): 

docker compose exec ollama ollama run <model-name> "Your prompt here"

# Example:
docker compose exec ollama ollama run llama3.2:1b "Write a haiku about Docker"

Via Web UI: 1. Open http://localhost:8080 2. Select model from dropdown 3. Start chatting!

See Chat UI Guide for web interface features.

Using the API

Generate completion: 

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:1b",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

Chat completion (with conversation history): 

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2:1b",
  "messages": [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi! How can I help?"},
    {"role": "user", "content": "What is Docker?"}
  ],
  "stream": false
}'

List models: 

curl http://localhost:11434/api/tags

Show model info: 

curl http://localhost:11434/api/show -d '{
  "name": "llama3.2:1b"
}'

See API Usage Guide for complete API documentation.

Saving and Deploying Models

Export Model Configuration

Save a model for deployment (interactive): 

make save-model

Or directly: 

bash scripts/save-model.sh <model-name> [output-directory]

# Examples:
bash scripts/save-model.sh my-chatbot
bash scripts/save-model.sh my-chatbot ./custom-output

This saves the Modelfile to ./models/saved/<model-name>.Modelfile

Export model's Modelfile: 

bash scripts/export-model.sh <model-name> <output-path>

# Example:
bash scripts/export-model.sh my-chatbot ./my-chatbot-v1.Modelfile

Deploy to Another Instance

Transfer Modelfile to target server: 

scp ./models/saved/my-chatbot.Modelfile user@server:/path/to/ollama-project/models/saved/

Deploy on target instance (interactive): 

make deploy-model

Or directly: 

bash scripts/deploy-model.sh <modelfile-path> [model-name]

# Example:
bash scripts/deploy-model.sh ./models/saved/my-chatbot.Modelfile my-chatbot

Backup All Models

Create timestamped backup: 

make backup-models

# Or directly:
bash scripts/backup-models.sh [output-directory]

This creates a backup in ./backups/models/YYYYMMDD_HHMMSS/ containing all your custom models' Modelfiles.

Importing External Models

Import GGUF Files

If you have a GGUF model file (from Hugging Face, fine-tuning, etc.):

  1. Place GGUF file in data directory: 

    cp /path/to/model.gguf ./data/gguf/

  2. Import the model: 

    bash scripts/import-model.sh <model-name> ./data/gguf/model.gguf

# Example:
bash scripts/import-model.sh my-custom-model ./data/gguf/my-model.gguf

  3. Use the model: 

    docker compose exec ollama ollama run my-custom-model

Using LoRA Adapters

If you have a LoRA adapter file:

  1. Place adapter in data directory: 

    cp /path/to/adapter.bin ./data/adapters/

  2. Create Modelfile at ./models/custom/adapted-model/Modelfile: 

    FROM llama3.2:1b
ADAPTER /data/adapters/my-adapter.bin

PARAMETER temperature 0.7

SYSTEM """
You are a specialized assistant.
"""

  3. Create the model: 

    bash scripts/create-custom-model.sh adapted-model ./models/custom/adapted-model/Modelfile

Model Parameters

Quick overview of the parameters you can adjust in Modelfiles:

Parameter Range Recommended Purpose
temperature 0.0-2.0 0.7 (0.3 factual, 1.2 creative) Randomness / creativity
num_ctx 512-32768 4096 Context window (tokens remembered)
top_p 0.0-1.0 0.9 Nucleus sampling / output diversity
top_k 1-100 40 Token selection pool size
repeat_penalty 0.0-2.0 1.1 Repetition control

The Parameter Guide is the canonical reference — it has copy-paste presets per use case (chatbot, code, support, creative, translator, data extraction) and troubleshooting advice. For the complete list of parameters and syntax, see the Modelfile Reference.

Advanced Operations

Testing Models

Quick end-to-end test: 

make quick-test

This automatically: 1. Creates a test model 2. Sends a test prompt 3. Displays the response 4. Deletes the test model

Validation tests: 

make test

Validates: - Docker Compose configuration - Service health - Directory structure - Example Modelfiles

Interactive Model Selection

The interactive numbered menus are provided directly by the scripts/interactive-*.sh scripts, which share helper functions from scripts/lib/common.sh:

  • scripts/interactive-create-model.sh: Select a Modelfile and create a model (make create-model)
  • scripts/interactive-chat.sh: Select an installed model and chat (make chat)
  • scripts/interactive-save-model.sh: Select a model to save (make save-model)
  • scripts/interactive-deploy-model.sh: Select a saved Modelfile to deploy (make deploy-model)
  • scripts/interactive-publish-model.sh: Select a model to publish to a registry (make publish-model)

All menus include option [0] to manually enter a custom path.

Delete Models

docker compose exec ollama ollama rm <model-name>

# Example:
docker compose exec ollama ollama rm old-model

Warning: This permanently deletes the model. Make sure to export/save it first if needed.

Performance Tips

Model Size vs Performance

  • 1B-3B models: Fast, low RAM, suitable for simple tasks
  • 7B-13B models: Balanced quality and speed
  • 30B+ models: High quality, slow, requires significant RAM/VRAM

GPU Acceleration

Enable GPU support for: - 5-10x faster inference - Ability to run larger models - Better concurrent user handling

See Installation Guide - GPU Support.

Disk Space Management

Models can be large. Check usage:

# Check Docker disk usage
docker system df

# Check volume size
docker volume inspect ollama_data

# List model sizes
docker compose exec ollama ollama list

Clean up unused models: 

docker compose exec ollama ollama rm <unused-model>

Next Steps