Ollama Python API: Build Local AI Apps in 30 Lines of Code

Ollama's OpenAI-compatible API makes it trivially easy to build local AI applications. If you have code that calls the OpenAI API, you can switch it to run fully locally with two lines changed. This guide shows every pattern you will need.

Installation

pip install ollama openai  # ollama client or openai-compatible

Basic Generation

import ollama

# Simple generation
response = ollama.generate(
    model='qwen3.6:14b',
    prompt='Explain GGUF quantization in plain English',
)
print(response['response'])

Chat Interface

import ollama

messages = [
    {'role': 'system', 'content': 'You are a local AI hardware expert.'},
    {'role': 'user', 'content': 'What GPU should I buy for running 32B models?'}
]

response = ollama.chat(model='qwen3.6:14b', messages=messages)
print(response['message']['content'])

Streaming Responses

import ollama

stream = ollama.chat(
    model='qwen3.6:14b',
    messages=[{'role': 'user', 'content': 'Write a Python function for token counting'}],
    stream=True
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

Using the OpenAI-Compatible API

from openai import OpenAI

# Drop-in replacement for OpenAI API
client = OpenAI(
    base_url='http://localhost:11434/v1',
    api_key='ollama'  # required but ignored
)

response = client.chat.completions.create(
    model='deepseek-r1:32b',
    messages=[{'role': 'user', 'content': 'Prove the Pythagorean theorem'}],
    temperature=0.6
)
print(response.choices[0].message.content)

Structured Output with JSON Schema

import ollama, json

response = ollama.chat(
    model='qwen3.6:14b',
    messages=[{
        'role': 'user',
        'content': 'List 3 GPUs for local LLM inference as JSON with name, vram, price fields'
    }],
    format='json'
)
data = json.loads(response['message']['content'])
print(data)

Embeddings for RAG

import ollama

embedding = ollama.embeddings(
    model='nomic-embed-text',
    prompt='RTX 4090 memory bandwidth for LLM inference'
)
vector = embedding['embedding']  # 768-dim float array
print(f"Embedding dims: {len(vector)}")

Model Management

import ollama

# List installed models
models = ollama.list()
for m in models['models']:
    size_gb = m['size'] / 1e9
    print(f"{m['name']}: {size_gb:.1f} GB")

# Pull a model programmatically with progress
for progress in ollama.pull('phi4', stream=True):
    print(progress.get('status', ''), end='
')

For hardware recommendations to run models accessed through the Ollama API, see LLMHardware.io's Ollama hardware guide.