Deploying DeepSeek-R1 & V3 Locally with Ollama & Open-WebUI: Complete Hardware Sizing & Setup Guide

1. Hardware Sizing Matrix: 1.5B to 671B MoE

Self-hosting DeepSeek-R1 offers complete data privacy, zero API rate limits, and uninterrupted offline inference. However, selecting the appropriate model tier is critical to avoid system thrashing.

Before downloading weights, map your hardware against the memory requirements below:

Need to calculate the exact VRAM footprint for custom context lengths and batch sizes? Use our interactive LLM GPU VRAM Calculator.

2. GGUF Quantization Deep Dive (Q4_K_M vs Q8 vs IQ2)

The GGUF container format developed by the llama.cpp community enables granular k-quant quantization:

• Q4_K_M (Recommended Standard): Employs 4-bit quantization for medium weight tensors while retaining 6-bit precision for attention key/value projections. Perplexity degradation is less than 0.15 compared to FP16, while memory footprint is reduced by 72%.
• Q8_0 (Near-Lossless): Preserves 8-bit precision across all tensors. Recommended if you are conducting rigorous mathematical proof validation and have surplus VRAM headroom.
• IQ2_XXS (Extreme MoE Compression): Utilizes importance matrix 2-bit quantization. Allows running the colossal 671B MoE model on a 192GB Mac Studio or dual 96GB workstation, enabling local testing of the full model.

3. Ollama Installation & Environment Configuration

Ollama serves as the underlying inference daemon, managing GPU offloading, CUDA runtime drivers, and memory-mapped model streaming.

Install Ollama on Linux, macOS, or Windows WSL2:

# Install Ollama CLI
curl -fsSL https://ollama.com/install.sh | sh

# Configure environment variables to maximize GPU concurrency
export OLLAMA_HOST="0.0.0.0:11434"
export OLLAMA_KEEP_ALIVE="24h"
export OLLAMA_NUM_PARALLEL=2
export CUDA_VISIBLE_DEVICES=0,1

# Launch Ollama service
ollama serve

4. Crafting the Production Modelfile (Parameters & KV Cache)

While default Ollama models are convenient, production deployments demand custom configuration to enforce appropriate context windows and thinking token behavior.

Create a file named Modelfile-R1-14B:

FROM deepseek-r1:14b

# Set active context window to 16k tokens
PARAMETER num_ctx 16384

# Control generation entropy (Reasoning models prefer 0.6)
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.05

# System prompt enforcing structured XML reasoning tags
SYSTEM '''You are DeepSeek-R1, an expert autonomous reasoning assistant.
Before delivering the final answer, conduct an exhaustive, rigorous exploration of the problem
inside <think> ... </think> tags. Challenge initial assumptions, verify intermediate calculations,
and format all final code blocks in clean, idiomatic syntax.'''

Build and register your customized model:

ollama create deepseek-r1-custom -f ./Modelfile-R1-14B

5. Deploying Open-WebUI with Docker Compose

To provide your engineering team with a top-tier user interface matching ChatGPT and Claude, deploy Open-WebUI via Docker Compose:

# docker-compose.yml
version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - WEBUI_AUTH=true
      - ENABLE_SIGNUP=false
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - open-webui-data:/app/backend/data

volumes:
  open-webui-data:

Launch the service with docker compose up -d and navigate to http://localhost:3000. Open-WebUI automatically renders reasoning blocks in collapsible accordions, supports direct document RAG uploads, and provides API key provisioning for team members.

6. Throughput Benchmarking & OOM Troubleshooting

If your generation speeds drop below 10 tokens/sec, check the following diagnostic indicators:

• CPU Layer Offloading: Check ollama ps. If the GPU offload percentage is less than 100% (e.g. 35/48 layers), parts of the model are spilling into system RAM across the PCIe bus, introducing severe latency bottlenecks. Reduce model size from 32B to 14B or lower your context window.
• CUDA Memory Fragmentation: On multi-GPU configurations, ensure you set export NCCL_P2P_DISABLE=0 to allow peer-to-peer memory access over PCIe/NVLink.

7. Frequently Asked Questions