Server Lifecycle

Overview

The portshim CLI wraps llama.cpp's server mode with engagement-aware lifecycle management. Instead of juggling terminal windows and config files, operators control the entire inference stack from a single command.

Server Commands

portshim server start          # Start llama-server with current model
portshim server stop           # Graceful shutdown
portshim server restart        # Stop then start (reloads config)
portshim server status         # Health check + metrics
portshim server switch MODEL   # Hot-swap to a different GGUF

Auto-Start Behaviour

When running in local mode, the server auto-starts at the beginning of each scan phase and shuts down after the gate review. This saves VRAM and power between active phases.

Manual override: portshim config set server.keep_alive true — keeps the server running between phases for faster iteration.

Model Management

Models are stored in ~/local-models/<model-name>/. The CLI detects available models on startup:

portshim model list            # List all downloaded models
portshim model download HUGGINGFACE_URL  # Download via aria2c
portshim model benchmark       # Run local benchmark

Recommended download tool: PortShim uses aria2c for multi-part downloads. Connect your Hugging Face token for gated models.

Health Monitoring

portshim server status reports:

• Server uptime and process ID
• Current model name and quantisation level
• GPU utilisation and VRAM consumption
• Prompt processing (PP) tokens/second
• Text generation (TG) tokens/second
• Active context window usage (current / max)
• Slot utilisation (how many concurrent requests)

Vulkan GPU Offload

llama-server on PortShim uses Vulkan for GPU inference. The CLI auto-detects GPU capabilities on first run:

portshim diagnose              # Check GPU + Vulkan compatibility
portshim server start --gpu-layers N  # Override layer offload count

On systems without a compatible GPU, PortShim falls back to CPU inference with reduced context size (8K vs 16K).