# Hyperparameter & llama-server Reference

This document explains key runtime flags used across `llm-runner` and the `llm-tui` form.

---

## 1. Engine & Acceleration Flags

| Flag | Recommended Value | Purpose |
| :--- | :--- | :--- |
| `-ngl, --n-gpu-layers` | `99` | Offload all layers to GPU VRAM. |
| `-fa, --flash-attn` | `on` | Enables Flash Attention kernel for faster generation and lower memory pressure. |
| `--load-mode` | `none` or `mmap` | Use `none` for large models under high VRAM pressure on ROCm to prevent memory stalls; use `mmap` for lighter models. |
| `-ctk` / `-ctv` | `q8_0` | Quantize KV cache vectors to 8-bit. |
| `-b, --batch-size` | `2048` | Logical batch size for prompt processing. |
| `-ub, --ubatch-size` | `512` | Micro-batch size for physical memory chunking. |
| `--reasoning-preserve` | `true` | Preserves `<think>...</think>` tags without stripping them during chat template processing. |

---

## 2. Sampling Hyperparameters

### Reasoning & Planning Profile (SuperQwen / Muse-Glimmer)
Optimized for logical step planning, architecture design, and math:
- **Temperature (`--temp`)**: `0.6` (prevents hallucination while allowing creative problem exploration)
- **Top-P (`--top-p`)**: `0.95`
- **Top-K (`--top-k`)**: `40`
- **Min-P (`--min-p`)**: `0.05` (truncates low-probability noise tokens)

### Instruct / Implementation Profile (Qwen 3.5 4B)
Optimized for precise code edits, syntax consistency, and fast tool execution:
- **Temperature (`--temp`)**: `0.7`
- **Top-P (`--top-p`)**: `0.80`
- **Top-K (`--top-k`)**: `20`
- **Min-P (`--min-p`)**: `0.05`

---

## 3. Multimodal / Vision Projector

When `--mmproj <file>` is passed:
- Enables visual image understanding (screenshots, architecture diagrams, charts).
- `--image-min-tokens 1024`: Ensures sufficient patch token density for grounding and OCR clarity.