diff --git a/docs/vram-budgeting.md b/docs/vram-budgeting.md index 808e06b..0c97a56 100644 --- a/docs/vram-budgeting.md +++ b/docs/vram-budgeting.md @@ -47,3 +47,18 @@ Using **`-ctk q8_0 -ctv q8_0`**: - **50% Memory Reduction**: Halves KV cache memory requirements. - **Near-Lossless Perplexity**: Q8_0 maintains >99.8% precision with no degradation in reasoning benchmarks. - **Native ROCm Acceleration**: Supported with Flash Attention (`-fa on`). + +--- + +## 4. Sleep & Hibernation Protection + +### The Problem +On AMD ROCm architectures, entering system suspend (`systemd-suspend`), hybrid sleep, or hibernation with active ~28-30GB VRAM allocations and open `/dev/kfd` contexts causes `amdgpu` driver deadlocks, resulting in full system freezes during or immediately after sleep state transitions. + +### Declarative Systemd Sleep Hook +In `dotfiles` ([`modules/nixos/gpu.nix`](file:///home/noelbautista91/Documents/Projects/dotfiles/modules/nixos/gpu.nix)), a declarative systemd service (`llama-server-sleep-stop.service`) is bound to `sleep.target`: +- Executes synchronously **before** `sleep.target`, `systemd-suspend`, and `systemd-hibernate`. +- Sends `SIGTERM` to allow clean buffer detachment, followed by a hard `SIGKILL` if processes linger. +- Flushes GPU VRAM and closes `/dev/kfd` handles before the kernel initiates D3 power state transition, preventing machine freezes. +- Once resumed, servers can be relaunched instantly with `F5` in `llm-tui` or via Rofi. +