Doğukan
@DogukanUrker
running Underdog-Saluki-27B on a single RTX 3060. 12GB VRAM. (config below)
it’s Qwen3.8-27b squeezed into 7.9GB (2-bit) and tuned to keep tool calling intact. dense, so every layer sits on the gpu. no expert offload.
170K context. ~22 tok/s decode, ~490 tok/s prefill.
running
it’s Qwen3.8-27b squeezed into 7.9GB (2-bit) and tuned to keep tool calling intact. dense, so every layer sits on the gpu. no expert offload.
170K context. ~22 tok/s decode, ~490 tok/s prefill.
running
2:48 15 127