Lecture 118: GSQ-RCO: Better Non-Uniform GGUF Quantization
GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget.
Read article