GSQ-RCO: Better Non-Uniform GGUF Quantization
GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO optimizes bit allocation under fixed size budgets, enabling state-of-the-art model compression.
Read article