Numeric Types in Neural Networks: FP32, BF16, FP8, INT8, and INT4
· â 4 min read · âī¸ k4i
A concise map of floating point, integer quantization, storage dtype, compute dtype, and accumulation dtype in neural networks.
Int8
A Survey of LLM Quantization: From Linear Quantization to Codebooks
· â 34 min read · âī¸ k4i
A practical survey of LLM quantization, covering linear quantization, codebook quantization, LLM.int8(), SmoothQuant, GPTQ, AWQ, NF4, AQLM, KV cache quantization, and FP8.
