GitHub

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

AWQ search for accurate quantization. Pre-computed AWQ model zoo for LLMs (LLaMA, Llama2, OPT, CodeLlama, StarCoder, Vicuna, LLaVA; load to generate quantized weights). Memory-efficient 4-bit Linear ...
GitHub

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Thanks to AWQ, TinyChat can deliver more efficient responses with LLM/VLM chatbots through 4-bit inference. TinyChat on RTX 4090 (3.4x faster than FP16): TinyChat on Jetson Orin (3.2x faster than FP16 ...
IEEE

DL-AQUA: Deep-Learning-Based Automatic Quantization for MMSE MIMO Detection

Abstract: Directly affecting both error performance and complexity, quantization is critical for MMSE MIMO detection. However, naively pruning quantization levels is ...
IEEE

GausiQ: Generalized Automatic Hybrid-Precision Quantization for MIMO Detection

Abstract: Automatic quantization generates efficient hybrid precision quantization schemes without manual effort, offering a promising approach for developing hardware-friendly MIMO detectors. However ...