Software description:
https://github.com/LUT-FPGA/LUT-LLM
LUT-LLM is the first FPGA accelerator that deploy 1B+ language model with memory-based computation, leveraging vector quantization. LUT-LLM features:
- Activation-weight Co-quantization: shrinked lookup tables with comparable accuracy compared with standard scalar quantization schemes.
- Bandwidth-aware Parallel Centroid Search: tradeoffs between resource consumption for parallel search and latency of pipeline propagation during decoding.
- Efficient 2D table lookup: extract rows and then copy to reduce fanout with low on-chip capacity required per operation at runtime.
- Temporal-Spatial Hybrid Execution: LUT-LLM sequentially execute between LUTLinear and other engines, and keep dataflow inside each engine.
Publications:
Year:
2026