ExLlama

Local LLM Inference Engines Fragment as Community Proposes Modular Library

A community proposal would turn local LLM inference optimizations into reusable, benchmarked modules, lowering deployment barriers and improving resou