EdgeLM
In progressIndependent · GitHub
The goal: a 3B-parameter LLM at 100–120 tokens a second on consumer Intel hardware with no discrete GPU. The same hardware manages 5–7 today. EdgeLM is an inference engine I’m writing from scratch in C, using BitNet 1.58-bit ternary weights, hand-tuned AVX2 and AVX-VNNI kernels, and the CPU and integrated GPU together. It is the basis for an upcoming paper.
Can LLMs Be Computers?
In progressWith Dr. Dhruv Kumar · CSIS Department, BITS Pilani
Can a transformer do what a computer does, exactly and every time? I’m testing whether LLMs can natively reproduce classical computing primitives: 2D grid transformers, attention that scales as O(n log n) instead of O(n²), and architectures that simulate deterministic state machines.