weight-compression
Neural Weight Compression: lossless LLM weight compression decoded inside the CUDA matvec. BF16 31 % smaller and faster than cuBLAS; fp8 13 % smaller at native fp8 speed. Bit-identical weights.
Run DFloat11 losslessly compressed BF16 checkpoints on Apple Silicon with MLX and a Metal decode kernel; today that runs FLUX.1 on a 32 GB Mac with weights about 30% smaller
