learned-tokenization
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
My attempts at applying Soundstream design on learned tokenization of text and then applying hierarchical attention to text generation
