This Friday we have another

arXiv Dive

with the paper author! We’re presenting Unlimiformer 🎉 with Author Guest: Amanda Bertsch 🎉.

Unlimiformer: Long-Range Transformers with Unlimited Length Input

In this work, the authors propose Unlimiformer: a general approach that wraps any existing pretrained encoder-decoder transformer, and offloads the cross-attention computation to a single k-nearest-neighbor (kNN) index, while the returned kNN distances are the attention dot-product scores.… sign up via link. https://lu.ma/arxivdive-2024-08-02