Home » Authors » Debopam Sanyal, Hongjie Chen, Riya Elizabeth Kuriyan, Arth Tak, Divya Kothandaraman, Akshay Mehra, Joshua Kimball, Alexey Tumanov

Debopam Sanyal, Hongjie Chen, Riya Elizabeth Kuriyan, Arth Tak, Divya Kothandaraman, Akshay Mehra, Joshua Kimball, Alexey Tumanov

Public reference bibliography and cataloged PDF materials authored or published by Debopam Sanyal, Hongjie Chen, Riya Elizabeth Kuriyan, Arth Tak, Divya Kothandaraman, Akshay Mehra, Joshua Kimball, Alexey Tumanov.

Found 1 document(s) Page 1 of 1

SVD-Attention: Efficiently Analyzing Attention Blocks in Transformers

Year: 2026 • Source: arxiv.org • Listed PDF

Pretrained transformers spend a large share of their inference compute inside self-attention, yet the per-head query-key interaction is structurally low-rank by construction. We ask a diagnostic question: across layers a...