arXiv:2509.22259v3 Announce Type: replace-cross
Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data. We find that rotating tokens depending on the spectrum of the graph Laplacian efficiently injects structural information into the attention mechanism, boosting performance in synthetic and real-world graph learning tasks. This approach, coined _Wave-Induced Rotary Encodings_ (WIRE), enjoys intriguing theoretical properties: it recovers regular RoPE on grids, and depends asymptotically on the graph effective resistance. Unlike bias-based relative position encodings, WIRE is compatible with linear attention.
Feasibility of PIANO-Cog for older adults: A randomised controlled pilot trial exploring changes in cognition and brain microstructure.
Background: Executive functions are a key target of cognitive interventions for older adults due to their central role in daily functioning and maintaining a good