Phase-Associative Memory: Sequence Modeling in Complex Hilbert Space

arXiv:2604.05030v2 Announce Type: replace-cross
Abstract: Experiments probing natural language processing by both humans and LLMs suggest that the meaning of a semantic expression is indeterminate prior to the act of interpretation rather than being specifiable simply as the sum of its parts (i.e. compositionality). This observer-dependent act dynamically actualizes meaning under genuine contextuality more consistent with quantum logical mechanisms than with classical Boolean approaches that assume separability, motivating an approach to language modeling that utilizes a Hilbert space formalism. In this work, we introduce Phase-Associative Memory (PAM) — a complex-valued sequence model whose state S_t in mathbbC^d times d accumulates outer products of complex token embeddings retrieved through the conjugate inner product $mathrmRelangle K mid Qrangle / sqrtd$ — and evaluate it against a structurally matched real-valued ablation. Both architectures train stably across a 5M–100M parameter sweep on WikiText-103 under identical conditions; PAM sits at higher absolute loss at every measured scale but improves more rapidly with parameter count, with power-law exponents of $-0.15$ vs. $-0.12$ in loss and $-0.65$ vs. $-0.49$ in perplexity that narrow the gap between the two architectures monotonically. Further investigation of complex-valued sequence modeling at larger scales could reveal that the loss plateau characteristic of real-valued state-of-the-art language models (e.g. transformers) is reachable with PAM-style architectures with an order of magnitude fewer parameters than the current frontier ($sim$1T), implying that similar capabilities are achievable at sizes runnable on consumer-grade hardware.