Complex-Valued Phase-Coherent Transformer
cs.LG
Submitted: 2026-05-11
Updated: 2026-09-24
Code: https://github.com/torchcvnn/torchcvnnhttps:
Terminology
Sources
- Attention Is All You Need
- Deep Complex Networks
- Building Blocks for a Complex-Valued Transformer Architecture
- Holographic Transformers for Complex-Valued Signal Processing: Integrating Phase Interference into Self-Attention
- Rethinking Attention with Performers
- Replacing softmax with ReLU in Vision Transformers
- Theory, Analysis, and Best Practices for Sigmoid Self-Attention
- Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
- Screening Is Enough
- Long Range Arena: A Benchmark for Efficient Transformers
- Efficiently Modeling Long Sequences with Structured State Spaces
- On the Parameterization and Initialization of Diagonal State Space Models
- Diagonal State Spaces are as Effective as Structured State Spaces
- Simplified State Space Layers for Sequence Modeling
- Resurrecting Recurrent Neural Networks for Long Sequences
- Mega: Moving Average Equipped Gated Attention
- DeepNet: Scaling Transformers to 1,000 Layers
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks