Efficient generative adversarial networks using linear additive-attention Transformers

arXiv:2401.09596 · cs.CV, cs.LG · Submitted 2024-01-17 · Read on arXiv

cs.CV, cs.LG

Submitted: 2024-01-17

Updated: 2026-09-16

Comments: Accepted at the 4th LIMIT Workshop, ECCV 2026

Code: https://github.com/milmor/LadaGAN

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Transformer — 72× in this paper · explained in 2 episodes
• Attention Mechanism — 16× in this paper · explained in 2 episodes
• Diffusion Model — 11× in this paper · explained in 8 episodes
• Diffusion Models — 10× in this paper · explained in 6 episodes
• Generative Model — 6× in this paper · explained in 2 episodes
• Generative Models — 6× in this paper · explained in 2 episodes
• Regularization — 6× in this paper · explained in 3 episodes
• Benchmark — 4× in this paper · explained in 2 episodes
• Consistency — 4× in this paper · explained in 3 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Embeddings — 3× in this paper · explained in 2 episodes
• Fidelity — 3× in this paper · explained in 2 episodes
• Generative Adversarial Networks (GANs) — 3× in this paper · explained in 3 episodes
• Latent Space — 3× in this paper · explained in 7 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• Adversarial Training — 2× in this paper · explained in 1 episode
• Data Augmentation — 2× in this paper · explained in 7 episodes
• Loss Function — 2× in this paper · explained in 2 episodes
• Ablated Diffusion Model — 1× in this paper
• Ablation Studies — 1× in this paper · explained in 1 episode
• Activation Function — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Covariate Shift — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Exposure Bias — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Mode Collapse — 1× in this paper · explained in 2 episodes
• Nash Equilibrium — 1× in this paper · explained in 2 episodes
• Precision and Inception Score — 1× in this paper
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• Colorization Transformer
• Augmenting Convolutional networks with attention-based aggregation
• Denoising Diffusion Implicit Models
• Consistency Models
• Fastformer: Additive Attention Can Be All You Need
• Neural Machine Translation by Jointly Learning to Align and Translate
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Input Perturbation Reduces Exposure Bias in Diffusion Models
• Adam: A Method for Stochastic Optimization

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models