Hardware-Aware Computational Analysis of Vision Transformer Attention Mechanisms
IEEE Access, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Basım Tarihi: 2026
- Doi Numarası: 10.1109/access.2026.3741681
- Dergi Adı: IEEE Access
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC, Directory of Open Access Journals
- Anahtar Kelimeler: Attention mechanisms, computer vision, hardware-aware analysis, vision transformers
- Atatürk Üniversitesi Adresli: Evet
Özet
Transformers have become a dominant paradigm in modern deep learning, achieving state-of-the-art performance across computer vision, natural language processing, and generative artificial intelligence. However, their high computational and memory demands remain a major limitation, particularly for real-time and resource-constrained applications. Inspired by prior studies on Convolutional Neural Networks (CNNs), where systematic analysis of convolutional kernel designs has significantly improved efficiency and deployment, a similar perspective is extended to self-attention mechanisms in Vision Transformers (ViTs). A comprehensive benchmarking study is conducted on three representative vision transformer architectures: the standard ViT-B/16 with global self-attention, a ViT-B/16 incorporating a Performer-inspired kernel-based linear self-attention mechanism, and Swin-T with shifted-window self-attention. The analysis includes parameter count, FLOPs, memory access, inference time, power consumption, and classification accuracy, with particular emphasis on runtime decomposition and hardware-aware evaluation across edge-oriented platforms. The results indicate that kernel-based linear self-attention significantly reduces computational complexity compared with global self-attention while maintaining comparable memory access patterns, leading to improved real-time efficiency. In contrast, although Swin-T reduces FLOPs through shifted-window self-attention, it introduces increased memory access due to frequent data reorganization operations, limiting its practical performance gains on memory-bound hardware. These findings establish that reducing FLOPs alone is insufficient for real-time optimization and demonstrate that arithmetic intensity and memory access provide complementary insights for designing efficient next-generation self-attention mechanisms..