Deep Learning Architectures: A Review of Convolutional, Recurrent, and Transformer Models
Keywords:
Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Transformer-Based Architectures, Deep Learning (DL)Abstract
Deep learning structures have greatly revolutionized artificial intelligence by offering advanced pattern recognition and automatic features learning using large and complex data. This review includes the complex analysis of three of the most significant deep learning architectures that are Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformer models. CNNs are very efficient in image processing whereas RNNs and their variants e.g. LSTM and GRU are efficient to model sequential and time-varying data. Transformer models, which are driven by the self-attention mechanisms, have transformed the natural language processing and have found their use in the vision and multimodal learning. The review is a synthesis of the recent literature which emphasizes architectural improvements, hybridization, optimization techniques, and domain-specific applications such as healthcare, remote sensing, cybersecurity, genomics and IoT. Even though they perform exceptionally, there are research issues like computational complexity, interpretability, and data dependency, as well as resource demands, that are still of critical importance. The paper offers an understanding of modern trends and the future of deep learning designs.
References
[1] N. L. Rane, S. K. Mallick, Ö. Kaya, and J. Rane, “Machine learning and deep learning architectures and trends: A review,” Deep Sci. Publ., pp. 1–38, 2024.
[2] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A Survey of the Recent Architectures of Deep Convolutional Neural Networks,” Artif. Intell. Rev., pp. 1–70, 2020.
[3] N. U. Ansari and P. Richhariya, “Deep Hybrid Intelligence: CNN-LSTM for Accurate Software Bug Prediction,” Int. J. Innov. Sci. Eng. Manag., vol. 3, no. 4, pp. 26–33, 2024, doi: 10.69968//ijisem.2024v3i426-33.
[4] T. Talaei Khoei, H. Ould Slimane, and N. Kaabouch, “Deep learning: systematic review, models, challenges, and research directions,” Neural Comput. Appl., vol. 35, no. 31, pp. 23103–23124, 2023, doi: 10.1007/s00521-023-08957-4.
[5] P. Dixit and P. B. Singh, “The Impact of Artificial Intelligence on Various Sectors: Benefits, Challenges, and Future Prospects,” Interantional J. Sci. Res. Eng. Manag., vol. 3, no. 2, pp. 140–144, 2024, doi: 10.69968/ijisem.2024v3si2140-144.
[6] M. Aamir, Z. Rahman, N. Choudhry, J. A. BHUTTO, W. A. Abro, and Z. Zhu, “From CNNs to Transformers: A Review of Evolving Deep Learning Architectures for Brain Tumor Classification,” IEEE Access, vol. 4, pp. 1–18, 2016, doi: 10.1109/ACCESS.2025.3625607.
[7] M. Tayebi and S. El Kafhali, “Performance analysis of recurrent neural networks for intrusion detection systems in Industrial-Internet of Things,” Franklin Open, vol. 12, no. 100310, 2025, doi: 10.1016/j.fraope.2025.100310.
[8] I. H. Sarker, “Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions,” SN Comput. Sci., vol. 2, no. 420, pp. 1–20, 2021, doi: 10.1007/s42979-021-00815-1.
[9] I. D. Mienye, T. G. Swart, and G. Obaido, “Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications,” Information, vol. 15, no. 517, pp. 1–34, 2024.
[10] I. D. Mienye and T. G. Swart, “A Comprehensive Review of Deep Learning: Architectures, Recent Advances, and Applications,” Information, vol. 15, no. 755, pp. 1–45, 2024.
[11] E. Algül et al., “A Comparative Study of Advanced Transformer Learning Frameworks for Water Potability Analysis Using Physicochemical Parameters Enes,” Appl. Sci., vol. 15, no. 7262, pp. 1–22, 2025.
[12] T. M. Krishn, A. Haripriya, K. Rekha, and K. Adilakshmi, “Improving Genomic Analysis with Convolutional, Recurrent, and Transformer Neural Networks,” Power Syst. Technol., vol. 48, no. 3, pp. 764–780, 2024.
[13] Y. Tang, F. Qiu, B. Wang, D. Wu, L. Jing, and Z. Sun, “A deep relearning method based on the recurrent neural network for land cover classification,” GIScience Remote Sens., vol. 59, no. 1, pp. 1244–1366, 2022, doi: 10.1080/15481603.2022.2115589.
[14] B. R. Jibinsingh, “Recent Developments in Convolutional Neural Networks for Image-Based Tasks,” in nternational Research on Sustainability and Innovation A Multi Disciplinary Approach to Global Developmen, 2025, pp. 274–283.
[15] M. M. Taye, “Theoretical Understanding of Convolutional Neural Network: Concepts, Architectures, Applications, Future Directions,” Computation, vol. 11, no. 52, pp. 1–23, 2023.
[16] A. Alorf, “Transformer and Convolutional Neural Network: A Hybrid Model for Multimodal Data in Multiclass Classification of Alzheimer’s Disease,” Mathematics, vol. 13, no. 1548, pp. 1–34, 2025.
[17] K. Alomar, H. Ibrahim, and A. Xiaohao, “CNNs, RNNs and Transformers in human action recognition: a survey and a hybrid model,” Artif. Intell. Rev., vol. 58, no. 387, pp. 1–44, 2025.
[18] T. Abdullah and A. Ahmet, “Deep Learning in Sentiment Analysis: Recent Architectures,” ACM Comput. Surv., vol. 55, no. 8, pp. 1–37, 2023, doi: 10.1145/3548772.
[19] C. Wang, “Comparative Evaluation of Transformer-Based and Hybrid Deep Learning Architectures for Supply Chain Demand Forecasting,” in International Conference on Digital Economy and Computer Science, 2025. doi: 10.1145/3785706.3785862.
[20] A. Sirmorya, M. Chaudhari, and S. Balasinor, “Review of Deep Learning: Architectures, Applications and Challenges,” Int. J. Comput. Appl., vol. 184, no. 18, pp. 1–13, 2022.
[21] X. Zhao, L. Wang, Y. Zhang, X. Han, and M. Deveci, A review of convolutional neural networks in computer vision, vol. 57, no. 99. Springer Netherlands, 2024. doi: 10.1007/s10462-024-10721-6.
[22] G. H. H. Nayak et al., “Transformer-based deep learning architecture for time series forecasting,” Softw. Impacts, vol. 22, no. 100716, 2024, doi: 10.1016/j.simpa.2024.100716.
[23] C. Zhou, Y. Liu, X. An, X. Xu, and H. Wang, “Optimization of deep learning architecture based on multi-path convolutional neural network algorithm,” Sci. Rep., vol. 15, no. 19532, pp. 1–12, 2025.
[24] S. R. Rayarao and N. Donikena, “Transformers in Deep Learning: A Comprehensive Technical Review,” Authorea, 2025.
[25] S. Belciug, “Learning deep neural networks’ architectures using differential evolution. Case study: Medical imaging processing,” Comput. Biol. Med., vol. 146, 2022, doi: 10.1016/j.compbiomed.2022.105623.
[26] F. Chen and J. Y. Tsou, “Assessing the effects of convolutional neural network architectural factors on model performance for remote sensing image classification: An in-depth investigation,” Int. J. Appl. Earth Obs. Geoinf., vol. 112, 2022, doi: 10.1016/j.jag.2022.102865.
[27] S. Sumathi, R. Rajesh, and S. Lim, “Recurrent and Deep Learning Neural Network Models for DDoS Attack Detection,” Hindawi J. Sensors, vol. 2022, pp. 1–21, 2022, doi: 10.1155/2022/8530312.
[28] L. Alzubaidi et al., “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions,” J. Big Data, vol. 8, no. 53, pp. 1–74, 2021, doi: 10.1186/s40537-021-00444-8.
