TRANSFORMER-BASED ABSTRACTIVE TEXT SUMMARIZATION:A COMPREHENSIVE REVIEW OF MODELS, CHALLENGES, AND EVALUATION
DOI:
https://doi.org/10.5281/zenodo.20457224Keywords:
Abstractive summarization, Transformer, BERT, BART, T5, PEGASUS, ROUGE, hallucination, factual consistency, pre-trained language models, natural language processing, deep learning.Abstract
The exponential growth of digital textual information across news portals, academic repositories, legal corpora, and social media platforms has intensified the need for efficient automated text summarization systems. Transformer-based architectures have emerged as the dominant paradigm in abstractive text summarization, offering superior capabilities in modeling long-range linguistic dependencies and generating fluent, coherent summaries. This review paper provides a comprehensive examination of the development, current state, and open challenges of transformer-based abstractive text summarization. We systematically analyze foundational encoder-decoder architectures, pre-trained language models such as BERT, GPT-2, BART, T5, and PEGASUS, and survey critical research contributions addressing hallucination mitigation, factual consistency, long-document processing, and evaluation methodology. We further review prominent benchmark datasets including CNN/Daily Mail, XSum, MultiNews, and ArXiv, and discuss evaluation metrics spanning ROUGE, BERTScore, BARTScore, and NLI-based faithfulness measures. Our analysis identifies persistent challenges in scalable inference, domain adaptation, multilingual summarization, and reliable automatic evaluation, and outlines promising directions for future research in this rapidly evolving field.
