Impact of Full Fine-Tuning on Performance of Bert-Family Models in Source Code Vulnerability Detection Task
Abstract
Deep learning-based approaches for source code vulnerability detection have gained considerable attention in software security. Among existing pretrained language models, BERT, CodeBERT, and GraphCodeBERT are widely adopted; however, their performance under a unified fine-tuning strategy has not been systematically examined. This study conducts a comparative assessment of the three models in the context of vulnerability detection, employing a full fine-tuning setup in which all model parameters are updated on task-specific training data. Experiments are carried out on three representative datasets, Devign, Big-Vul, MegaVul, and Juliet, covering diverse vulnerability patterns and levels of semantic complexity. The results indicate that GraphCodeBERT consistently delivers the strongest performance, particularly for vulnerabilities governed by data-flow semantics; CodeBERT shows stable behavior on pattern-oriented cases, whereas BERT lags behind due to its lack of pretraining on programming languages. The empirical findings highlight how architectural differences and pretraining objectives influence downstream vulnerability detection. This work establishes a reliable and reproducible baseline for subsequent studies in a broader investigation of fine-tuning strategies for language models in code analysis.
How to Cite This Article
Nin Ho Le Viet, Tuan Nguyen Kim, Cuong Dang Van, Chieu Ta Quang (2026). Impact of Full Fine-Tuning on Performance of Bert-Family Models in Source Code Vulnerability Detection Task . International Journal of Future Engineering Innovations (IJFEI), 3(3), 26-35. DOI: https://doi.org/10.54660/IJFEI.2026.3.3.26-35