The widespread reliance on open-source software dramatically increases the risk of vulnerability exploitation, underscoring the need for effective and scalable vulnerability detection (VD). Existing VD techniques, whether traditional machine learning-based or LLM-based approaches like prompt engineering, supervised fine-tuning, or off-policy preference optimization, remain fundamentally limited in their ability to perform context-aware analysis: They depend on fixed inputs or static preference datasets, cannot adaptively explore repository-level dependencies, and are constrained by function-level benchmarks that overlook critical vulnerability context. This paper introduces Vulnerability-Adaptive Policy Optimization (VULPO), an on-policy LLM reinforcement learning framework for context-aware VD. To support training and evaluation, we first construct ContextVul, a new dataset that augments high-quality function-level samples with lightweight method to extract repository-level context information. We then design multi-dimensional reward structuring that jointly captures prediction correctness, vulnerability localization accuracy, and the semantic relevance of vulnerability analysis, thereby guiding the model toward comprehensive contextual reasoning. To address the asymmetric difficulty of different vulnerability cases and mitigate reward hacking, VULPO incorporates label-level and sample-level difficulty-adaptive reward scaling, encouraging the model to explore challenging cases while maintaining balanced reward distribution. Extensive experiments demonstrate the superiority of our VULPO framework in context-aware VD: Our VULPO-4B substantially outperforms existing VD baselines based on prompt engineering and off-policy optimization, improving F1 by 85% over Qwen3-4B and achieving performance comparable to a 150x larger-scale model, DeepSeek-R1-0528.
@misc{li2025vulpocontextawarevulnerabilitydetection,title={VULPO: Context-Aware Vulnerability Detection via On-Policy LLM Optimization},author={Li, Youpeng and Yu, Fuxun and Wang, Xinda},year={2025},eprint={2511.11896},archiveprefix={arXiv},primaryclass={cs.CR},}
ASIACCS 2026
Revisiting Pre-trained Language Models for Vulnerability Detection
Youpeng Li, Weiliang Qi, Xuyu Wang, Fuxun Yu, and Xinda Wang
The rapid advancement of pre-trained language models (PLMs) has demonstrated promising results for various code-related tasks. However, their effectiveness in detecting real-world vulnerabilities remains a critical challenge. While existing empirical studies evaluate PLMs for vulnerability detection (VD), they suffer from data leakage, limited scope, and superficial analysis, hindering theaccuracy and comprehensiveness of evaluations. This paper begins by revisiting the common issues in existing research on PLMs for VD through the evaluation pipeline. It then proceeds with an accurate and extensive evaluation of 18 PLMs, spanning model parameters from millions to billions, on high-quality datasets that feature accurate labeling, diverse vulnerability types, and various projects. Specifically, we compare the performance of PLMs under both fine-tuning and prompt engineering, assess their effectiveness and generalizability across various training and testing settings, and analyze their robustness to perturbations such as code normalization, abstraction, and semantic-preserving transformations. Our findings reveal that, for function-level VD, PLMs incorporating pre-training tasks designed to capture the syntactic and semantic patterns of code outperform both general-purpose PLMs and those solely pre-trained or fine-tuned on large code corpora. However, these models face notable challenges in real-world scenarios, such as difficulties in detecting vulnerabilities with complex dependencies, handling perturbations introduced by code normalization and abstraction, and identifying semantic-preserving vulnerable code transformations. Also, the truncation caused by the limited context windows of PLMs can lead to a non-negligible number of labeling errors, which is overlooked by previous work. This study underscores the importance of thorough evaluations of model performance in practical scenarios and outlines future directions to help enhance the effectiveness of PLMs for realistic VD applications.
@misc{li2025revisitingpretrainedlanguagemodels,title={Revisiting Pre-trained Language Models for Vulnerability Detection},author={Li, Youpeng and Qi, Weiliang and Wang, Xuyu and Yu, Fuxun and Wang, Xinda},year={2025},eprint={2507.16887},archiveprefix={arXiv},primaryclass={cs.CR},}
TPS-ISA 2025
MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement
Youpeng Li, Kartik Joshi, Xinda Wang, and Eric Wong
The widespread adoption of open-source software (OSS) necessitates the mitigation of vulnerability risks. Most vulnerability detection (VD) methods are limited by inadequate contextual understanding, restrictive single-round interactions, and coarse-grained evaluations, resulting in undesired model performance and biased evaluation results. To address these challenges, we propose MAVUL, a novel multi-agent VD system that integrates contextual reasoning and interactive refinement. Specifically, a vulnerability analyst agent is designed to flexibly leverage tool-using capabilities and contextual reasoning to achieve cross-procedural code understanding and effectively mine vulnerability patterns. Through iterative feedback and refined decision-making within cross-role agent interactions, the system achieves reliable reasoning and vulnerability prediction. Furthermore, MAVUL introduces multi-dimensional ground truth information for fine-grained evaluation, thereby enhancing evaluation accuracy and reliability. Extensive experiments conducted on a pairwise vulnerability dataset demonstrate MAVUL’s superior performance. Our findings indicate that MAVUL significantly outperforms existing multi-agent systems with over 62% higher pairwise accuracy and single-agent systems with over 600% higher average performance. The system’s effectiveness is markedly improved with increased communication rounds between the vulnerability analyst agent and the security architect agent, underscoring the importance of contextual reasoning in tracing vulnerability flows and the crucial feedback role. Additionally, the integrated evaluation agent serves as a critical, unbiased judge, ensuring a more accurate and reliable estimation of the system’s real-world applicability by preventing misleading binary comparisons.
@misc{li2025mavulmultiagentvulnerabilitydetection,title={MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement},author={Li, Youpeng and Joshi, Kartik and Wang, Xinda and Wong, Eric},year={2025},eprint={2510.00317},archiveprefix={arXiv},primaryclass={cs.CR},}
TPS-ISA 2025
NatGVD: Natural Adversarial Example Attack towards Graph-based Vulnerability Detection
Avilash Rath, Weiliang Qi, Youpeng Li, and Xinda Wang
Graph-based models learn rich code graph structural information and present superior performance on various code analysis tasks. However, the robustness of these models against adversarial example attacks in the context of vulnerability detection remains an open question. This paper proposes NatGVD, a novel attack methodology that generates natural adversarial vulnerable code to circumvent GNN-based and graph-aware transformer-based vulnerability detectors. NatGVD employs a set of code transformations that modify graph structure while preserving code semantics. Instead of injecting dead or unrelated code like previous works, NatGVD considers naturalness requirements: generated examples should not be easily recognized by humans or program analysis tools. With extensive evaluation of NatGVD on state-of-the-art vulnerability detection systems, the results reveal up to 53.04% evasion rate across GNN-based detectors and graph-aware transformer-based detectors. We also explore potential defense strategies to enhance the robustness of these systems against NatGVD.
@misc{rath2025natgvdnaturaladversarialexample,title={NatGVD: Natural Adversarial Example Attack towards Graph-based Vulnerability Detection},author={Rath, Avilash and Qi, Weiliang and Li, Youpeng and Wang, Xinda},year={2025},eprint={2510.04987},archiveprefix={arXiv},primaryclass={cs.CR},}
2024
ACSAC 2024
FedCAP: Robust Federated Learning via Customized Aggregation and Personalization
Youpeng Li, Xinda Wang, Fuxun Yu, Lichao Sun, Wenbin Zhang, and Xuyu Wang
Federated learning (FL), an emerging distributed machine learning paradigm, has been applied to various privacy-preserving scenarios. However, due to its distributed nature, FL faces two key issues: the non-independent and identical distribution (non-IID) of user data and vulnerability to Byzantine threats. To address these challenges, in this paper, we propose FedCAP, a robust FL framework against both data heterogeneity and Byzantine attacks. The core of FedCAP is a model update calibration mechanism to help a server capture the differences in the direction and magnitude of model updates among clients. Furthermore, we design a customized model aggregation rule that facilitates collaborative training among similar clients while accelerating the model deterioration of malicious clients. With a Euclidean norm-based anomaly detection mechanism, the server can quickly identify and permanently remove malicious clients. Moreover, the impact of data heterogeneity and Byzantine attacks can be further mitigated through personalization on the client side. We conduct extensive experiments, comparing multiple state-of-the-art baselines, to demonstrate that FedCAP performs well in several non-IID settings and shows strong robustness under a series of poisoning attacks.
@misc{li2024fedcaprobustfederatedlearning,title={FedCAP: Robust Federated Learning via Customized Aggregation and Personalization},author={Li, Youpeng and Wang, Xinda and Yu, Fuxun and Sun, Lichao and Zhang, Wenbin and Wang, Xuyu},year={2024},month=oct,eprint={2410.13083},archiveprefix={arXiv},primaryclass={cs.LG},booktitle={Annual Computer Security Applications Conference},keywords={federated learning, data heterogeneity, Byzantine-robustness},doi={https://doi.org/10.48550/arXiv.2410.13083}}
IMWUT 2024
SARS: A Personalized Federated Learning Framework Towards Fairness and Robustness against Backdoor Attacks
Weibin Zhang, Youpeng Li, Lingling An, Bo Wan, and Xuyu Wang
Federated Learning (FL), an emerging distributed machine learning framework that enables each client to collaboratively train a global model by sharing local knowledge without disclosing local private data, is vulnerable to backdoor model poisoning attacks. By compromising some users, the attacker manipulates their local training process, and uploads malicious gradient updates to poison the global model, resulting in the poisoned global model behaving abnormally on the sub-tasks specified by the malicious user. Prior research has proposed various strategies to mitigate backdoor attacks. However, existing FL backdoor defense methods affect the fairness of the FL system, while fair FL performance may not be robust. Motivated by these concerns, in this paper, we propose Self-Awareness Revision (SARS), a personalized FL framework designed to resist backdoor attacks and ensure the fairness of the FL system. SARS consists of two key modules: adaptation feature extraction and knowledge mapping. In the adaptation feature extraction module, benign users can adaptively extract clean global knowledge with self-awareness and self-revision of the backdoor knowledge transferred from the global model. Based on the previous module, users can effectively ensure the correct mapping of clean sample features and labels. Through extensive experimental results, SARS can defend against backdoor attacks and improve the fairness of the FL system by comparing several state-of-the-art FL backdoor defenses or fair FL methods, including FedAvg, Ditto, WeakDP, FoolsGold, and FLAME.
@article{10.1145/3678571,author={Zhang, Weibin and Li, Youpeng and An, Lingling and Wan, Bo and Wang, Xuyu},title={SARS: A Personalized Federated Learning Framework Towards Fairness and Robustness against Backdoor Attacks},year={2024},issue_date={December 2024},publisher={Association for Computing Machinery},address={New York, NY, USA},volume={8},number={4},url={https://doi.org/10.1145/3678571},doi={10.1145/3678571},journal={Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},month=nov,articleno={140},numpages={24},keywords={Attention Distillation, Backdoor Attack, Fairness, Federated Learning},}
ACSAC WS 2024
When AI Meets Code Analysis: A Study of Adversarial Attacks on Deep Learning-based Code Models via Program Transformation
Avilash Rath, Youpeng Li, Troy Davis, Braden Bronaugh, Darsh Poddar, Sophia Li, and Xinda Wang
Semantics-preserving program transformations (SPTs) are widely used to generate adversarial example attacks against deep learning-based models for code analysis tasks. This work studies 34 recent representative papers that utilize SPTs to challenge the robustness of code models. We summarize trends in how SPTs are designed, evaluated, and combined in different architectures and tasks. Additionally, we identify core elements and consequences of adversarial attacks against models for code analysis tasks. We perform fine-grained and holistic qualitative analysis of our studied works across 19 metrics to provide a high quality overview of the studied field. Our findings reveal that, while single and combined SPTs are adequately studied separately, there is insufficient joint evaluation of their effects. We also identify critical issues such as inconsistencies in evaluation metrics and limited assessment of a specific category of deep learning-based model. Finally, we evaluate replicability and external validity of existing approaches by inspecting datasets and code availability. Based on the results of our study, we provide recommendations for how developers and researchers should defend against SPT-based attacks and outline serious gaps in the existing literature. Our work aims to guide future research directions to improve the overall robustness of deep learning-based code models against adversarial attacks utilizing SPTs.
@inproceedings{10918233,author={Rath, Avilash and Li, Youpeng and Davis, Troy and Bronaugh, Braden and Poddar, Darsh and Li, Sophia and Wang, Xinda},booktitle={2024 Annual Computer Security Applications Conference Workshops (ACSAC Workshops)},title={When AI Meets Code Analysis: A Study of Adversarial Attacks on Deep Learning-based Code Models via Program Transformation},year={2024},volume={},number={},pages={85-96},doi={10.1109/ACSACW65225.2024.00017},}
2023
IMWUT 2023
Hierarchical Clustering-Based Personalized Federated Learning for Robust and Fair Human Activity Recognition
Currently, federated learning (FL) can enable users to collaboratively train a global model while protecting the privacy of user data, which has been applied to human activity recognition (HAR) tasks. However, in real HAR scenarios, deploying an FL system needs to consider multiple aspects, including system accuracy, fairness, robustness, and scalability. Most existing FL frameworks aim to solve specific problems while ignoring other properties. In this paper, we propose FedCHAR, a personalized FL framework with a hierarchical clustering method for robust and fair HAR, which not only improves the accuracy and the fairness of model performance by exploiting the intrinsically similar relationship between users but also enhances the robustness of the system by identifying malicious nodes through clustering in attack scenarios. In addition, to enhance the scalability of FedCHAR, we also propose FedCHAR-DC, a scalable and adaptive FL framework which is featured by dynamic clustering and adapting to the addition of new users or the evolution of datasets for realistic FL-based HAR scenarios. We conduct extensive experiments to evaluate the performance of FedCHAR on seven datasets of different sizes. The results demonstrate that FedCHAR could obtain better performance on different datasets than the other five state-of-the-art methods in terms of accuracy, robustness, and fairness. We further validate that FedCHAR-DC exhibits satisfactory scalability on three large-scale datasets regardless of the number of participants.
@article{10.1145/3580795,author={Li, Youpeng and Wang, Xuyu and An, Lingling},title={Hierarchical Clustering-Based Personalized Federated Learning for Robust and Fair Human Activity Recognition},year={2023},issue_date={March 2023},publisher={Association for Computing Machinery},address={New York, NY, USA},volume={7},number={1},doi={10.1145/3580795},journal={Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},month=mar,articleno={20},numpages={38},keywords={fairness, attack and defense, human activity recognition, federated learning}}