VAU-R1: Advancing Video Anomaly Understanding via
Reinforcement Fine-Tuning

📊 Results

We evaluate VAU-R1 on the VAU-Bench dataset, which encompasses diverse real-world scenarios from MSAD, UCF-Crime, and ECVA. The evaluation focuses on four key tasks: multiple-choice question answering (QA), temporal anomaly grounding, anomaly reasoning, and anomaly classification.

🧠 Multi-choice QA & QA-Guided Reasoning

• Naïve reasoning can hurt accuracy: Base models often perform worse when generating answers with reasoning (Accw/think) than without (Accw/o think), suggesting that unguided Chain-of-Thought generation may introduce hallucinations.
• RFT improves both accuracy and reasoning: Reinforcement fine-tuning consistently boosts QA performance with reasoning (e.g., +11.25 on MSAD) and enhances the structure and clarity of generated outputs.
• RFT excels across VAU-Eval dimensions: It delivers consistent gains across classification, key matching, fluency, informativeness, and factuality. Notably, Qwen2.5-VL-3B+RFT achieves the highest total score (33.38) on MSAD, outperforming its SFT counterpart.

📍 Temporal Anomaly Grounding

• RFT consistently outperforms base models in both inference modes, improving temporal localization and generalization. Notably, the RFT-finetuned 3B model achieves higher mIoU than the larger 7B base model on ECVA.
• Chain-of-Thought prompting does not always help grounding; in some cases, it slightly degrades performance.
• RFT generalizes better to unseen data. While SFT can occasionally match RFT, its outputs are often repetitive and less interpretable.

Comparison of temporal anomaly grounding performance on the three datasets. we evaluate temporal anomaly grounding on three datasets: MSAD, ECVA, and UCF-Crime. All models are trained only on MSAD and ECVA, while UCF-Crime is treated as an out-of-distribution (OOD) test set to assess cross-dataset generalization.

🧪 Task Co-Training for Anomaly Classification

• TAG boosts perception: RFT with temporal grounding alone yields the best binary (74.14) and strong multi-class (46.14) accuracy under w/ think, underscoring the importance of temporal context.
• QA + TAG are synergistic: Combining QA and TAG improves performance, though TAG alone remains most effective.
• SFT tends to overfit: Despite high binary accuracy (83.37), SFT underperforms in multi-class classification, indicating reduced precision.
• Multi-task RFT balances trade-offs: Jointly training with QA, TAG, and CLS via RFT leads to well-rounded reasoning and accuracy.

Ablation study of task co-training for anomaly classification. Bin. Acc. = binary accuracy (normal vs. abnormal); Multi Acc. = multi-class accuracy across 19 anomaly types plus the normal class.

🔑 Key Insights

• RFT outperforms SFT: Reinforcement Fine-Tuning leads to better reasoning accuracy and stronger generalization compared to standard supervised fine-tuning.
• Interpretability through CoT: While Chain-of-Thought reasoning may not always boost raw visual understanding performance, it greatly enhances the interpretability of model outputs.
• Reward-aligned decomposition matters: Breaking down complex visual tasks into reward-aligned subtasks enables more stable and robust learning during training.

✏️ Case Study

Qualitative case of the QA task. The correct answer is highlighted in orange. RFT yields more precise, interpretable QA choices, while SFT's output is less informative.

Qualitative case of the TAG task. The ground-truth is highlighted in orange. RFT yields more precise anomaly intervals, while SFT’s output is inaccurate.

Qualitative case of the Anomaly Reasoning task. Correct descriptions and analyses are highlighted in orange. VAU-R1 identifies the anomaly with high fluency, though omits reasoning for the core event. SFT's output is less accurate and tends to repeat.

📚 Dataset Examples

An explosion case in an outdoor backyard, highlighting complex anomaly detection and dynamic scene understanding. The clip is labeled with a question-answer pair, key visual evidence, anomaly type, and a multi-part reasoning chain covering location, cause-effect, and a high-level conclusion.

An example of a stealing incident, demonstrating capabilities in human activity recognition and intent analysis.

A normal scene, used to evaluate model robustness against false positives and to enhance dataset diversity.

BibTeX

@misc{zhu2025vaur1,
      title={VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning}, 
      author={Liyun Zhu and Qixiang Chen and Xi Shen and Xiaodong Cun},
      year={2025},
      eprint={2505.23504},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.23504}, 
}