VAU-R1: Advancing Video Anomaly Understanding via
Reinforcement Fine-Tuning

📊 Results

We evaluate VAU-R1 on the VAU-Bench dataset, which encompasses diverse real-world scenarios from MSAD, UCF-Crime, and ECVA. The evaluation focuses on four key tasks: multiple-choice question answering (QA), temporal anomaly grounding, anomaly classification, and anomaly analysis.

🧠 Multiple-Choice QA

• Reasoning hurts without structure: Base models often underperform when forced to reason via <think> prompts, indicating unstructured reasoning introduces noise.
• RFT consistently boosts performance: Across MSAD and UCF-Crime, RFT significantly improves both Accw/o think and Accw/think, while maintaining output structure.
• Best overall QA accuracy: VAU-R1 (Qwen2.5-VL-3B+RFT) achieves 88.33% QA accuracy on MSAD and 92.03% on UCF-Crime, outperforming all baselines.

🧩 Anomaly Analysis

• RFT improves reasoning depth: On MSAD, Qwen2.5-VL-3B+RFT achieves a total VAU-Eval score of 33.38, outperforming SFT (15.96) and base models.
• Balanced reasoning quality: RFT achieves notable gains in classification accuracy (CLS), key matching (KM), and factual consistency (FAC), providing coherent and factual explanations.
• Generalization to unseen data: On UCF-Crime, RFT models maintain higher VAU-Eval scores despite domain shift, showing better reasoning robustness than SFT.

📍 Temporal Anomaly Grounding

• RFT consistently outperforms base models in both inference modes, improving temporal localization and generalization. Notably, the RFT-finetuned 3B model achieves higher mIoU than the larger 7B base model on ECVA.
• Chain-of-Thought prompting does not always help grounding; in some cases, it slightly degrades performance.
• RFT generalizes better to unseen data. While SFT can occasionally match RFT, its outputs are often repetitive and less interpretable.

Comparison of temporal anomaly grounding performance on the three datasets. we evaluate temporal anomaly grounding on three datasets: MSAD, ECVA, and UCF-Crime. All models are trained only on MSAD and ECVA, while UCF-Crime is treated as an out-of-distribution (OOD) test set to assess cross-dataset generalization.

🧪 Task Co-Training for Anomaly Classification

• TAG boosts perception: RFT with temporal grounding alone yields the best binary (74.14) and strong multi-class (46.14) accuracy under w/ think, underscoring the importance of temporal context.
• QA + TAG are synergistic: Combining QA and TAG improves performance, though TAG alone remains most effective.
• SFT tends to overfit: Despite high binary accuracy (83.37), SFT underperforms in multi-class classification, indicating reduced precision.
• Multi-task RFT balances trade-offs: Jointly training with QA, TAG, and CLS via RFT leads to well-rounded reasoning and accuracy.

Ablation study of task co-training for anomaly classification. Bin. Acc. = binary accuracy (normal vs. abnormal); Multi Acc. = multi-class accuracy across 19 anomaly types plus the normal class.

🔑 Key Insights

• RFT outperforms SFT: Reinforcement Fine-Tuning leads to better reasoning accuracy and stronger generalization compared to standard supervised fine-tuning.
• Interpretability through CoT: While Chain-of-Thought reasoning may not always boost raw visual understanding performance, it greatly enhances the interpretability of model outputs.
• Reward-aligned decomposition matters: Breaking down complex visual tasks into reward-aligned subtasks enables more stable and robust learning during training.

✏️ Case Study

Qualitative case of the QA task. The correct answer is highlighted in orange. RFT yields more precise, interpretable QA choices, while SFT's output is less informative.

Qualitative case of the TAG task. The ground-truth is highlighted in orange. RFT yields more precise anomaly intervals, while SFT’s output is inaccurate.

Qualitative case of the Anomaly Analysis task. Correct descriptions and analyses are highlighted in orange. VAU-R1 identifies the anomaly with high fluency, though omits reasoning for the core event. SFT's output is less accurate and tends to repeat.

📚 Dataset Examples

An explosion case in an outdoor backyard, highlighting complex anomaly detection and dynamic scene understanding. The clip is labeled with a question-answer pair, key visual evidence, anomaly type, and a multi-part reasoning chain covering location, cause-effect, and a high-level conclusion.

An example of a stealing incident, demonstrating capabilities in human activity recognition and intent analysis.

A normal scene, used to evaluate model robustness against false positives and to enhance dataset diversity.

BibTeX

@misc{zhu2025vaur1,
      title={VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning}, 
      author={Liyun Zhu and Qixiang Chen and Xi Shen and Xiaodong Cun},
      year={2025},
      eprint={2505.23504},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.23504}, 
}