User Guide
Use Our Dataset
This page explains how to access and use the VidNum-1.4K benchmark, including benchmark questions, evaluation settings, and full video access for reproducible experiments.
Step 1
Download Benchmark JSONL
Download the official benchmark JSONL file with options and answers:
VidNum1_4K.jsonl.
Step 2
Access Full Videos
Full benchmark videos are large (10+ GB), so we do not host them on GitHub. Please use our Hugging Face dataset page for full clip access: https://huggingface.co/datasets/JoeyCCC/VidNum-1.4K.
1. You can download the dataset from our Hugging Face page (the datacuts folder).
2. Video naming rule in this folder: each question corresponds to {oss_link}_{start_time}_{end_time}.mp4.
Step 3
Run Evaluation and Report Results
Use the official NoCoT and ZeroShot-CoT prompting protocols, then report per-level and overall accuracy. For model ranking, submit your aggregate results on the leaderboard page.