AI Insight
Researchers have created Short-Films 20K (SF20K), the largest publicly available movie dataset containing 20,143 amateur films totaling 3,582 hours of video, with an average length of 12 minutes per film. The dataset addresses limitations in existing video understanding datasets that focus primarily on short clips with simple narratives. The accompanying benchmark, SF20K-Test, includes 95 movies with 979 question-answer pairs and demonstrates that training vision-language models on this large-scale dataset substantially improves their ability to understand long-term video narratives.
Why it matters
This dataset enables AI systems to better understand complex, feature-length narratives rather than just short video clips, which is essential for applications like automated content analysis, video summarization, and accessibility tools. The use of amateur films minimizes data leakage issues common with commercial movies whose content may have been included in AI training data.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Recent developments in vision-language models have significantly advanced video understanding. Existing datasets and tasks, however, have notable limitations. Most datasets are confined to short videos with limited events and narrow narratives. For example, datasets with instructional and egocentric videos often depict the activities of one person in a single scene. Although existing movie datasets offer richer content, they are often limited to short-term tasks, lack publicly available videos, and frequently encounter data leakage issues given the use of subtitles and other information about commercial movies during LLM pretraining. To address the above limitations, we propose Short-Films 20K (SF20K), the largest publicly available movie dataset. SF20K consists of 20,143 amateur films, amounting to 3,582 hours of video, with an average of 12 minutes per movie. We accompany this dataset with SF20K-Test, a manual, open-ended question answering benchmark. SF20K-Test consists of 95 movies and 979 question-answer pairs. Our extensive analysis of SF20K-Test reveals limited data leakage, emphasizes the need for long-term reasoning, and demonstrates the strong performance of recent VLMs. Finally, we show that instruction tuning on the large-scale dataset substantially improves model performance, paving the way for future progress in long-term video understanding.
Source: Long Story Short: Story-level Video Understanding from 20K Short Films