• Home
• Results
  EVALUATION 2012 DATASET RESULTS 2014 DATASET RESULTS
• Datasets
  OVERVIEW 2012 DATASET 2014 DATASET
• Utilities
• Upload
  Upload for 2012 DATASET Upload for 2014 DATASET
• CDW-2012
• CDW-2014

Overview

The main objective of the 2014 IEEE Change Detection Workshop (CDW-2014) is to challenge research teams from around the world to test their change and motion detection algorithms on the expanded and enhanced CDNET video dataset. Compared to CDW-2012, when the challenge included video sequences in 6 categories, the new dataset has been expanded by the addition of 5 new categories:

• Challenging Weather (4 videos, winter weather conditions),
• Low Frame-Rate (4 videos, frame-rate between 0.17 fps and 1 fps),
• Night (6 videos, primarily with motor traffic at night),
• PTZ (1 video with slow continuous camera pan, 1 video with intermittent pan, 1 video with a 2-position patrol-mode PTZ, 1 video with zoom-in/zoom-out),
• Air Turbulence (4 videos showing air turbulence caused by heat),

In addition, whereas ground truths for all frames were made publicly available for the 2012 DATASET for testing and evaluation, in the 2014 DATASET, ground truths of only the first half of every video in the 5 new categories is made publicly available for testing. The evaluation will, however, be across all frames for all the videos (both new and old) as in CDW-2012. This will, we hope, reduce the possibility of overtuning algorithm parameters.

The videos have been selected to cover a wide range of detection challenges and are representative of typical visual data captured today in surveillance, smart environment, and video database scenarios. The dataset includes a comprehensive set of carefully manually-annotated ground-truth motion areas to enable a precise quantitative comparison and ranking of various algorithms. This dataset aims to provide a rigorous academic benchmarking facility for testing and validating existing and new algorithms for motion and change detection.

Several best-performing algorithms submitted to the workshop will be invited for both oral and poster presentation, while several more will be invited for poster presentation only. Papers in both oral and poster sessions will be published in the 2014 CVPR Workshop Proceedings. All submissions that meet minimum standards will be reported in the dataset on-line and in an overview-paper associated with the workshop. The workshop will also include an invited talk and a panel discussion involving a mix of prominent researchers from academia and industry to discuss the current state-of-the-art in motion and change detection algorithms as well as challenges to overcome.

Key Dates

• 31 Mar (Mon) 07 Apr (Mon): Submission of short-paper (up to 4 pages) describing methodology and results.
• 07 Apr (Mon) 11 Apr (Fri): Notification of acceptance.
• 28 Apr (Mon): Submission of camera-ready paper.
• To be determined: Early workshop registration deadline
• 28 Jun (Sat): CDW-2014 workshop.

Workshop held in conjunction with CVPR-2014 June 28, 2014 Program 1:00 – 1:25 Opening remarks and description of the challenge 1:25 – 1:50 "A Fast Self-tuning Background Subtraction Algorithm"    Bin Wang, Piotr Dudek 1:50 – 2:15 "Spectral-360: A Physics-Based Technique for Change Detection"    Mohamed Sedky, Mansour Moniri, Claude C. Chibelushi 2:15 – 2:30 Break 2:30 – 2:55 "Change Detection with Weightless Neural Networks"    Massimo De Gregorio, Maurizio Giordano 2:55 – 3:20 "Flexible Background SubtractionWith Self-Balanced Local Sensitivity"    Pierre-Luc St-Charles, Guillaume-Alexandre Bilodeau, Robert Bergevin 3:20 – 3:50 Break 3:50 – 4:15 "Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models"    Rui Wang, Filiz Bunyak, Guna Seetharaman, Kannappan Palaniappan 4:15 – 4:40 Conclusion and future works

Rules of participation

• Researchers from both the academia and the industry are welcome to submit results
• The 2014 DATASET contains 11 video categories with 4 to 6 video sequences in each category. Results must be reported for each video of each category.
• Only one set of tuning parameters should be used for all videos.
• Numerical scores can be computed using Matlab or Python programs available under UTILITIES. Both programs take the output produced by an algorithm, the ground-truth, and a region-of-interest mask and compute performance metrics described on the 2014 RESULTS page.
• In order for a method to be ranked on this website, upload your results via the 2014 UPLOAD page.
• Methods published in the past can be submitted as long as extensive evaluation over all 11 video categories is performed.

Paper submission

• Prospective authors are invited to submit short papers, up to 4 pages long, describing methodology and results, including figures, tables, and references, by the due date (please see Key Dates above) at the CDW-2014 submission site (Microsoft CMT)
• Each submission must be formatted for double-blind review using one of the templates available at the CVPR 2014 Author Guidelines page
• Submissions not using the above templates or disclosing identity of the authors will be rejected without review.
• Each submission will be peer-reviewed by at least two experts.
• Methods published in the past can be submitted as long as extensive evaluation over all 11 video categories is performed.
• A paper submission implies that, if the paper is accepted, one of the authors, or a proxy, will present the paper at the workshop.
• CDW-2014 has adopted a double-blind review procedure, i.e., the identity of authors is hidden from reviewers and vice versa. Therefore, it is essential that authors remain anonymous in the submitted paper. Please be sure to read the instructions below and those in the templates.
• Please follow these instructions to assure author anonymity. First, an author must register on the submission site and enter paper title and abstract to generate a "Paper ID". This "Paper ID" must replace author names and affiliations in the paper. Secondly, there must be no identifiable self-references in paper content. Many authors misunderstand the concept of anonymizing for blind review. Blind review does not mean that one must remove citations to one’s own work - in fact it is often impossible to review a paper unless the previous citations are known and available. Blind review means that you do not use the words “my” or “our” when citing your previous work. Also, avoid providing information that may identify the authors in the acknowledgments (e.g., co-workers and grant IDs). Avoid providing links to websites that identify the authors. If a paper is accepted, the final version can (and should) include such self-references.

Organizers

Acknowledgment

The 2012 dataset, original website and utilities associated with this benchmarking facility wound not have materialized without the tireless efforts of Masters student Nil Goyette at Université de Sherbrooke nil.goyette@USherbrooke.ca who has given his heart and soul to this marathon undertaking. We would also like to recognize the following individuals for their contributions to this effort:

Results (June 28, 2014)

Metrics :

• Average ranking accross categories : (rank:Baseline + rank:Dynamic Background + rank:Camera Jitter + rank:Intermittent Object Motion + rank:Shadow + rank:Thermal) / 6
• Average ranking : (rank:Recall + rank:Spec + rank:FPR + rank:FNR + rank:PWC + rank:FMeasure + rank:Precision) / 7
• TP : True Positive
• FP : False Positive
• FN : False Negative
• TN : True Negative
• Re (Recall) : TP / (TP + FN)
• Sp (Specficity) : TN / (TN + FP)
• FPR (False Positive Rate) : FP / (FP + TN)
• FNR (False Negative Rate) : FN / (TP + FN)
• PWC (Percentage of Wrong Classifications) : 100 * (FN + FP) / (TP + FN + FP + TN)
• F-Measure : (2 * Precision * Recall) / (Precision + Recall)
• Precision : TP / (TP + FP)
• FPR-S : Average False positive rate in hard shadow areas

References :

• 1. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger, "Comparative study of background subtraction algorithms", J. of Elec. Imaging, 19(3):1–12, 2010.
• 2. A. Elgammal, D. Harwood, and L. Davis, "Non-parametric model for background subtraction", in Proc. Eur. Conf. on Computer Vision, Lect. Notes Comput. Sci. 1843, 751-767 2000.
• 3. Z. Zivkovic and F. van der Heijden, "Efficient adaptive density estimation per image pixel for the task of background subtraction", Pattern Recognition Letters, vol. 27, no. 7, pages 773-780, 2006
• 4. C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking", inProc. Int. Conf. on Computer Vi-sion and Pattern Recognition, Vol. 2, IEEE, Piscataway, NJ (1999).
• 5. Z. Zivkovic, "Improved adaptive Gaussian mixture model for back-ground subtraction", in Proc. Int. Conf. Pattern Recognition, pp. 28-31, IEEE, Piscataway, NJ 2004.
• 6. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger, "Comparative study of background subtraction algorithms", J. of Elec. Imaging, 19(3):1–12, 2010.
• 7. M. De Gregorio and M. Giordano, "Change Detection with Weightless Neural Networks", in proc of IEEE Workshop on Change Detection, 2014.
• 8. M.Sedky, M.Moniri and C. C. Chibelushi, "Spectral-360: A physical-based technique for change detection", in proc of IEEE Workshop on Change Detection, 2014.
• 9. R. Wang, F. Bunyak, G. Seetharaman and K. Palaniappan, "Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models", in proc of IEEE Workshop on Change Detection, 2014.
• 10. L. Maddalena, A. Petrosino, "The SOBS algorithm: what are the limits?", in proc of IEEE Workshop on Change Detection, CVPR 2012.
• 11. B. Wang and P. Dudek, "A Fast Self-tuning Background Subtraction Algorithm", in proc of IEEE Workshop on Change Detection, 2014.
• 12. Dong Liang, Shun'ichi Kaneko, "Improvements and Experiments of a Compact Statistical Background Model", arXiv:1405.6275.
• 13. Xiqun Lu "A multiscale spatio-temporal background model for motion detection", ICIP 2014.
• 14. P-L St-Charles, G-A Bilodeau and R. Bergevin, "Flexible Background Subtraction With Self-Balanced Local Sensitivity" in proc of IEEE Workshop on Change Detection, 2014.

The authors of the top performing method receiving a price from the workshop organizers. From left to right, Pierre-Marc Jodoin, Kannappan Palaniappan, Rui Wang, Fatih Porikli, and Guna Seetharaman.