Results for CD.net 2012
Results, all categories combined.
Warning!!!
Methods with the "(supervised method)" tag
involve a supervised machine learning algorithm potentially trained on the groundtruth data used to produce the metrics reported in this page. Thus, these methods should not be
compared directly with the other unsupervised methods without further investigation and careful analysis. Please refer to the original papers for more details.
Click on method name for more details.
| Method | Average ranking across categories | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 24.67 | 23.86 | 0.7882 | 0.9818 | 0.0182 | 0.2118 | 2.5642 | 0.7159 | 0.7179 |
| GMM | KaewTraKulPong [2] | 26.67 | 23.86 | 0.5072 | 0.9947 | 0.0053 | 0.4928 | 3.1051 | 0.5904 | 0.8228 |
| KDE - ElGammal [3] | 28.33 | 31.57 | 0.7442 | 0.9757 | 0.0243 | 0.2558 | 3.4602 | 0.6719 | 0.6843 |
| GMM | Zivkovic [4] | 35.00 | 30.00 | 0.6964 | 0.9845 | 0.0155 | 0.3036 | 3.1504 | 0.6596 | 0.7079 |
| Mahalanobis distance [5] | 39.17 | 34.29 | 0.7607 | 0.9599 | 0.0401 | 0.2393 | 4.6631 | 0.6259 | 0.6040 |
| Euclidean distance [6] | 40.17 | 35.86 | 0.7048 | 0.9692 | 0.0308 | 0.2952 | 4.3465 | 0.6111 | 0.6223 |
| Local-Self similarity [7] | 34.33 | 29.57 | 0.9354 | 0.8512 | 0.1488 | 0.0646 | 14.2954 | 0.5016 | 0.4139 |
| KDE - Integrated Spatio-temporal Features [8] | 26.17 | 24.29 | 0.6507 | 0.9932 | 0.0068 | 0.3493 | 2.8905 | 0.6418 | 0.7663 |
| PSP-MRF [9] | 18.33 | 20.14 | 0.8037 | 0.9830 | 0.0170 | 0.1963 | 2.3937 | 0.7372 | 0.7512 |
| RMoG (Region-based Mixture of Gaussians) [35] | 22.33 | 20.29 | 0.6042 | 0.9950 | 0.0050 | 0.3958 | 2.7798 | 0.6607 | 0.8247 |
| KDE - Spatio-temporal change detection [10] | 29.33 | 26.71 | 0.6576 | 0.9910 | 0.0090 | 0.3424 | 3.0022 | 0.6437 | 0.7341 |
| GMM | RECTGAUSS-Tex [11] | 34.83 | 32.57 | 0.5156 | 0.9862 | 0.0138 | 0.4844 | 3.6842 | 0.5221 | 0.7190 |
| CDet [36] | 3.50 | 5.00 | 0.9034 | 0.9917 | 0.0083 | 0.0966 | 1.1574 | 0.8608 | 0.8397 |
| PAWCS [37] | 4.00 | 2.00 | 0.8547 | 0.9949 | 0.0051 | 0.1453 | 1.1402 | 0.8579 | 0.8746 |
| SuBSENSE [38] | 7.00 | 5.00 | 0.8281 | 0.9938 | 0.0062 | 0.1719 | 1.5447 | 0.8260 | 0.8576 |
| PBAS [14] | 14.00 | 15.71 | 0.7840 | 0.9898 | 0.0102 | 0.2160 | 1.7693 | 0.7532 | 0.8160 |
| Chebyshev prob. with Static Object detection [15] | 21.33 | 21.86 | 0.7133 | 0.9888 | 0.0112 | 0.2867 | 2.3856 | 0.7001 | 0.7856 |
| SC-SOBS [16] | 21.17 | 20.86 | 0.8017 | 0.9831 | 0.0169 | 0.1983 | 2.4081 | 0.7283 | 0.7315 |
| Bayesian Background [17] | 31.17 | 33.00 | 0.6018 | 0.9826 | 0.0174 | 0.3982 | 3.3879 | 0.6272 | 0.7435 |
| GMM | Stauffer & Grimson [18] | 32.83 | 27.57 | 0.7108 | 0.9860 | 0.0140 | 0.2892 | 3.1037 | 0.6624 | 0.7012 |
| KNN [19] | 25.33 | 24.29 | 0.6707 | 0.9907 | 0.0093 | 0.3293 | 2.7954 | 0.6785 | 0.7882 |
| SBBS [42] | 13.83 | 19.00 | 0.7506 | 0.9859 | 0.0141 | 0.2494 | 2.1276 | 0.7678 | 0.8378 |
| SGMM [21] | 25.33 | 21.14 | 0.7073 | 0.9910 | 0.0090 | 0.2927 | 2.5311 | 0.7008 | 0.7812 |
| pROST [39] | 35.00 | 33.57 | 0.6735 | 0.9790 | 0.0210 | 0.3265 | 3.2534 | 0.6350 | 0.6734 |
| Histogram [23] | 36.83 | 34.71 | 0.7698 | 0.9343 | 0.0657 | 0.2302 | 6.9682 | 0.5485 | 0.5251 |
| CDPS [24] | 21.33 | 21.43 | 0.7769 | 0.9848 | 0.0152 | 0.2231 | 2.2747 | 0.7281 | 0.7610 |
| GRBM [43] | 22.17 | 15.14 | 0.8155 | 0.9879 | 0.0121 | 0.1845 | 1.8146 | 0.7748 | 0.7632 |
| GRBM_without tuning [44] | 25.50 | 26.43 | 0.7589 | 0.9811 | 0.0189 | 0.2411 | 2.7227 | 0.7162 | 0.7206 |
| DPGMM [25] | 15.33 | 14.71 | 0.8275 | 0.9855 | 0.0145 | 0.1725 | 2.1159 | 0.7763 | 0.7928 |
| Spectral-360 [26] | 10.17 | 12.14 | 0.7770 | 0.9920 | 0.0080 | 0.2230 | 1.8516 | 0.7770 | 0.8461 |
| Multi-Layer Background Subtraction [27] | 22.50 | 23.86 | 0.6936 | 0.9888 | 0.0112 | 0.3064 | 2.7658 | 0.6993 | 0.7960 |
| SOBS_CF [34] | 20.17 | 22.14 | 0.8211 | 0.9788 | 0.0212 | 0.1789 | 2.6466 | 0.7273 | 0.7139 |
| SBM [45] | 10.83 | 9.71 | 0.8063 | 0.9919 | 0.0081 | 0.1937 | 1.6820 | 0.8030 | 0.8252 |
| SGMM-SOD [28] | 11.00 | 12.00 | 0.7697 | 0.9938 | 0.0062 | 0.2303 | 1.4960 | 0.7661 | 0.8339 |
| CwisarD [29] | 16.17 | 19.71 | 0.8178 | 0.9781 | 0.0219 | 0.1822 | 2.6607 | 0.7780 | 0.7739 |
| STBM [46] | 8.00 | 8.29 | 0.8350 | 0.9911 | 0.0089 | 0.1650 | 1.6521 | 0.8157 | 0.8210 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 10.83 | 7.57 | 0.7894 | 0.9939 | 0.0061 | 0.2106 | 1.4597 | 0.8092 | 0.8486 |
| Multimode Background Subtraction(MBS) [41] | 8.67 | 7.00 | 0.8103 | 0.9931 | 0.0069 | 0.1897 | 1.2808 | 0.8217 | 0.8480 |
| GPRMF [31] | 14.33 | 19.57 | 0.8372 | 0.9734 | 0.0266 | 0.1628 | 3.1583 | 0.7944 | 0.8144 |
| TUBITAK UZAY 1 [32] | 35.00 | 30.71 | 0.7794 | 0.9756 | 0.0244 | 0.2206 | 3.7014 | 0.6475 | 0.6237 |
| PBAS-PID [33] | 12.67 | 13.86 | 0.7967 | 0.9902 | 0.0098 | 0.2033 | 1.6904 | 0.7720 | 0.8162 |
Results, for the baseline category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 11.57 | 0.9193 | 0.9980 | 0.0020 | 0.0807 | 0.4332 | 0.9251 | 0.9313 |
| GMM | KaewTraKulPong [2] | 24.86 | 0.5863 | 0.9987 | 0.0013 | 0.4137 | 1.9381 | 0.7119 | 0.9532 |
| KDE - ElGammal [3] | 20.00 | 0.8969 | 0.9977 | 0.0023 | 0.1031 | 0.5499 | 0.9092 | 0.9223 |
| GMM | Zivkovic [4] | 30.00 | 0.8085 | 0.9972 | 0.0028 | 0.1915 | 1.3298 | 0.8382 | 0.8993 |
| Mahalanobis distance [5] | 27.71 | 0.8872 | 0.9963 | 0.0037 | 0.1128 | 0.7290 | 0.8954 | 0.9071 |
| Euclidean distance [6] | 30.57 | 0.8385 | 0.9955 | 0.0045 | 0.1615 | 1.0260 | 0.8720 | 0.9114 |
| Local-Self similarity [7] | 27.29 | 0.9732 | 0.9865 | 0.0135 | 0.0268 | 1.3352 | 0.8494 | 0.7564 |
| KDE - Integrated Spatio-temporal Features [8] | 37.71 | 0.7472 | 0.9954 | 0.0046 | 0.2528 | 1.8058 | 0.7392 | 0.7998 |
| PSP-MRF [9] | 12.14 | 0.9319 | 0.9978 | 0.0022 | 0.0681 | 0.4127 | 0.9289 | 0.9261 |
| RMoG (Region-based Mixture of Gaussians) [35] | 27.00 | 0.7082 | 0.9981 | 0.0019 | 0.2918 | 1.5935 | 0.7848 | 0.9125 |
| KDE - Spatio-temporal change detection [10] | 38.29 | 0.7551 | 0.9940 | 0.0060 | 0.2449 | 1.9154 | 0.7554 | 0.7833 |
| GMM | RECTGAUSS-Tex [11] | 28.57 | 0.6669 | 0.9979 | 0.0021 | 0.3331 | 1.5342 | 0.7500 | 0.9175 |
| CDet [36] | 8.14 | 0.9704 | 0.9974 | 0.0026 | 0.0296 | 0.3589 | 0.9458 | 0.9238 |
| PAWCS [37] | 8.43 | 0.9408 | 0.9980 | 0.0020 | 0.0592 | 0.4491 | 0.9397 | 0.9394 |
| SuBSENSE [38] | 4.43 | 0.9520 | 0.9982 | 0.0018 | 0.0480 | 0.3574 | 0.9503 | 0.9495 |
| PBAS [14] | 18.29 | 0.9594 | 0.9970 | 0.0030 | 0.0406 | 0.4858 | 0.9242 | 0.8941 |
| Chebyshev prob. with Static Object detection [15] | 27.86 | 0.8266 | 0.9970 | 0.0030 | 0.1734 | 0.8304 | 0.8646 | 0.9143 |
| SC-SOBS [16] | 8.14 | 0.9327 | 0.9980 | 0.0020 | 0.0673 | 0.3747 | 0.9333 | 0.9341 |
| Bayesian Background [17] | 21.14 | 0.7327 | 0.9984 | 0.0016 | 0.2673 | 0.9037 | 0.8271 | 0.9620 |
| GMM | Stauffer & Grimson [18] | 35.57 | 0.8180 | 0.9948 | 0.0052 | 0.1820 | 1.5325 | 0.8245 | 0.8461 |
| KNN [19] | 24.57 | 0.7934 | 0.9979 | 0.0021 | 0.2066 | 1.2840 | 0.8411 | 0.9245 |
| UBA [20] | 33.43 | 0.9017 | 0.9912 | 0.0088 | 0.0983 | 1.0169 | 0.8132 | 0.7423 |
| SGMM [21] | 32.29 | 0.8680 | 0.9949 | 0.0051 | 0.1320 | 1.2436 | 0.8594 | 0.8584 |
| Quasi-Continuous Histograms based Motion Detection [22] | 41.43 | 0.7044 | 0.9923 | 0.0077 | 0.2956 | 2.2142 | 0.6616 | 0.7009 |
| pROST [39] | 34.43 | 0.8415 | 0.9937 | 0.0063 | 0.1585 | 1.1588 | 0.8289 | 0.8181 |
| Histogram [23] | 23.14 | 0.8777 | 0.9972 | 0.0028 | 0.1223 | 0.6679 | 0.9004 | 0.9254 |
| CDPS [24] | 21.14 | 0.9488 | 0.9965 | 0.0035 | 0.0512 | 0.6238 | 0.9208 | 0.8969 |
| GRBM [43] | 23.43 | 0.9108 | 0.9956 | 0.0044 | 0.0892 | 0.6674 | 0.9143 | 0.9210 |
| GRBM_without tuning [44] | 23.43 | 0.9108 | 0.9956 | 0.0044 | 0.0892 | 0.6674 | 0.9143 | 0.9210 |
| DPGMM [25] | 17.29 | 0.9632 | 0.9969 | 0.0031 | 0.0368 | 0.4949 | 0.9286 | 0.8984 |
| Spectral-360 [26] | 15.29 | 0.9615 | 0.9968 | 0.0032 | 0.0385 | 0.4263 | 0.9330 | 0.9066 |
| Multi-Layer Background Subtraction [27] | 16.71 | 0.8456 | 0.9984 | 0.0016 | 0.1544 | 0.8993 | 0.9004 | 0.9655 |
| SOBS_CF [34] | 11.00 | 0.9347 | 0.9978 | 0.0022 | 0.0653 | 0.3912 | 0.9299 | 0.9254 |
| SBM [45] | 15.86 | 0.9270 | 0.9973 | 0.0027 | 0.0730 | 0.4420 | 0.9250 | 0.9233 |
| SGMM-SOD [28] | 17.57 | 0.9334 | 0.9974 | 0.0026 | 0.0666 | 0.5494 | 0.9212 | 0.9113 |
| CwisarD [29] | 23.43 | 0.8989 | 0.9971 | 0.0029 | 0.1011 | 0.6630 | 0.9075 | 0.9171 |
| STBM [46] | 12.00 | 0.9524 | 0.9973 | 0.0027 | 0.0476 | 0.3840 | 0.9345 | 0.9188 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 12.00 | 0.9158 | 0.9979 | 0.0021 | 0.0842 | 0.4361 | 0.9287 | 0.9431 |
| Multimode Background Subtraction(MBS) [41] | 12.00 | 0.9158 | 0.9979 | 0.0021 | 0.0842 | 0.4361 | 0.9287 | 0.9431 |
| SBBS [42] | 17.71 | 0.9417 | 0.9973 | 0.0027 | 0.0583 | 0.4947 | 0.9192 | 0.8994 |
| GPRMF [31] | 13.29 | 0.9060 | 0.9979 | 0.0021 | 0.0940 | 0.4669 | 0.9280 | 0.9524 |
| TUBITAK UZAY 1 [32] | 37.29 | 0.8936 | 0.9847 | 0.0153 | 0.1064 | 2.0851 | 0.7633 | 0.6767 |
| PBAS-PID [33] | 17.57 | 0.9576 | 0.9971 | 0.0029 | 0.0424 | 0.4862 | 0.9248 | 0.8968 |
Results, for the dynamic background category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 28.57 | 0.8798 | 0.9843 | 0.0157 | 0.1202 | 1.6367 | 0.6439 | 0.5856 |
| GMM | KaewTraKulPong [2] | 23.71 | 0.6303 | 0.9983 | 0.0017 | 0.3697 | 0.5405 | 0.6697 | 0.7700 |
| KDE - ElGammal [3] | 33.00 | 0.8012 | 0.9856 | 0.0144 | 0.1988 | 1.6393 | 0.5961 | 0.5732 |
| SuBSENSE [38] | 10.71 | 0.7768 | 0.9994 | 0.0006 | 0.2232 | 0.4042 | 0.8177 | 0.8915 |
| GMM | Zivkovic [4] | 28.71 | 0.8019 | 0.9903 | 0.0097 | 0.1981 | 1.1725 | 0.6328 | 0.6213 |
| Mahalanobis distance [5] | 35.00 | 0.8132 | 0.9698 | 0.0302 | 0.1868 | 3.1407 | 0.5261 | 0.4517 |
| Euclidean distance [6] | 37.71 | 0.7757 | 0.9714 | 0.0286 | 0.2243 | 3.0095 | 0.5081 | 0.4487 |
| Local-Self similarity [7] | 33.57 | 0.8983 | 0.7694 | 0.2306 | 0.1017 | 22.7868 | 0.0949 | 0.0518 |
| KDE - Integrated Spatio-temporal Features [8] | 27.43 | 0.8401 | 0.9908 | 0.0092 | 0.1599 | 1.1501 | 0.6016 | 0.5413 |
| PSP-MRF [9] | 23.43 | 0.8955 | 0.9859 | 0.0141 | 0.1045 | 1.4514 | 0.6960 | 0.6576 |
| RMoG (Region-based Mixture of Gaussians) [35] | 19.00 | 0.7892 | 0.9978 | 0.0022 | 0.2108 | 0.4238 | 0.7352 | 0.7288 |
| KDE - Spatio-temporal change detection [10] | 23.00 | 0.8935 | 0.9908 | 0.0092 | 0.1065 | 1.0142 | 0.6574 | 0.5888 |
| GMM | RECTGAUSS-Tex [11] | 39.43 | 0.4776 | 0.9838 | 0.0162 | 0.5224 | 1.9735 | 0.4296 | 0.6478 |
| Chebyshev probability approach [12] | 13.71 | 0.8182 | 0.9982 | 0.0018 | 0.1818 | 0.3436 | 0.7656 | 0.7633 |
| PAWCS [37] | 5.86 | 0.8868 | 0.9989 | 0.0011 | 0.1132 | 0.1917 | 0.8938 | 0.9038 |
| Color Histogram Backprojection [13] | 42.86 | 0.6307 | 0.8906 | 0.1094 | 0.3693 | 11.0493 | 0.2675 | 0.1980 |
| PBAS [14] | 20.29 | 0.6955 | 0.9989 | 0.0011 | 0.3045 | 0.5394 | 0.6829 | 0.8326 |
| Chebyshev prob. with Static Object detection [15] | 16.00 | 0.8182 | 0.9976 | 0.0024 | 0.1818 | 0.4086 | 0.7520 | 0.7339 |
| SC-SOBS [16] | 26.86 | 0.8918 | 0.9836 | 0.0164 | 0.1082 | 1.6899 | 0.6686 | 0.6283 |
| Bayesian Background [17] | 34.14 | 0.5962 | 0.9917 | 0.0083 | 0.4038 | 1.2427 | 0.5369 | 0.6898 |
| GMM | Stauffer & Grimson [18] | 27.14 | 0.8344 | 0.9896 | 0.0104 | 0.1656 | 1.2083 | 0.6330 | 0.5989 |
| KNN [19] | 22.86 | 0.8047 | 0.9937 | 0.0063 | 0.1953 | 0.8059 | 0.6865 | 0.6931 |
| SBBS [42] | 10.14 | 0.7772 | 0.9994 | 0.0006 | 0.2228 | 0.2682 | 0.8128 | 0.9037 |
| SGMM [21] | 27.71 | 0.7715 | 0.9933 | 0.0067 | 0.2285 | 0.9132 | 0.6380 | 0.6665 |
| Quasi-Continuous Histograms based Motion Detection [22] | 26.14 | 0.8909 | 0.9896 | 0.0104 | 0.1091 | 1.1301 | 0.6430 | 0.5347 |
| pROST [39] | 30.00 | 0.7314 | 0.9952 | 0.0048 | 0.2686 | 0.6612 | 0.6180 | 0.5969 |
| Histogram [23] | 36.71 | 0.8069 | 0.9401 | 0.0599 | 0.1931 | 6.0488 | 0.2426 | 0.1516 |
| CDPS [24] | 24.00 | 0.7590 | 0.9947 | 0.0053 | 0.2410 | 0.7281 | 0.7495 | 0.8086 |
| GRBM [43] | 21.00 | 0.7011 | 0.9984 | 0.0016 | 0.2989 | 0.4164 | 0.7117 | 0.7463 |
| GRBM_without tuning [44] | 25.00 | 0.6287 | 0.9982 | 0.0018 | 0.3713 | 0.5427 | 0.6754 | 0.7436 |
| DPGMM [25] | 13.71 | 0.8852 | 0.9966 | 0.0034 | 0.1148 | 0.4121 | 0.8137 | 0.7762 |
| STBM [46] | 10.00 | 0.7805 | 0.9994 | 0.0006 | 0.2195 | 0.3087 | 0.8193 | 0.8732 |
| Spectral-360 [26] | 13.71 | 0.7748 | 0.9993 | 0.0007 | 0.2252 | 0.3464 | 0.7872 | 0.8590 |
| Multi-Layer Background Subtraction [27] | 31.29 | 0.7584 | 0.9912 | 0.0088 | 0.2416 | 1.0758 | 0.6278 | 0.6466 |
| SOBS_CF [34] | 26.71 | 0.9014 | 0.9820 | 0.0180 | 0.0986 | 1.8391 | 0.6519 | 0.5953 |
| SBM [45] | 18.43 | 0.7660 | 0.9982 | 0.0018 | 0.2340 | 0.4508 | 0.7882 | 0.8324 |
| SGMM-SOD [28] | 22.43 | 0.7786 | 0.9966 | 0.0034 | 0.2214 | 0.6041 | 0.6883 | 0.7044 |
| CwisarD [29] | 11.71 | 0.8355 | 0.9982 | 0.0018 | 0.1645 | 0.3389 | 0.8086 | 0.8096 |
| DMB [30] | 6.00 | 0.9155 | 0.9987 | 0.0013 | 0.0845 | 0.2282 | 0.8262 | 0.7877 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 20.00 | 0.7637 | 0.9972 | 0.0028 | 0.2363 | 0.4848 | 0.7904 | 0.8606 |
| Multimode Background Subtraction(MBS) [41] | 19.00 | 0.7641 | 0.9972 | 0.0028 | 0.2359 | 0.4845 | 0.7915 | 0.8651 |
| GPRMF [31] | 20.43 | 0.8991 | 0.9877 | 0.0123 | 0.1009 | 1.2694 | 0.7726 | 0.7414 |
| TUBITAK UZAY 1 [32] | 26.57 | 0.8176 | 0.9920 | 0.0080 | 0.1824 | 1.0428 | 0.6078 | 0.5903 |
| CDet [36] | 2.57 | 0.9216 | 0.9992 | 0.0008 | 0.0784 | 0.1503 | 0.8991 | 0.8824 |
| PBAS-PID [33] | 18.71 | 0.7542 | 0.9989 | 0.0011 | 0.2458 | 0.4624 | 0.7357 | 0.8291 |
Results, for the camera jitter category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 21.57 | 0.8007 | 0.9787 | 0.0213 | 0.1993 | 2.7479 | 0.7086 | 0.6399 |
| GMM | KaewTraKulPong [2] | 26.57 | 0.5074 | 0.9888 | 0.0112 | 0.4926 | 3.0233 | 0.5761 | 0.6897 |
| KDE - ElGammal [3] | 30.43 | 0.7375 | 0.9562 | 0.0438 | 0.2625 | 5.1349 | 0.5720 | 0.4862 |
| SuBSENSE [38] | 6.14 | 0.8243 | 0.9908 | 0.0092 | 0.1757 | 1.6469 | 0.8152 | 0.8115 |
| GMM | Zivkovic [4] | 33.86 | 0.6900 | 0.9665 | 0.0335 | 0.3100 | 4.4057 | 0.5670 | 0.4872 |
| Mahalanobis distance [5] | 34.00 | 0.7356 | 0.9431 | 0.0569 | 0.2644 | 6.4390 | 0.4960 | 0.3813 |
| Euclidean distance [6] | 35.86 | 0.7115 | 0.9456 | 0.0544 | 0.2885 | 6.2957 | 0.4874 | 0.3753 |
| Local-Self similarity [7] | 30.29 | 0.9764 | 0.6158 | 0.3842 | 0.0236 | 36.9570 | 0.2074 | 0.1202 |
| KDE - Integrated Spatio-temporal Features [8] | 21.00 | 0.7316 | 0.9857 | 0.0143 | 0.2684 | 2.4238 | 0.7110 | 0.6993 |
| PSP-MRF [9] | 14.71 | 0.8211 | 0.9825 | 0.0175 | 0.1789 | 2.2781 | 0.7502 | 0.7009 |
| RMoG (Region-based Mixture of Gaussians) [35] | 24.14 | 0.6669 | 0.9864 | 0.0136 | 0.3331 | 2.6794 | 0.7010 | 0.7605 |
| KDE - Spatio-temporal change detection [10] | 21.43 | 0.7562 | 0.9816 | 0.0184 | 0.2438 | 2.7450 | 0.7122 | 0.6793 |
| GMM | RECTGAUSS-Tex [11] | 30.71 | 0.7649 | 0.9497 | 0.0503 | 0.2351 | 5.6663 | 0.5370 | 0.4179 |
| CDet [36] | 12.00 | 0.8962 | 0.9816 | 0.0184 | 0.1038 | 2.1987 | 0.8180 | 0.7689 |
| PAWCS [37] | 5.71 | 0.7840 | 0.9935 | 0.0065 | 0.2160 | 1.4220 | 0.8137 | 0.8660 |
| Color Histogram Backprojection [13] | 33.14 | 0.4688 | 0.9821 | 0.0179 | 0.5312 | 3.7175 | 0.4822 | 0.5296 |
| PBAS [14] | 19.43 | 0.7373 | 0.9838 | 0.0162 | 0.2627 | 2.4882 | 0.7220 | 0.7586 |
| Chebyshev prob. with Static Object detection [15] | 28.86 | 0.7223 | 0.9725 | 0.0275 | 0.2777 | 3.6203 | 0.6416 | 0.5960 |
| SC-SOBS [16] | 22.43 | 0.8113 | 0.9768 | 0.0232 | 0.1887 | 2.8794 | 0.7051 | 0.6286 |
| Bayesian Background [17] | 26.57 | 0.5441 | 0.9886 | 0.0114 | 0.4559 | 2.8807 | 0.5988 | 0.6678 |
| GMM | Stauffer & Grimson [18] | 29.29 | 0.7334 | 0.9666 | 0.0334 | 0.2666 | 4.2269 | 0.5969 | 0.5126 |
| KNN [19] | 24.71 | 0.7351 | 0.9778 | 0.0222 | 0.2649 | 3.1104 | 0.6894 | 0.7018 |
| SBBS [42] | 15.57 | 0.7322 | 0.9874 | 0.0126 | 0.2678 | 2.1608 | 0.7347 | 0.7950 |
| SGMM [21] | 19.71 | 0.7088 | 0.9869 | 0.0131 | 0.2912 | 2.3761 | 0.7251 | 0.7752 |
| pROST [39] | 13.57 | 0.7692 | 0.9872 | 0.0128 | 0.2308 | 2.0370 | 0.7478 | 0.7338 |
| Histogram [23] | 37.86 | 0.7111 | 0.8412 | 0.1588 | 0.2889 | 16.2797 | 0.2784 | 0.1756 |
| CDPS [24] | 36.71 | 0.6025 | 0.9613 | 0.0387 | 0.3975 | 5.3593 | 0.4865 | 0.4397 |
| GRBM [43] | 27.00 | 0.7154 | 0.9788 | 0.0212 | 0.2846 | 3.1889 | 0.6555 | 0.6717 |
| GRBM_without tuning [44] | 26.57 | 0.8134 | 0.9594 | 0.0406 | 0.1866 | 4.6494 | 0.6126 | 0.5014 |
| DPGMM [25] | 13.43 | 0.6988 | 0.9930 | 0.0070 | 0.3012 | 1.7707 | 0.7477 | 0.8426 |
| Spectral-360 [26] | 17.57 | 0.6709 | 0.9906 | 0.0094 | 0.3291 | 2.0806 | 0.7156 | 0.8392 |
| Multi-Layer Background Subtraction [27] | 17.71 | 0.6903 | 0.9905 | 0.0095 | 0.3097 | 2.1628 | 0.7311 | 0.7905 |
| SOBS_CF [34] | 20.57 | 0.8218 | 0.9768 | 0.0232 | 0.1782 | 2.8437 | 0.7150 | 0.6405 |
| SBM [45] | 18.29 | 0.7072 | 0.9886 | 0.0114 | 0.2928 | 2.3277 | 0.7413 | 0.7871 |
| SGMM-SOD [28] | 20.29 | 0.6113 | 0.9907 | 0.0093 | 0.3887 | 2.3608 | 0.6724 | 0.8040 |
| CwisarD [29] | 9.43 | 0.7645 | 0.9916 | 0.0084 | 0.2355 | 1.7886 | 0.7814 | 0.8091 |
| STBM [46] | 16.29 | 0.7641 | 0.9852 | 0.0148 | 0.2359 | 2.3825 | 0.7522 | 0.7616 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 3.43 | 0.8321 | 0.9929 | 0.0071 | 0.1679 | 1.5408 | 0.8367 | 0.8443 |
| Multimode Background Subtraction(MBS) [41] | 3.43 | 0.8321 | 0.9929 | 0.0071 | 0.1679 | 1.5408 | 0.8367 | 0.8443 |
| GPRMF [31] | 3.29 | 0.8159 | 0.9953 | 0.0047 | 0.1841 | 1.1062 | 0.8596 | 0.9244 |
| TUBITAK UZAY 1 [32] | 27.71 | 0.8646 | 0.9439 | 0.0561 | 0.1354 | 5.8825 | 0.5661 | 0.4247 |
| PBAS-PID [33] | 20.71 | 0.7210 | 0.9853 | 0.0147 | 0.2790 | 2.4659 | 0.7206 | 0.7586 |
Results, for the intermittent object motion category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 25.57 | 0.7057 | 0.9507 | 0.0493 | 0.2943 | 6.1324 | 0.5628 | 0.5531 |
| GMM | KaewTraKulPong [2] | 26.86 | 0.3476 | 0.9892 | 0.0108 | 0.6524 | 5.9854 | 0.3903 | 0.6953 |
| KDE - ElGammal [3] | 36.71 | 0.5035 | 0.9309 | 0.0691 | 0.4965 | 10.0695 | 0.4088 | 0.4609 |
| GMM | Zivkovic [4] | 26.14 | 0.5467 | 0.9712 | 0.0288 | 0.4533 | 5.4986 | 0.5325 | 0.6458 |
| Mahalanobis distance [5] | 32.43 | 0.7165 | 0.8886 | 0.1114 | 0.2835 | 11.5341 | 0.4968 | 0.4535 |
| Euclidean distance [6] | 33.00 | 0.5919 | 0.9336 | 0.0664 | 0.4081 | 8.9975 | 0.4892 | 0.4995 |
| Local-Self similarity [7] | 28.57 | 0.9027 | 0.8222 | 0.1778 | 0.0973 | 15.8827 | 0.5329 | 0.4445 |
| KDE - Integrated Spatio-temporal Features [8] | 16.71 | 0.4512 | 0.9964 | 0.0036 | 0.5488 | 4.4191 | 0.5454 | 0.8166 |
| PSP-MRF [9] | 25.14 | 0.7010 | 0.9530 | 0.0470 | 0.2990 | 6.0594 | 0.5645 | 0.5727 |
| RMoG (Region-based Mixture of Gaussians) [35] | 17.86 | 0.4488 | 0.9950 | 0.0050 | 0.5512 | 4.6882 | 0.5431 | 0.8026 |
| KDE - Spatio-temporal change detection [10] | 22.00 | 0.4372 | 0.9923 | 0.0077 | 0.5628 | 4.6997 | 0.5039 | 0.7212 |
| GMM | RECTGAUSS-Tex [11] | 25.86 | 0.2190 | 0.9977 | 0.0023 | 0.7810 | 5.2547 | 0.3146 | 0.5850 |
| CDet [36] | 5.14 | 0.8865 | 0.9891 | 0.0109 | 0.1135 | 1.6116 | 0.8039 | 0.7821 |
| PAWCS [37] | 4.71 | 0.7487 | 0.9945 | 0.0055 | 0.2513 | 2.3536 | 0.7764 | 0.8392 |
| SuBSENSE [38] | 12.57 | 0.6578 | 0.9915 | 0.0085 | 0.3422 | 3.8349 | 0.6569 | 0.7957 |
| PBAS [14] | 20.14 | 0.6700 | 0.9751 | 0.0249 | 0.3300 | 4.2871 | 0.5745 | 0.7045 |
| Chebyshev prob. with Static Object detection [15] | 29.14 | 0.3570 | 0.9807 | 0.0193 | 0.6430 | 6.4700 | 0.3863 | 0.7688 |
| SC-SOBS [16] | 20.57 | 0.7237 | 0.9613 | 0.0387 | 0.2763 | 5.2207 | 0.5918 | 0.5896 |
| Bayesian Background [17] | 37.43 | 0.4813 | 0.9304 | 0.0696 | 0.5187 | 9.9632 | 0.4081 | 0.4747 |
| GMM | Stauffer & Grimson [18] | 23.71 | 0.5142 | 0.9835 | 0.0165 | 0.4858 | 5.1955 | 0.5207 | 0.6688 |
| KNN [19] | 23.29 | 0.4617 | 0.9865 | 0.0135 | 0.5383 | 5.1370 | 0.5026 | 0.7121 |
| UBA [20] | 13.00 | 0.7205 | 0.9827 | 0.0173 | 0.2795 | 3.0544 | 0.6886 | 0.7310 |
| SGMM [21] | 22.29 | 0.5013 | 0.9853 | 0.0147 | 0.4987 | 4.9180 | 0.5397 | 0.6993 |
| Quasi-Continuous Histograms based Motion Detection [22] | 31.00 | 0.4407 | 0.9797 | 0.0203 | 0.5593 | 6.0490 | 0.4367 | 0.5384 |
| pROST [39] | 35.00 | 0.5156 | 0.9317 | 0.0683 | 0.4844 | 8.5201 | 0.4127 | 0.4740 |
| Histogram [23] | 29.86 | 0.7512 | 0.8656 | 0.1344 | 0.2488 | 13.1359 | 0.5112 | 0.4859 |
| CDPS [24] | 10.86 | 0.8084 | 0.9765 | 0.0235 | 0.1916 | 3.4650 | 0.7406 | 0.7624 |
| GRBM [43] | 5.57 | 0.8467 | 0.9875 | 0.0125 | 0.1533 | 2.4804 | 0.8115 | 0.8023 |
| GRBM_without tuning [44] | 22.00 | 0.7466 | 0.9540 | 0.0460 | 0.2534 | 6.7084 | 0.5987 | 0.5623 |
| DPGMM [25] | 26.71 | 0.6763 | 0.9470 | 0.0530 | 0.3237 | 6.8457 | 0.5418 | 0.6525 |
| Spectral-360 [26] | 21.00 | 0.5945 | 0.9811 | 0.0189 | 0.4055 | 5.4443 | 0.5656 | 0.7192 |
| Multi-Layer Background Subtraction [27] | 31.14 | 0.5012 | 0.9629 | 0.0371 | 0.4988 | 7.0245 | 0.4816 | 0.6024 |
| SOBS_CF [34] | 23.14 | 0.7641 | 0.9381 | 0.0619 | 0.2359 | 6.7454 | 0.5810 | 0.5464 |
| SBM [45] | 14.00 | 0.7068 | 0.9850 | 0.0150 | 0.2932 | 4.1545 | 0.6755 | 0.7352 |
| SGMM-SOD [28] | 7.86 | 0.7363 | 0.9909 | 0.0091 | 0.2637 | 2.5238 | 0.7151 | 0.8141 |
| CwisarD [29] | 25.71 | 0.7847 | 0.9003 | 0.0997 | 0.2153 | 10.0314 | 0.5674 | 0.5013 |
| STBM [46] | 14.00 | 0.7166 | 0.9837 | 0.0163 | 0.2834 | 4.1910 | 0.6780 | 0.7220 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 10.57 | 0.6386 | 0.9931 | 0.0069 | 0.3614 | 3.1858 | 0.7092 | 0.8201 |
| Multimode Background Subtraction(MBS) [41] | 8.86 | 0.7418 | 0.9862 | 0.0138 | 0.2582 | 2.3008 | 0.7568 | 0.7827 |
| SBBS [42] | 19.86 | 0.7616 | 0.9399 | 0.0601 | 0.2384 | 6.3180 | 0.6795 | 0.6772 |
| GPRMF [31] | 33.71 | 0.6101 | 0.8753 | 0.1247 | 0.3899 | 13.0733 | 0.4870 | 0.5863 |
| TUBITAK UZAY 1 [32] | 28.43 | 0.5823 | 0.9645 | 0.0355 | 0.4177 | 6.3265 | 0.5091 | 0.5533 |
| PBAS-PID [33] | 17.86 | 0.7048 | 0.9759 | 0.0241 | 0.2952 | 3.9085 | 0.6267 | 0.7055 |
Results, for the shadow category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision | Average FPR-S |
|---|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 30.43 | 0.8350 | 0.9836 | 0.0164 | 0.1650 | 2.3366 | 0.7716 | 0.7219 | 0.5689 |
| GMM | KaewTraKulPong [2] | 23.00 | 0.6323 | 0.9936 | 0.0064 | 0.3677 | 2.3015 | 0.7176 | 0.8577 | 0.4069 |
| KDE - ElGammal [3] | 23.86 | 0.8536 | 0.9885 | 0.0115 | 0.1464 | 1.6881 | 0.8028 | 0.7660 | 0.6217 |
| GMM | Zivkovic [4] | 31.00 | 0.7770 | 0.9878 | 0.0122 | 0.2230 | 2.1957 | 0.7319 | 0.7232 | 0.5428 |
| Mahalanobis distance [5] | 39.00 | 0.7845 | 0.9708 | 0.0292 | 0.2155 | 3.7896 | 0.6348 | 0.5685 | 0.5899 |
| Euclidean distance [6] | 36.14 | 0.8001 | 0.9783 | 0.0217 | 0.1999 | 2.8987 | 0.6785 | 0.6112 | 0.5763 |
| Local-Self similarity [7] | 31.71 | 0.9584 | 0.9442 | 0.0558 | 0.0416 | 5.5496 | 0.5951 | 0.4673 | 0.6377 |
| KDE - Integrated Spatio-temporal Features [8] | 20.57 | 0.7197 | 0.9930 | 0.0070 | 0.2803 | 2.1292 | 0.7545 | 0.8244 | 0.3901 |
| PSP-MRF [9] | 26.00 | 0.8736 | 0.9829 | 0.0171 | 0.1264 | 2.2414 | 0.7907 | 0.7281 | 0.5861 |
| RMoG (Region-based Mixture of Gaussians) [35] | 23.71 | 0.6680 | 0.9936 | 0.0064 | 0.3320 | 2.1720 | 0.7212 | 0.8073 | 0.3097 |
| KDE - Spatio-temporal change detection [10] | 31.00 | 0.6970 | 0.9898 | 0.0102 | 0.3030 | 2.4865 | 0.7136 | 0.7559 | 0.3977 |
| GMM | RECTGAUSS-Tex [11] | 31.29 | 0.7189 | 0.9886 | 0.0114 | 0.2811 | 2.4111 | 0.7331 | 0.7840 | 0.4764 |
| Chebyshev probability approach [12] | 18.14 | 0.8669 | 0.9887 | 0.0113 | 0.1331 | 1.5552 | 0.8333 | 0.8104 | 0.4204 |
| PAWCS [37] | 4.29 | 0.9172 | 0.9932 | 0.0068 | 0.0828 | 1.0230 | 0.8913 | 0.8710 | 0.4815 |
| SuBSENSE [38] | 3.71 | 0.9419 | 0.9920 | 0.0080 | 0.0581 | 1.0120 | 0.8986 | 0.8646 | 0.5996 |
| PBAS [14] | 11.57 | 0.9133 | 0.9904 | 0.0096 | 0.0867 | 1.2753 | 0.8597 | 0.8143 | 0.5789 |
| Chebyshev prob. with Static Object detection [15] | 18.86 | 0.8670 | 0.9887 | 0.0113 | 0.1330 | 1.5561 | 0.8333 | 0.8103 | 0.4204 |
| SC-SOBS [16] | 29.57 | 0.8502 | 0.9834 | 0.0166 | 0.1498 | 2.3000 | 0.7786 | 0.7230 | 0.6035 |
| Bayesian Background [17] | 29.29 | 0.6537 | 0.9916 | 0.0084 | 0.3463 | 2.4695 | 0.6955 | 0.7791 | 0.3293 |
| GMM | Stauffer & Grimson [18] | 30.43 | 0.7960 | 0.9871 | 0.0129 | 0.2040 | 2.1951 | 0.7370 | 0.7156 | 0.5352 |
| KNN [19] | 23.71 | 0.7478 | 0.9916 | 0.0084 | 0.2522 | 2.0569 | 0.7468 | 0.7788 | 0.3979 |
| UBA [20] | 32.00 | 0.9084 | 0.9707 | 0.0293 | 0.0916 | 3.2250 | 0.7123 | 0.6095 | 0.6147 |
| SGMM [21] | 22.57 | 0.8580 | 0.9889 | 0.0111 | 0.1420 | 1.7965 | 0.7944 | 0.7617 | 0.4865 |
| Quasi-Continuous Histograms based Motion Detection [22] | 34.00 | 0.6949 | 0.9887 | 0.0113 | 0.3051 | 2.5870 | 0.7072 | 0.7378 | 0.3483 |
| pROST [39] | 37.71 | 0.7541 | 0.9791 | 0.0209 | 0.2459 | 2.9907 | 0.6765 | 0.6239 | 0.5167 |
| Histogram [23] | 38.00 | 0.8308 | 0.9686 | 0.0314 | 0.1692 | 3.7098 | 0.6589 | 0.6009 | 0.5881 |
| CDPS [24] | 20.57 | 0.9233 | 0.9846 | 0.0154 | 0.0767 | 1.9516 | 0.8092 | 0.7567 | 0.5902 |
| GRBM [43] | 22.71 | 0.8725 | 0.9857 | 0.0143 | 0.1275 | 1.7683 | 0.8046 | 0.7559 | 0.5946 |
| GRBM_without tuning [44] | 27.00 | 0.7427 | 0.9902 | 0.0098 | 0.2573 | 1.9146 | 0.7405 | 0.7553 | 0.4654 |
| DPGMM [25] | 15.71 | 0.8545 | 0.9916 | 0.0084 | 0.1455 | 1.5947 | 0.8127 | 0.8240 | 0.4179 |
| Spectral-360 [26] | 7.57 | 0.9366 | 0.9905 | 0.0095 | 0.0634 | 1.1784 | 0.8843 | 0.8412 | 0.6213 |
| Multi-Layer Background Subtraction [27] | 16.43 | 0.8588 | 0.9912 | 0.0088 | 0.1412 | 1.5621 | 0.8216 | 0.8099 | 0.4879 |
| SOBS_CF [34] | 28.14 | 0.8699 | 0.9828 | 0.0172 | 0.1301 | 2.2579 | 0.7721 | 0.7045 | 0.5899 |
| SBM [45] | 13.14 | 0.8629 | 0.9910 | 0.0090 | 0.1371 | 1.3780 | 0.8458 | 0.8459 | 0.4363 |
| SGMM-SOD [28] | 10.43 | 0.9191 | 0.9902 | 0.0098 | 0.0809 | 1.2534 | 0.8646 | 0.8226 | 0.6343 |
| CwisarD [29] | 15.57 | 0.8872 | 0.9897 | 0.0103 | 0.1128 | 1.3757 | 0.8412 | 0.8056 | 0.5552 |
| STBM [46] | 13.86 | 0.8979 | 0.9896 | 0.0104 | 0.1021 | 1.3643 | 0.8529 | 0.8221 | 0.5742 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 20.43 | 0.7762 | 0.9918 | 0.0082 | 0.2238 | 1.5794 | 0.7784 | 0.8063 | 0.3481 |
| Multimode Background Subtraction(MBS) [41] | 15.86 | 0.7920 | 0.9924 | 0.0076 | 0.2080 | 1.4940 | 0.7968 | 0.8262 | 0.3481 |
| SBBS [42] | 21.29 | 0.5981 | 0.9970 | 0.0030 | 0.4019 | 1.8693 | 0.7105 | 0.8934 | 0.1228 |
| GPRMF [31] | 4.71 | 0.9253 | 0.9922 | 0.0078 | 0.0747 | 1.0712 | 0.8889 | 0.8671 | 0.4932 |
| TUBITAK UZAY 1 [32] | 31.86 | 0.8594 | 0.9768 | 0.0232 | 0.1406 | 2.8893 | 0.7509 | 0.6843 | 0.5651 |
| CDet [36] | 12.29 | 0.9259 | 0.9892 | 0.0108 | 0.0741 | 1.4108 | 0.8644 | 0.8122 | 0.6611 |
| PBAS-PID [33] | 10.86 | 0.9115 | 0.9907 | 0.0093 | 0.0885 | 1.2606 | 0.8617 | 0.8193 | 0.5747 |
Results, for the thermal category.
Click on method name for more details.
| Method | Average ranking | Average Re | Average Sp | Average FPR | Average FNR | Average PWC | Average F-Measure | Average Precision |
|---|---|---|---|---|---|---|---|---|
| SOBS [1] | 24.29 | 0.5888 | 0.9956 | 0.0044 | 0.4112 | 2.0983 | 0.6834 | 0.8754 |
| GMM | KaewTraKulPong [2] | 24.29 | 0.3395 | 0.9993 | 0.0007 | 0.6605 | 4.8419 | 0.4767 | 0.9709 |
| KDE - ElGammal [3] | 17.00 | 0.6725 | 0.9955 | 0.0045 | 0.3275 | 1.6795 | 0.7423 | 0.8974 |
| GMM | Zivkovic [4] | 28.86 | 0.5542 | 0.9942 | 0.0058 | 0.4458 | 4.3002 | 0.6548 | 0.8706 |
| Mahalanobis distance [5] | 29.71 | 0.6270 | 0.9906 | 0.0094 | 0.3730 | 2.3462 | 0.7065 | 0.8617 |
| Euclidean distance [6] | 33.00 | 0.5111 | 0.9907 | 0.0093 | 0.4889 | 3.8516 | 0.6313 | 0.8877 |
| Local-Self similarity [7] | 26.14 | 0.9036 | 0.9692 | 0.0308 | 0.0964 | 3.2612 | 0.7297 | 0.6433 |
| KDE - Integrated Spatio-temporal Features [8] | 26.14 | 0.4147 | 0.9981 | 0.0019 | 0.5853 | 5.4152 | 0.4989 | 0.9164 |
| PSP-MRF [9] | 19.86 | 0.5991 | 0.9962 | 0.0038 | 0.4009 | 1.9189 | 0.6932 | 0.9218 |
| RMoG (Region-based Mixture of Gaussians) [35] | 24.86 | 0.3441 | 0.9991 | 0.0009 | 0.6559 | 5.1222 | 0.4788 | 0.9365 |
| KDE - Spatio-temporal change detection [10] | 28.00 | 0.4065 | 0.9973 | 0.0027 | 0.5935 | 5.1527 | 0.5199 | 0.8761 |
| GMM | RECTGAUSS-Tex [11] | 25.57 | 0.2461 | 0.9994 | 0.0006 | 0.7539 | 5.2656 | 0.3682 | 0.9619 |
| Chebyshev probability approach [12] | 14.00 | 0.6940 | 0.9962 | 0.0038 | 0.3060 | 1.3285 | 0.7259 | 0.8910 |
| PAWCS [37] | 17.57 | 0.8504 | 0.9910 | 0.0090 | 0.1496 | 1.4018 | 0.8324 | 0.8280 |
| SuBSENSE [38] | 22.00 | 0.8161 | 0.9908 | 0.0092 | 0.1839 | 2.0125 | 0.8171 | 0.8328 |
| PBAS [14] | 17.00 | 0.7283 | 0.9934 | 0.0066 | 0.2717 | 1.5398 | 0.7556 | 0.8922 |
| Chebyshev prob. with Static Object detection [15] | 14.71 | 0.6887 | 0.9963 | 0.0037 | 0.3113 | 1.4283 | 0.7230 | 0.8906 |
| SC-SOBS [16] | 22.43 | 0.6003 | 0.9957 | 0.0043 | 0.3997 | 1.9841 | 0.6923 | 0.8857 |
| Bayesian Background [17] | 23.43 | 0.6026 | 0.9952 | 0.0048 | 0.3974 | 2.8676 | 0.6969 | 0.8877 |
| GMM | Stauffer & Grimson [18] | 28.14 | 0.5691 | 0.9946 | 0.0054 | 0.4309 | 4.2642 | 0.6621 | 0.8652 |
| KNN [19] | 25.29 | 0.4817 | 0.9970 | 0.0030 | 0.5183 | 4.3783 | 0.6046 | 0.9186 |
| UBA [20] | 23.29 | 0.6880 | 0.9939 | 0.0061 | 0.3120 | 1.6684 | 0.7283 | 0.7962 |
| SGMM [21] | 22.71 | 0.5363 | 0.9970 | 0.0030 | 0.4637 | 3.9394 | 0.6481 | 0.9263 |
| Quasi-Continuous Histograms based Motion Detection [22] | 28.57 | 0.3350 | 0.9982 | 0.0018 | 0.6650 | 5.1493 | 0.4651 | 0.8784 |
| pROST [39] | 39.14 | 0.4290 | 0.9872 | 0.0128 | 0.5710 | 4.1526 | 0.5260 | 0.7936 |
| Histogram [23] | 26.86 | 0.6412 | 0.9933 | 0.0067 | 0.3588 | 1.9669 | 0.6996 | 0.8110 |
| CDPS [24] | 20.43 | 0.6195 | 0.9950 | 0.0050 | 0.3805 | 1.5205 | 0.6619 | 0.9014 |
| GRBM [43] | 26.57 | 0.8464 | 0.9812 | 0.0188 | 0.1536 | 2.3663 | 0.7511 | 0.6818 |
| GRBM_without tuning [44] | 24.86 | 0.7114 | 0.9893 | 0.0107 | 0.2886 | 1.8537 | 0.7558 | 0.8397 |
| DPGMM [25] | 21.43 | 0.8869 | 0.9882 | 0.0118 | 0.1131 | 1.5773 | 0.8134 | 0.7629 |
| Spectral-360 [26] | 16.14 | 0.7238 | 0.9939 | 0.0061 | 0.2762 | 1.6337 | 0.7764 | 0.9114 |
| Multi-Layer Background Subtraction [27] | 21.43 | 0.5072 | 0.9986 | 0.0014 | 0.4928 | 3.8704 | 0.6331 | 0.9611 |
| SOBS_CF [34] | 21.29 | 0.6347 | 0.9953 | 0.0047 | 0.3653 | 1.8021 | 0.7140 | 0.8715 |
| SBM [45] | 16.14 | 0.8677 | 0.9914 | 0.0086 | 0.1323 | 1.3390 | 0.8423 | 0.8275 |
| SGMM-SOD [28] | 14.71 | 0.6396 | 0.9971 | 0.0029 | 0.3604 | 1.6846 | 0.7353 | 0.9471 |
| CwisarD [29] | 22.14 | 0.7357 | 0.9918 | 0.0082 | 0.2643 | 1.7664 | 0.7619 | 0.8007 |
| STBM [46] | 15.14 | 0.8986 | 0.9913 | 0.0087 | 0.1014 | 1.2822 | 0.8571 | 0.8280 |
| Multimode Background Subtraction Version 0 (MBS V0) [40] | 21.29 | 0.8101 | 0.9908 | 0.0092 | 0.1899 | 1.5315 | 0.8115 | 0.8174 |
| Multimode Background Subtraction(MBS) [41] | 17.57 | 0.8162 | 0.9920 | 0.0080 | 0.1838 | 1.4289 | 0.8194 | 0.8268 |
| SBBS [42] | 19.71 | 0.6929 | 0.9941 | 0.0059 | 0.3071 | 1.6545 | 0.7499 | 0.8579 |
| GPRMF [31] | 19.57 | 0.8666 | 0.9917 | 0.0083 | 0.1334 | 1.9628 | 0.8305 | 0.8145 |
| TUBITAK UZAY 1 [32] | 29.14 | 0.6589 | 0.9920 | 0.0080 | 0.3411 | 3.9821 | 0.6877 | 0.8127 |
| CDet [36] | 13.00 | 0.8200 | 0.9940 | 0.0060 | 0.1800 | 1.2142 | 0.8337 | 0.8686 |
| PBAS-PID [33] | 16.57 | 0.7308 | 0.9936 | 0.0064 | 0.2692 | 1.5590 | 0.7622 | 0.8881 |
Results for methods [1, 3, 5, 21] have been obtained by the organizing committee using authors' original code. Results for methods [2, 4, 6, 7] have been obtained by the organizing committee using their own implementation or OpenCV. A 5x5 median filter has been applied in a post-processing step.
Metrics:
- Average ranking across categories : (rank:Baseline + rank:Dynamic Background + rank:Camera Jitter + rank:Intermittent Object Motion + rank:Shadow + rank:Thermal) / 6
- Average ranking : (rank:Recall + rank:Spec + rank:FPR + rank:FNR + rank:PWC + rank:FMeasure + rank:Precision) / 7
- TP : True Positive
- FP : False Positive
- FN : False Negative
- TN : True Negative
- Re (Recall) : TP / (TP + FN)
- Sp (Specificity) : TN / (TN + FP)
- FPR (False Positive Rate) : FP / (FP + TN)
- FNR (False Negative Rate) : FN / (TP + FN)
- PWC (Percentage of Wrong Classifications) : 100 * (FN + FP) / (TP + FN + FP + TN)
- F-Measure : (2 * Precision * Recall) / (Precision + Recall)
- Precision : TP / (TP + FP)
- FPR-S : Average False positive rate in hard shadow areas
References:
- 1. L. Maddalena, A. Petrosino, A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications, IEEE Transactions on Image Processing, Vol. 17, no.7, 2008, p1168-1177
- 2. P. KaewTraKulPong and R. Bowden, "An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection", in proc of Workshop on Advanced Video Based Surveillance Systems, 2001
- 3. A. Elgammal, D. Harwood, and L. Davis, "Non-parametric model for background subtraction," in Proc. Eur. Conf. on Computer Vision, Lect. Notes Comput. Sci. 1843, 751-767 2000.
- 4. Z. Zivkovic, "Improved adaptive Gaussian mixture model for back-ground subtraction," in Proc. Int. Conf. Pattern Recognition, pp. 28-31, IEEE, Piscataway, NJ 2004
- 5. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger. Comparative study of background subtraction algorithms.J. of Elec. Imaging, 19(3):1–12, 2010.
- 6. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger. Comparative study of background subtraction algorithms.J. of Elec. Imaging, 19(3):1–12, 2010.
- 7. J-P Jodoin, G-A Bilodeau, N Saunier "Background subtraction based on Local Shape", arXiv:1204.6326v1
- 8. Y.Nonaka, A. Shimada, H.Nagahara, R. Taniguchi "Evaluation Report of Integrated Background Modeling Based on Spatio-temporal Features", in proc of IEEE Workshop on Change Detection, 2012
- 9. A. Schick, M.Bäuml, R.Stiefelhagen "Improving Foreground Segmentations with Probabilistic Superpixel Markov Random Fields", in proc of IEEE Workshop on Change Detection, 2012
- 10. Satoshi Yoshinaga, Atsushi Shimada, Hajime Nagahara, Rin-ichiro Taniguchi, Background Model Based on Intensity Change Similarity Among Pixels, the 19th Japan-Korea Joint Workshop on Frontiers of Computer Vision, pp.276-280, 2013.01
- 11. Riahi, D., St-Onge, P.L., Bilodeau, G.A. (2012). RECTGAUSS-Tex: Block-based Background Subtraction, Technical Report, École Polytechnique de Montréal, EPM-RT-2012-03
- 12. A. Morde, X. Ma, S. Guler "Learning a background model for change detection", in proc of IEEE Workshop on Change Detection, 2012 [IntuVision]
- 13. D. Kit, B. T. Sullivan, and D. H. Ballard. Novelty detection using growing neural gas for visuo-spatial memory. In IROS, pages 1194–1200. IEEE, 2011.
- 14. M. Hofmann, P.Tiefenbacher, G. Rigoll "Background Segmentation with Feedback: The Pixel-Based Adaptive Segmenter", in proc of IEEE Workshop on Change Detection, 2012
- 15. A. Morde, X. Ma, S. Guler [IntuVision] "Learning a background model for change detection", in proc of IEEE Workshop on Change Detection, 2012
- 16. L. Maddalena, A. Petrosino, "The SOBS algorithm: what are the limits?", in proc of IEEE Workshop on Change Detection, CVPR 2012
- 17. F. Porikli and O. Tuzel. "Bayesian background modeling for foreground detection" in proc. of ACM Visual Surveillance and Sensor Network, 2005.
- 18. C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking," inProc. Int. Conf. on Computer Vi-sion and Pattern Recognition, Vol. 2, IEEE, Piscataway, NJ (1999)
- 19. Efficient adaptive density estimation per image pixel for the task of background subtraction" Z. Zivkovic , F. van der Heijden Pattern Recognition Letters, vol. 27, no. 7, pages 773-780, 2006
- 20. D. Park, H. Byun, Object-wise multilayer background ordering for pubic area surveillance", in IEEE International Conference on AVSS, 2009
- 21. Heras Evangelio, R. and Pätzold, M. and Sikora, T. "Splitting Gaussians in Mixture Models", Proceedings of the 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, 2012
- 22. O. Strauss, D. Sidibé, W. Puech "Quasi-continuous histogram based motion detection",Technical Report, LE2I, 2012
- 23. J. Zheng, Y. Wang, N.L. Nihan, and M.E. Hallenbeck. Extracting roadway background image : Mode-based approach. Transportation Research Record : Journal of the Transportation Research Board, 1944(-1) :82–88, 2006
- 24. Francisco J. Hernandez-Lopez and Mariano Rivera. Change Detection by Probabilistic Segmentation from Monocular View. Machine Vision and Applications, pages 1-21, 2013.
- 25. T. S. F. Haines and T. Xiang "Background Subtraction with Dirichlet Processes" ECCV 2012
- 26. SEDKY Mohamed Hamed Ismail, MONIRI Mansour and CHIBELUSHI Claude Chilufya “Object Segmentation Using Full-Spectrum Matching of Albedo Derived from Colour Images” UK patent application no. 0822953.6 16.12.2008 GB, 2008, PCT patent application international application no. PCT/GB2009/002829, EP2374109, 2009, US patent no. 2374109 12.10.2011 US, 2011
- 27. Jian Yao and Jean-Marc Odobez, Multi-Layer Background Subtraction Based on Color and Texture. Visual Surveillance workshop (CVPR-VS), Minneapolis, June 2007.
- 28. R. Heras and T. Sikora "Complementary Background Models for the Detection of Static and Moving Objects in Crowded Environments", in 8th Proceedings of the IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS)
- 29. M. De Gregorio and M. Giordano "A WiSARD-based approach to CDnet" 1st BRICS Countries Congress (BRICS-CCI) - 2013
- 30. Yin, Baocai; Zhang, Jing; Wang, Zengfu: 'Background segmentation of dynamic scenes based on dual model', IET Computer Vision, 2014, DOI: 10.1049/iet-cvi.2013.0319 IET Digital Library, http://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2013.0319;
- 31. Wang N, Yao T, Wang J, et al. A probabilistic approach to robust matrix factorization[M]//Computer Vision–ECCV 2012. Springer Berlin Heidelberg, 2012: 126-139.
- 32. Lee, Dar-Shyang. "Effective Gaussian mixture learning for video background subtraction." Pattern Analysis and Machine Intelligence, IEEE Transactions on 27.5 (2005): 827-832.
- 33. Tiefenbacher P, Hofmann M, Merget D, et al. PID-based regulation of background dynamics for foreground segmentation[C]//Image Processing (ICIP). 2014: 3282-3286.
- 34. L. Maddalena, A. Petrosino, A Fuzzy Spatial Coherence-based Approach to Background/ Foreground Separation for Moving Object Detection, Neural Computing and Applications, Springer London, Vol. 19, pp. 179–186, 2010
- 35. Varadarajan, S.; Miller, P.; Huiyu Zhou, "Spatial mixture of Gaussians for dynamic background modelling," Advanced Video and Signal Based Surveillance (AVSS), 2013 10th IEEE International Conference on , pp.63,68, 27-30 Aug. 2013
- 36. Anonymous
- 37. P.-L. St-Charles, G.-A. Bilodeau, R. Bergevin, "A Self-Adjusting Approach to Change Detection Based on Background Word Consensus" in IEEE Winter Conference on Applications of Computer Vision (WACV). Big Island, Hawaii, USA, Jan. 6-9, 2015.
- 38. St-Charles, Pierre-Luc, Guillaume-Alexandre Bilodeau, and Robert Bergevin. "SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity." Image Processing, IEEE Transactions on 24.1 (2015): 359-373.
- 39. F. Seidel, C. Hage, M. Kleinsteuber "pROST - A Smoothed Lp-norm Robust Online Subspace Tracking Method for Realtime Background Subtraction in Video". In Machine Vision and Applications, 25(5):1227-1240, 2014
- 40. Sajid, Hasan, and Sen-Ching Samson Cheung. "Background subtraction for static & moving camera." In Image Processing (ICIP), 2015 IEEE International Conference on, pp. 4530-4534. IEEE, 2015.
- 41. Sajid, Hasan, and Sen-Ching Samson Cheung. "Universal multimode background subtraction." IEEE Transactions on Image Processing 26, no. 7 (2017): 3249-3260
- 42. Varghese, Arun, and G. Sreelekha. "Sample-based integrated background subtraction and shadow detection." IPSJ Transactions on Computer Vision and Applications 9.1 (2017): 25. (https://doi.org/10.1186/s41074-017-0036-1)
- 43. Background subtraction using GRBM
- 44. Anonymous
- 45. M. Chen, Q. Yang, Q. Li, G. Wang, and M. Yang, “Spatiotemporal background subtraction using minimum spanning tree and optical flow,” in ECCV, 2014, pp. 521–534.
- 46. M. Chen, Q. Yang, Q. Li, G. Wang, and M. Yang, “Spatiotemporal background subtraction using minimum spanning tree and optical flow,” in ECCV, 2014, pp. 521–534.