State-of-the-Art Object Proposal Benchmark

Quantitative evaluation of the current object proposal techniques

Average Best Overlap

This measure first gets the best proposal for each annotated object, computes the Jaccard index (or Overlap, or Intersection over Union) between the best proposal and the groud truth and performs the mean over all objects. We plot this value with respect to the mean number of proposals in all images. Put the mouse on top of some point to see the details.

Object Recall

As before, we first get the best proposal for each annotated object and we compute the Jaccard index between the best proposal and the groud truth. We then count the percentage of objects for which the best proposal's Jaccard (Intersection over Union) is above a certain threshold T. As a summary value, we compute the average recall for different T from 0.5 to 0.95.
Please choose which type of recall to show:

Notes

  • Solid lines represent ranked proposals methods, i.e., you can generate them once, and then pick the top N.
  • Dashed lines joining points represent methods that have to be run with different parameterizations to get different number of proposals.
  • Isolated points where methods run only for one parameterization.
  • Dashed lines without points represent baseline methods.

Legend

  • SharpMaskRegions: Learning to Refine Object Segments, P. O. Pinheiro, T-Y Lin, R. Collobert, and P. Dollár, ECCV 2016.
  • DeepMaskRegions: Learning to Segment Object Candidates, P. O. Pinheiro, R. Collobert, and P. Dollár, NIPS 2015.
  • COB: Convolutional Oriented Boundaries, K.K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool, ECCV 2016.
  • POISE: The Middle Child Problem: Revisiting Parametric Min-cut and Seeds for Object Proposals, A. Humayun, F. Li, and J.M. Rehg, ICCV 2015.
  • MCG and SCG: Multiscale Combinatorial Grouping, P. Arbeláez, J. Pont-Tuset, J. Barron, F. Marques, and J. Malik, CVPR 2014.
  • LPO: Learning to Propose Objects, P. Krähenbühl, and V. Koltun, CVPR 2015.
  • GOP: Geodesic Object Proposals, P. Krähenbühl, and V. Koltun, ECCV 2014.
  • GLS: Generating Object Segmentation Proposals using Global and Local Search, P. Rantalankila, J. Kannala, and E. Rahtu, CVPR 2014.
  • RIGOR: Recycling Inference in Graph Cuts for generating Object Regions, A. Humayun, F. Li, and J.M. Rehg, CVPR 2014.
  • SeSe: Selective Search for Object Recognition, J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders, IJCV 2013.
  • QT: Baseline method consisting in a homogenous Quad-Tree, regardless of the image content.
Is your technique missing although it's published and the code is public? Let us know and we'll add it.

Discussion

Would you like to discuss something about the evaluation? Let us know below!