State-of-the-Art Object Proposal Benchmark

Quantitative evaluation of the current object proposal techniques

Per-Category Evaluation

Per-category Average Best Overlap (ABO) on COCO val 2014. The categories are sorted by category difficulty, that is, by the mean ABO over all state-of-the-art methids. All techniques are evaluated at around 650 proposals per image, so for those techniques producing a non-ranked set of proposals larger than 650, we randomly selected 650 out of them on every image.

Specific Category Evaluation

Click on a specific category in the plot above to see the detailed evaluation on that category. Each technique is in this case evalauted in their "native" number of proposals.

Notes

  • Solid lines represent ranked proposals methods, i.e., you can generate them once, and then pick the top N.
  • Dashed lines joining points represent methods that have to be run with different parameterizations to get different number of proposals.
  • Isolated points where methods run only for one parameterization.
  • Dashed lines without points represent baseline methods.

Legend

  • SharpMaskRegions: Learning to Refine Object Segments, P. O. Pinheiro, T-Y Lin, R. Collobert, and P. Dollár, ECCV 2016.
  • DeepMaskRegions: Learning to Segment Object Candidates, P. O. Pinheiro, R. Collobert, and P. Dollár, NIPS 2015.
  • COB: Convolutional Oriented Boundaries, K.K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool, ECCV 2016.
  • POISE: The Middle Child Problem: Revisiting Parametric Min-cut and Seeds for Object Proposals, A. Humayun, F. Li, and J.M. Rehg, ICCV 2015.
  • MCG and SCG: Multiscale Combinatorial Grouping, P. Arbeláez, J. Pont-Tuset, J. Barron, F. Marques, and J. Malik, CVPR 2014.
  • LPO: Learning to Propose Objects, P. Krähenbühl, and V. Koltun, CVPR 2015.
  • GOP: Geodesic Object Proposals, P. Krähenbühl, and V. Koltun, ECCV 2014.
  • GLS: Generating Object Segmentation Proposals using Global and Local Search, P. Rantalankila, J. Kannala, and E. Rahtu, CVPR 2014.
  • RIGOR: Recycling Inference in Graph Cuts for generating Object Regions, A. Humayun, F. Li, and J.M. Rehg, CVPR 2014.
  • SeSe: Selective Search for Object Recognition, J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders, IJCV 2013.
  • QT: Baseline method consisting in a homogenous Quad-Tree, regardless of the image content.
Is your technique missing although it's published and the code is public? Let us know and we'll add it.

Discussion

Would you like to discuss something about the evaluation? Let us know below!