State-of-the-Art Object Proposal Benchmark

In order to promote reproducible research, this page shows the wraper scripts around the public code for each technique. In all cases, the available instructions were followed and when the API was not clear, the choice that we believed was most sensible was made. Please contact us to discuss how we can improve if you believe we have not treated some technique fairly.

DeepMask and SharpMask

Learning to Refine Object Segments, P. O. Pinheiro, T-Y Lin, R. Collobert, and P. Dollár, ECCV 2016.
Learning to Segment Object Candidates, P. O. Pinheiro, R. Collobert, and P. Dollár, NIPS 2015.
Pre-computed proposals obtained from here.

COB

Convolutional Oriented Boundaries, K.K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool, ECCV 2016.
Code obtained from here.

% Input image 'im'

proposals = im2prop(im);

% Save proposals.superpixels, proposals.labels, proposals.scores

POISE

The Middle Child Problem: Revisiting Parametric Min-cut and Seeds for Object Proposals, A. Humayun, F. Li, and J.M. Rehg, ICCV 2015.
Code obtained from here.

% Input image file 'im_file'

num_seeds = how_many_seeds(num_proposals, 'voc'); % Or 'coco'

[indicator_matrix, superpixels] = gen_obj_proposals(im_file, num_seeds, 'io', true, 'includes_done', true, 'force_recompute', true);

% Save indicator_matrix and superpixels

LPO

Learning to Propose Objects, P. Krähenbühl and V. Koltun, CVPR 2015.
Code obtained from here.

% Input image 'im'

% Startup
init_lpo;

% Set a boundary detector by calling (before creating an OverSegmentation!):
lpo_mex( 'setDetector', 'MultiScaleStructuredForest("data/sf.dat")' );

p = Proposal( 'model', 'models/lpo_VOC_0.03.dat' );

% Create an over-segmentation
os = OverSegmentation( im );

% Generate proposals. This function returns a set of over-segmentations
% and proposals that match it, you should use all over-segmentations
% and all proposals during the evaluation.
[superpixels,indicator_matrix] = p.propose( os );

% Process output
labels = cell(size(indicator_matrix));
for jj=1:length(superpixels)
    
    % Cast superpixels
    assert(size(im,1)==size(superpixels{jj},1))
    assert(size(im,2)==size(superpixels{jj},2))
    superpixels{jj} = uint32(superpixels{jj}+1); % Watch out +1 !!!
    
    % Cast labels
    assert(length(unique(superpixels{jj}))==size(indicator_matrix{jj},2))
    assert(max(superpixels{jj}(:))==size(indicator_matrix{jj},2))
    n_cands = size(indicator_matrix{jj},1);
    labels{jj} = cell(n_cands,1);
    for kk=1:n_cands
        labels{jj}{kk} = uint32(find(indicator_matrix{jj}(kk,:)));
    end
end

% Save labels and superpixels

GOP

Geodesic Object Proposals, P. Krähenbühl and V. Koltun, ECCV 2014.
Code obtained from here (v1.1).

% Python script using binary file 'gop_release'
% Input 'image_id'

my_env = os.environ.copy()
my_env["DYLD_LIBRARY_PATH"] = "/Applications/MATLAB_R2012b.app/bin/maci64"
bin = 'gop_release'

in_file  = imagedir + image_id + '.jpg'
out_file = results_dir + image_id + '.mat'
p = subprocess.Popen([bin, in_file, out_file],env=my_env)
p.wait()

MCG and SCG

Multiscale Combinatorial Grouping, P. Arbeláez, J. Pont-Tuset, J. Barron, F. Marques, and J. Malik, CVPR 2014.
Code obtained from here.

% Input image 'im'
% mode = 'accurate' for MCG
% mode = 'fast' for SCG

proposals = im2mcg(im,mode);

% Save proposals.superpixels, proposals.labels, proposals.scores

GLS

Generating Object Segmentation Proposals using Global and Local Search, P. Rantalankila, J. Kannala, and E. Rahtu, CVPR 2014.
Code obtained from here.

% Input image 'im'

% Load options variable 'opts'
opts = spagglom_options();

% Call the actual code
[labels, orig_sp] = spagglom(im, opts);

% Create superpixels matrix
superpixels = zeros(size(im,1), size(im,2));
for jj=1:length(orig_sp)
    assert(unique(superpixels(orig_sp{jj}.spind))==0);
    superpixels(orig_sp{jj}.spind) = jj;
end
assert(sum(superpixels(:)==0)==0)

% Save labels and superpixels

RIGOR

Recycling Inference in Graph Cuts for generating Object Regions, A. Humayun, F. Li, and J.M. Rehg, CVPR 2014.
Code obtained from here.

% Input image file 'im_file'

[indicator_matrix, superpixels] = rigor_obj_segments(im_file,'pmc_maxflow_method','multiseed_noparallel','io', true,'includes_done',true);

% Save indicator_matrix and superpixels

SeSe

Selective Search for Object Recognition, J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders, IJCV 2013.
Code obtained from here.

% Input image 'im'

% Parameters. Note that this controls the number of hierarchical
% segmentations which are combined.
colorTypes = {'Hsv', 'Lab', 'RGI', 'H', 'Intensity'};

% Here you specify which similarity functions to use in merging
simFunctionHandles = {@SSSimColourTextureSizeFillOrig, @SSSimTextureSizeFill, @SSSimBoxFillOrig, @SSSimSize};

% Thresholds for the Felzenszwalb and Huttenlocher segmentation algorithm.
% Note that by default, we set minSize = k, and sigma = 0.8.
ks = [50 100 150 300]; % controls size of segments of initial segmentation. 
sigma = 0.8;

hBlobs = {};
for j=1:length(ks)
    k = ks(j); % Segmentation threshold k
    minSize = k; % We set minSize = k
    for n = 1:length(colorTypes)
        colorType = colorTypes{n};
        [~, blobIndIm, blobBoxes, hierarchy] = Image2HierarchicalGrouping(im, sigma, k, minSize, colorType, simFunctionHandles);
        if ~isempty(hierarchy)
            hBlobs = cat(1,hBlobs, RecreateBlobHierarchyIndIm(blobIndIm, blobBoxes, hierarchy{1}));
        end
        idx = idx + 1;
    end
end

% Save hBlobs

QT

Baseline method consisting in a homogenous Quad-Tree, regardless of the image content.

Discussion

Would you like to discuss something about the evaluation? Let us know below!