Image and Video Segmentation @ ETHZ CVL

From evaluation to State-of-the-Art Results

Deep Extreme Cut: From Extreme Points to Object Segmentation

DEXTR explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos. We do so by adding an extra channel to the image in the input of a convolutional neural network (CNN), which contains a Gaussian centered in each of the extreme points.

One-Shot Video Object Segmentation

OSVOS is based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one-shot). OSVOS is very fast and improves the state of the art by a significant margin.

Convolutional Oriented Boundaries

COB is a technique that produces multiscale oriented contours and region hierarchies from a single forward pass of a generic image classification CNN. The whole pipeline runs in the order of 0.8 seconds per image to produce state-of-the-art hierarchical boundaries.

Retinal Image Segmentation and Understanding

This project presents Deep Retinal Image Understanding (DRIU), a unified framework of retinal image analysis that provides both retinal vessel and optic disc segmentation using deep Convolutional Neural Networks (CNNs). We show both qualitative and quantitative experimental validation in four public datasets, on which DRIU presents super-human performance.

Video Object Segmentation Benchmark

In this project we present a new benchmark dataset and evaluation methodology for video object segmentation - DAVIS (Densely Annotated VIdeo Segmentation), which consists of fifty high quality, Full HD video sequences, accompanied by densely annotated, pixel-accurate and per-frame ground truth segmentation. We provide a comprehensive analysis of several state-of-the-art segmentation approaches using three complementary metrics.

Image Segmentation Evaluation

This project tackles the supervised evaluation of image segmentation algorithms, it proposes a new measure: the precision-recall for objects and parts, and it defines three quantitative meta-measures involving six state of the art segmentation methods. As a conclusion, this paper proposes the precision-recall curves for boundaries and for objects-and-parts as the tool of choice for the supervised evaluation of image segmentation.

Segmentation Hierarchies Alignment

This work tries to modify the existing hierarchical algorithm by improving their alignment, that is, by trying to modify the depth of the regions in the tree to better couple depth and scale.

Boosting and Benchmarking Segmented Object Proposals

In-depth evaluation of object proposals and the challenges faced when generalizing the experiments from PASCAL to COCO. Interactive browsing of all the state of the art.

Object Proposals: Multiscale Combinatorial Grouping

State-of-the-art object proposals starting from a segmentation hierarchy and generating combinations of pairs and triplets of neighboring regions.

Publications:

  • K.K. Maninis, S. Caelles, Y. Chen, J. Pont-Tuset, L. Leal-Taixé and L. Van Gool
    Video Object Segmentation Without Temporal Information
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018
    [BibTeX] [PDF] [Project Page] 

    @article{Man+18b,
  author 	= {K.-K. Maninis and S. Caelles and Y. Chen and J. Pont-Tuset and L. Leal Taix\'{e} and L. {Van Gool}},
  title 	= {Video Object Segmentation Without Temporal Information},
  journal	= {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year 		= {2018}
}

  • K.K. Maninis, S. Caelles, J. Pont-Tuset and L. Van Gool
    Deep Extreme Cut: From Extreme Points to Object Segmentation
    Computer Vision and Pattern Recognition (CVPR), 2018
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Man+18,
  author 	= {K.-K. Maninis and S. Caelles and J. Pont-Tuset and L. {Van Gool}},
  title 	= {Deep Extreme Cut: From Extreme Points to Object Segmentation},
  booktitle	= {Computer Vision and Pattern Recognition (CVPR)},
  year 		= {2018}
}

  • K.K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool
    Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI),
    vol. 40, no. 4, pp. 819 - 833, 2018.    , 2018
    [BibTeX] [PDF] [Project Page] 

    @article{Maninis2017,
  author = {K.K. Maninis and J. Pont-Tuset and P. Arbel\'{a}ez and L. Van Gool},
  title = {Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year = {2017},
  volume = {40},
  number = {4},
  pages = {819 - 833}
}

  • K.K. Maninis, J. Pont-Tuset, P. Arbeláez and L. Van Gool
    Convolutional Oriented Boundaries
    European Conference on Computer Vision (ECCV), 2016
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Maninis2016a,
  author = {K.K. Maninis and J. Pont-Tuset and P. Arbel\'{a}ez and L. Van Gool},
  title = {Convolutional Oriented Boundaries},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year = {2016}
}

  • Y. Chen, D. Dai, J. Pont-Tuset and L. Van Gool
    Scale-Aware Alignment of Hierarchical Image Segmentation
    Computer Vision and Pattern Recognition (CVPR), 2016
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Chen2016,
  author = {Y. Chen and D. Dai and J. Pont-Tuset and L. Van Gool},
  title = {Scale-Aware Alignment of Hierarchical Image Segmentation},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2016}
}

  • J. Pont-Tuset and F. Marques
    Supervised Evaluation of Image Segmentation and Object Proposal Techniques
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 38, no. 7, pp. 1465-1478, 2016
    [BibTeX] [PDF] [Project Page] 

    @article{Pont-Tuset2016a,
  author = {Jordi Pont-Tuset and Ferran Marques},
  title = {Supervised Evaluation of Image Segmentation and Object Proposal Techniques},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year = {2016},
  volume = {38},
  number = {7},
  pages = {1465--1478}
}

  • J. Pont-Tuset and L. Van Gool
    Boosting Object Proposals: From Pascal to COCO
    International Conference on Computer Vision (ICCV), 2015
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Pont-Tuset2015b,
  author = {Jordi Pont-Tuset and Luc Van Gool},
  title = {Boosting Object Proposals: From Pascal to COCO},
  booktitle = {International Conference on Computer Vision (ICCV)},
  year = {2015}
}

  • S. Caelles, K.K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool
    One-Shot Video Object Segmentation
    Computer Vision and Pattern Recognition (CVPR), 2017
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Caelles2017,
  author = {S. Caelles and K.-K. Maninis and J. Pont-Tuset and L. Leal-Taix\'e and D. Cremers and L. {Van Gool},
  title = {One-Shot Video Object Segmentation},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2017}
}

  • F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross and A. Sorkine-Hornung
    A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
    Computer Vision and Pattern Recognition (CVPR), 2016
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Perazzi2016,
  author = {F. Perazzi and J. Pont-Tuset and B. McWilliams and L. Van Gool and M. Gross and A. Sorkine-Hornung},
  title = {A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2016}
}

  • K.K. Maninis, J. Pont-Tuset, P. Arbeláez and L. Van Gool
    Deep Retinal Image Understanding
    Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2016
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Maninis2016,
  author = {K.K. Maninis and J. Pont-Tuset and P. Arbeláez and L. Van Gool},
  title = {Deep Retinal Image Understanding},
  booktitle = {Medical Image Computing and Computer-Assisted Intervention (MICCAI)},
  year = {2016}
}

  • J. Pont-Tuset, P. Arbeláez, J. Barron, F. Marques and J. Malik
    Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 39, no. 1, pp. 128-140, 2017
    [BibTeX] [PDF] [Project Page] 

    @article{Pont-Tuset2017,
  author = {J. Pont-Tuset and P. Arbel\'{a}ez and J. Barron and F.Marques and J. Malik},
  title = {Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year = {2017},
  volume = {39},
  number = {1},
  pages = {128 - 140}
}

  • P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marques and J. Malik
    Multiscale Combinatorial Grouping
    Computer Vision and Pattern Recognition (CVPR), 2014
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Arbelaez2014,
  author = {Arbelaez, Pablo and Pont-Tuset, J. and Barron, Jon and Marques, F. and Malik, Jitendra},
  title = {Multiscale Combinatorial Grouping},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2014}
}

  • J. Pont-Tuset and F. Marques
    Measures and Meta-Measures for the Supervised Evaluation of Image Segmentation
    Computer Vision and Pattern Recognition (CVPR), 2013
    [BibTeX] [PDF] [Project Page] 

    @inproceedings{Pont-Tuset2013,
  author = {Pont-Tuset, J. and Marques, F.},
  title = {Measures and Meta-Measures for the Supervised Evaluation of Image Segmentation},
  booktitle = {Computer Vision and Pattern Recognition (CVPR)},
  year = {2013}
}

Team:

/readinggroup//readinggroup/