-
FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures
Authors:
Lisa Mais,
Peter Hirsch,
Claire Managan,
Ramya Kandarpa,
Josef Lorenz Rumberger,
Annika Reinke,
Lena Maier-Hein,
Gudrun Ihrke,
Dagmar Kainmueller
Abstract:
Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-rangi…
▽ More
Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-ranging, thin filamentous and widely branching morphologies, multiple neurons are tightly inter-weaved, and partial volume effects, uneven illumination and noise inherent to light microscopy severely impede local disentangling as well as long-range tracing of individual neurons. These properties reflect a current key challenge in machine learning research, namely to effectively capture long-range dependencies in the data. While respective methodological research is buzzing, to date methods are typically benchmarked on synthetic datasets. To address this gap, we release the FlyLight Instance Segmentation Benchmark (FISBe) dataset, the first publicly available multi-neuron light microscopy dataset with pixel-wise annotations. In addition, we define a set of instance segmentation metrics for benchmarking that we designed to be meaningful with regard to downstream analyses. Lastly, we provide three baselines to kick off a competition that we envision to both advance the field of machine learning regarding methodology for capturing long-range data dependencies, and facilitate scientific discovery in basic neuroscience.
△ Less
Submitted 29 March, 2024;
originally announced April 2024.
-
CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting
Authors:
Simon Graham,
Quoc Dang Vu,
Mostafa Jahanifar,
Martin Weigert,
Uwe Schmidt,
Wenhua Zhang,
Jun Zhang,
Sen Yang,
Jinxi Xiang,
Xiyue Wang,
Josef Lorenz Rumberger,
Elias Baumann,
Peter Hirsch,
Lihao Liu,
Chenyang Hong,
Angelica I. Aviles-Rivero,
Ayushi Jain,
Heeyoung Ahn,
Yiyu Hong,
Hussam Azzuni,
Min Xu,
Mohammad Yaqub,
Marie-Claire Blache,
Benoît Piégu,
Bertrand Vernay
, et al. (64 additional authors not shown)
Abstract:
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of repro…
▽ More
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.
△ Less
Submitted 14 March, 2023; v1 submitted 10 March, 2023;
originally announced March 2023.
-
Nondestructive thermographic detection of internal defects using pixel-pattern based laser excitation and photothermal super resolution reconstruction
Authors:
Julien Lecompagnon,
Philipp Daniel Hirsch,
Christian Rupprecht,
Mathias Ziegler
Abstract:
In this work, we present a novel approach to photothermal super resolution based thermographic resolution of internal defects using two-dimensional pixel pattern-based active photothermal laser heating in conjunction with subsequent numerical reconstruction to achieve a high-resolution reconstruction of internal defect structures. With the proposed adoption of pixelated patterns generated using la…
▽ More
In this work, we present a novel approach to photothermal super resolution based thermographic resolution of internal defects using two-dimensional pixel pattern-based active photothermal laser heating in conjunction with subsequent numerical reconstruction to achieve a high-resolution reconstruction of internal defect structures. With the proposed adoption of pixelated patterns generated using laser coupled high-power DLP projector technology the complexity for achieving true two-dimensional super resolution can be dramatically reduced taking a crucial step forward towards widespread practical viability. Furthermore, based on the latest developments in high-power DLP projectors, we present their first application for structured pulsed thermographic inspection of macroscopic metal samples. In addition, a forward solution to the underlying inverse problem is proposed along with an appropriate heuristic to find the regularization parameters necessary for the numerical inversion in a laboratory setting. This allows the generation of synthetic measurement data, opening the door for the application of machine learning based methods for future improvements towards full automation of the method. Finally, the proposed method is experimentally validated and shown to outperform several established conventional thermographic testing techniques while conservatively improving the required measurement times by a factor of 8 compared to currently available photothermal super resolution techniques.
△ Less
Submitted 3 January, 2023; v1 submitted 8 November, 2022;
originally announced November 2022.
-
Tracking by weakly-supervised learning and graph optimization for whole-embryo C. elegans lineages
Authors:
Peter Hirsch,
Caroline Malin-Mayor,
Anthony Santella,
Stephan Preibisch,
Dagmar Kainmueller,
Jan Funke
Abstract:
Tracking all nuclei of an embryo in noisy and dense fluorescence microscopy data is a challenging task. We build upon a recent method for nuclei tracking that combines weakly-supervised learning from a small set of nuclei center point annotations with an integer linear program (ILP) for optimal cell lineage extraction. Our work specifically addresses the following challenging properties of C. eleg…
▽ More
Tracking all nuclei of an embryo in noisy and dense fluorescence microscopy data is a challenging task. We build upon a recent method for nuclei tracking that combines weakly-supervised learning from a small set of nuclei center point annotations with an integer linear program (ILP) for optimal cell lineage extraction. Our work specifically addresses the following challenging properties of C. elegans embryo recordings: (1) Many cell divisions as compared to benchmark recordings of other organisms, and (2) the presence of polar bodies that are easily mistaken as cell nuclei. To cope with (1), we devise and incorporate a learnt cell division detector. To cope with (2), we employ a learnt polar body detector. We further propose automated ILP weights tuning via a structured SVM, alleviating the need for tedious manual set-up of a respective grid search. Our method outperforms the previous leader of the cell tracking challenge on the Fluo-N3DH-CE embryo dataset. We report a further extensive quantitative evaluation on two more C. elegans datasets. We will make these datasets public to serve as an extended benchmark for future method development. Our results suggest considerable improvements yielded by our method, especially in terms of the correctness of division event detection and the number and length of fully correct track segments. Code: https://github.com/funkelab/linajea
△ Less
Submitted 24 August, 2022;
originally announced August 2022.
-
Panoptic segmentation with highly imbalanced semantic labels
Authors:
Josef Lorenz Rumberger,
Elias Baumann,
Peter Hirsch,
Andrew Janowczyk,
Inti Zlobec,
Dagmar Kainmueller
Abstract:
We describe here the panoptic segmentation method we devised for our participation in the CoNIC: Colon Nuclei Identification and Counting Challenge at ISBI 2022. Key features of our method are a weighted loss specifically engineered for semantic segmentation of highly imbalanced cell types, and a state-of-the art nuclei instance segmentation model, which we combine in a Hovernet-like architecture.
We describe here the panoptic segmentation method we devised for our participation in the CoNIC: Colon Nuclei Identification and Counting Challenge at ISBI 2022. Key features of our method are a weighted loss specifically engineered for semantic segmentation of highly imbalanced cell types, and a state-of-the art nuclei instance segmentation model, which we combine in a Hovernet-like architecture.
△ Less
Submitted 19 April, 2022; v1 submitted 3 March, 2022;
originally announced March 2022.
-
Thermographic detection of internal defects using 2D photothermal super resolution reconstruction with sequential laser heating
Authors:
Julien Lecompagnon,
Samim Ahmadi,
Philipp Hirsch,
Christian Rupprecht,
Mathias Ziegler
Abstract:
Thermographic photothermal super resolution reconstruction enables the resolution of internal defects/inhomogeneities below the classical limit which is governed by the diffusion properties of thermal wave propagation. Based on a combination of the application of special sampling strategies and a subsequent numerical optimization step in post-processing, thermographic super resolution has already…
▽ More
Thermographic photothermal super resolution reconstruction enables the resolution of internal defects/inhomogeneities below the classical limit which is governed by the diffusion properties of thermal wave propagation. Based on a combination of the application of special sampling strategies and a subsequent numerical optimization step in post-processing, thermographic super resolution has already proven to be superior to standard thermographic methods in the detection of one-dimensional defect/inhomogeneity structures. In our work, we report an extension of the capabilities of the method for efficient detection and resolution of defect cross sections with fully two-dimensional structured laser-based heating. The reconstruction is carried out using one of two different algorithms which are proposed within this work. Both algorithms utilize the combination of several coherent measurements using convex optimization and exploit the sparse nature of defects/inhomogeneities as is typical for most nondestructive testing scenarios. Finally, the performance of each algorithm is rated on reconstruction quality and algorithmic complexity. The presented experimental approach is based on repeated spatially structured heating by a high power laser. As a result, a two-dimensional sparse defect/inhomogeneity map can be obtained. In addition, the obtained results are compared with those of conventional thermographic inspection methods which make use of homogeneous illumination. Due to the sparse nature of the reconstructed defect/inhomogeneity map, this comparison is performed qualitatively.
△ Less
Submitted 24 April, 2022; v1 submitted 1 March, 2022;
originally announced March 2022.
-
Common Limitations of Image Processing Metrics: A Picture Story
Authors:
Annika Reinke,
Minu D. Tizabi,
Carole H. Sudre,
Matthias Eisenmann,
Tim Rädsch,
Michael Baumgartner,
Laura Acion,
Michela Antonelli,
Tal Arbel,
Spyridon Bakas,
Peter Bankhead,
Arriel Benis,
Matthew Blaschko,
Florian Buettner,
M. Jorge Cardoso,
Jianxu Chen,
Veronika Cheplygina,
Evangelia Christodoulou,
Beth Cimini,
Gary S. Collins,
Sandy Engelhardt,
Keyvan Farahani,
Luciana Ferrer,
Adrian Galdran,
Bram van Ginneken
, et al. (68 additional authors not shown)
Abstract:
While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using spe…
▽ More
While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using specific metrics for a given image analysis task. These are typically related to (1) the disregard of inherent metric properties, such as the behaviour in the presence of class imbalance or small target structures, (2) the disregard of inherent data set properties, such as the non-independence of the test cases, and (3) the disregard of the actual biomedical domain interest that the metrics should reflect. This living dynamically document has the purpose to illustrate important limitations of performance metrics commonly applied in the field of image analysis. In this context, it focuses on biomedical image analysis problems that can be phrased as image-level classification, semantic segmentation, instance segmentation, or object detection task. The current version is based on a Delphi process on metrics conducted by an international consortium of image analysis experts from more than 60 institutions worldwide.
△ Less
Submitted 6 December, 2023; v1 submitted 12 April, 2021;
originally announced April 2021.
-
How Shift Equivariance Impacts Metric Learning for Instance Segmentation
Authors:
Josef Lorenz Rumberger,
Xiaoyan Yu,
Peter Hirsch,
Melanie Dohmen,
Vanessa Emanuela Guarino,
Ashkan Mokarian,
Lisa Mais,
Jan Funke,
Dagmar Kainmueller
Abstract:
Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly b…
▽ More
Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly been reported in the context of tile-and-stitch approaches, where discontinuities at tile boundaries have been observed. To date, neither of the reported issues have undergone thorough formal analysis. In our work, we contribute a comprehensive formal analysis of the shift equivariance properties of encoder-decoder-style CNNs, which yields a clear picture of what can and cannot be achieved with metric learning in the face of same-looking objects. In particular, we prove that a standard encoder-decoder network that takes $d$-dimensional images as input, with $l$ pooling layers and pooling factor $f$, has the capacity to distinguish at most $f^{dl}$ same-looking objects, and we show that this upper limit can be reached. Furthermore, we show that to avoid discontinuities in a tile-and-stitch approach, assuming standard batch size 1, it is necessary to employ valid convolutions in combination with a training output window size strictly greater than $f^l$, while at test-time it is necessary to crop tiles to size $n\cdot f^l$ before stitching, with $n\geq 1$. We complement these theoretical findings by discussing a number of insightful special cases for which we show empirical results on synthetic data.
△ Less
Submitted 19 August, 2021; v1 submitted 14 January, 2021;
originally announced January 2021.
-
Laser excited super resolution thermal imaging for nondestructive inspection of internal defects
Authors:
Samim Ahmadi,
Julien Lecompagnon,
Philipp Daniel Hirsch,
Peter Burgholzer,
Peter Jung,
Giuseppe Caire,
Mathias Ziegler
Abstract:
A photothermal super resolution technique is proposed for an improved inspection of internal defects. To evaluate the potential of the laser-based thermographic technique, an additively manufactured stainless steel specimen with closely spaced internal cavities is used. Four different experimental configurations in transmission, reflection, stepwise and continuous scanning are investigated. The ap…
▽ More
A photothermal super resolution technique is proposed for an improved inspection of internal defects. To evaluate the potential of the laser-based thermographic technique, an additively manufactured stainless steel specimen with closely spaced internal cavities is used. Four different experimental configurations in transmission, reflection, stepwise and continuous scanning are investigated. The applied image post-processing method is based on compressed sensing and makes use of the block sparsity from multiple measurement events. This concerted approach of experimental measurement strategy and numerical optimization enables the resolution of internal defects and outperforms conventional thermographic inspection techniques.
△ Less
Submitted 2 November, 2020; v1 submitted 7 July, 2020;
originally announced July 2020.
-
An Auxiliary Task for Learning Nuclei Segmentation in 3D Microscopy Images
Authors:
Peter Hirsch,
Dagmar Kainmueller
Abstract:
Segmentation of cell nuclei in microscopy images is a prevalent necessity in cell biology. Especially for three-dimensional datasets, manual segmentation is prohibitively time-consuming, motivating the need for automated methods. Learning-based methods trained on pixel-wise ground-truth segmentations have been shown to yield state-of-the-art results on 2d benchmark image data of nuclei, yet a resp…
▽ More
Segmentation of cell nuclei in microscopy images is a prevalent necessity in cell biology. Especially for three-dimensional datasets, manual segmentation is prohibitively time-consuming, motivating the need for automated methods. Learning-based methods trained on pixel-wise ground-truth segmentations have been shown to yield state-of-the-art results on 2d benchmark image data of nuclei, yet a respective benchmark is missing for 3d image data. In this work, we perform a comparative evaluation of nuclei segmentation algorithms on a database of manually segmented 3d light microscopy volumes. We propose a novel learning strategy that boosts segmentation accuracy by means of a simple auxiliary task, thereby robustly outperforming each of our baselines. Furthermore, we show that one of our baselines, the popular three-label model, when trained with our proposed auxiliary task, outperforms the recent StarDist-3D. As an additional, practical contribution, we benchmark nuclei segmentation against nuclei detection, i.e. the task of merely pinpointing individual nuclei without generating respective pixel-accurate segmentations. For learning nuclei detection, large 3d training datasets of manually annotated nuclei center points are available. However, the impact on detection accuracy caused by training on such sparse ground truth as opposed to dense pixel-wise ground truth has not yet been quantified. To this end, we compare nuclei detection accuracy yielded by training on dense vs. sparse ground truth. Our results suggest that training on sparse ground truth yields competitive nuclei detection rates.
△ Less
Submitted 7 February, 2020;
originally announced February 2020.
-
PatchPerPix for Instance Segmentation
Authors:
Peter Hirsch,
Lisa Mais,
Dagmar Kainmueller
Abstract:
We present a novel method for proposal free instance segmentation that can handle sophisticated object shapes which span large parts of an image and form dense object clusters with crossovers. Our method is based on predicting dense local shape descriptors, which we assemble to form instances. All instances are assembled simultaneously in one go. To our knowledge, our method is the first non-itera…
▽ More
We present a novel method for proposal free instance segmentation that can handle sophisticated object shapes which span large parts of an image and form dense object clusters with crossovers. Our method is based on predicting dense local shape descriptors, which we assemble to form instances. All instances are assembled simultaneously in one go. To our knowledge, our method is the first non-iterative method that yields instances that are composed of learnt shape patches. We evaluate our method on a diverse range of data domains, where it defines the new state of the art on four benchmarks, namely the ISBI 2012 EM segmentation benchmark, the BBBC010 C. elegans dataset, and 2d as well as 3d fluorescence microscopy data of cell nuclei. We show furthermore that our method also applies to 3d light microscopy data of Drosophila neurons, which exhibit extreme cases of complex shape clusters
△ Less
Submitted 8 December, 2022; v1 submitted 21 January, 2020;
originally announced January 2020.
-
Electron diffraction based techniques in scanning electron microscopy of bulk materials
Authors:
AJ Wilkinson,
PB Hirsch
Abstract:
The three scanning electron microscope diffraction based techniques of electron channelling patterns (ECPs), electron channelling contrast imaging (ECCI), and electron back scatter diffraction (EBSD) are reviewed. The dynamical diffraction theory is used to describe the physics of electron channelling, and hence the contrast observed in ECPs (and EBSD) and ECCI images of dislocations. Models for c…
▽ More
The three scanning electron microscope diffraction based techniques of electron channelling patterns (ECPs), electron channelling contrast imaging (ECCI), and electron back scatter diffraction (EBSD) are reviewed. The dynamical diffraction theory is used to describe the physics of electron channelling, and hence the contrast observed in ECPs (and EBSD) and ECCI images of dislocations. Models for calculating channelling contrast are described and their limitations discussed. The practicalities of the experimental methods, including detector-specimen configurations, spatial resolution and sensitivities are given. Examples are given of the use of ECCI for imaging and characterising lattice defects, both individually and in groups, in semiconductor heterostructures and fatigued metals. Applications of the EBSD technique to orientation determination, phase identification and strain measurement are given and compared with use of ECPs.. It is concluded that these techniques make the SEM a powerful instrument for characterising the local crystallography of bulk materials at the mesoscopic scale.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
