Publications
Journal Article
A multi-center polyp detection and segmentation dataset for generalisability assessment
Nature Scientific Data 10 (2023).Status: Published
A multi-center polyp detection and segmentation dataset for generalisability assessment
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | Nature Scientific Data |
Volume | 10 |
Publisher | Nature |
URL | https://doi.org/10.1038/s41597-023-01981-y |
DOI | 10.1038/s41597-023-01981-y |
Journal Article
FANet: A Feedback Attention Network for Improved Biomedical Image Segmentation
IEEE Transactions on Neural Networks and Learning Systems (2022): 1-14.Status: Published
FANet: A Feedback Attention Network for Improved Biomedical Image Segmentation
The increase of available large clinical and experimental datasets has contributed to a substantial amount of important contributions in the area of biomedical image analysis. Image segmentation, which is crucial for any quantitative analysis, has especially attracted attention. Recent hardware advancement has led to the success of deep learning approaches. However, although deep learning models are being trained on large datasets, existing methods do not use the information from different learning epochs effectively. In this work, we leverage the information of each training epoch to prune the prediction maps of the subsequent epochs. We propose a novel architecture called feedback attention network (FANet) that unifies the previous epoch mask with the feature map of the current training epoch. The previous epoch mask is then used to provide hard attention to the learned feature maps at different convolutional layers. The network also allows rectifying the predictions in an iterative fashion during the test time. We show that our proposed feedback attention model provides a substantial improvement on most segmentation metrics tested on seven publicly available biomedical imaging datasets demonstrating the effectiveness of FANet. The source code is available at https://github.com/nikhilroxtomar/FANet.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | IEEE Transactions on Neural Networks and Learning Systems |
Pagination | 1 - 14 |
Date Published | Jan-01-2022 |
Publisher | IEEE |
ISSN | 2162-237X |
URL | https://ieeexplore.ieee.org/document/9741842 |
DOI | 10.1109/TNNLS.2022.3159394 |
Meta-learning with implicit gradients in a few-shot setting for medical image segmentation
Computers in Biology and Medicine 143 (2022): 105227.Status: Published
Meta-learning with implicit gradients in a few-shot setting for medical image segmentation
Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are labour-intensive and expensive, but can also be used to model on a dataset coming from a new population. To this end, we propose to exploit an optimization-based implicit model agnostic meta-learning (iMAML) algorithm under few-shot settings for medical image segmentation. Our approach can leverage the learned weights from diverse but small training samples to perform analysis on unseen datasets with high accuracy. We show that, unlike classical few-shot learning approaches, our method improves generalization capability. To our knowledge, this is the first work that exploits iMAML for medical image segmentation and explores the strength of the model on scenarios such as meta-training on unique and mixed instances of lesion datasets. Our quantitative results on publicly available skin and polyp datasets show that the proposed method outperforms the naive supervised baseline model and two recent few-shot segmentation approaches by large margins. In addition, our iMAML approach shows an improvement of 2%–4% in dice score compared to its counterpart MAML for most experiments.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Computers in Biology and Medicine |
Volume | 143 |
Pagination | 105227 |
Date Published | Jan-04-2022 |
Publisher | Elsevier |
ISSN | 00104825 |
URL | https://linkinghub.elsevier.com/retrieve/pii/S0010482522000191 |
DOI | 10.1016/j.compbiomed.2022.105227 |
MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation
IEEE Journal of Biomedical and Health Informatics 26, no. 5 (2022): 2252-2263.Status: Published
MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation
Methods based on convolutional neural networks have improved the performance of biomedical image segmentation. However, most of these methods cannot efficiently segment objects of variable sizes and train on small and biased datasets, which are common for biomedical use cases. While methods exist that incorporate multi-scale fusion approaches to address the challenges arising with variable sizes, they usually use complex models that are more suitable for general semantic segmentation problems. In this paper, we propose a novel architecture called Multi-Scale Residual Fusion Network (MSRF-Net), which is specially designed for medical image segmentation. The proposed MSRF-Net is able to exchange multi-scale features of varying receptive fields using a Dual-Scale Dense Fusion (DSDF) block. Our DSDF block can exchange information rigorously across two different resolution scales, and our MSRF sub-network uses multiple DSDF blocks in sequence to perform multi-scale fusion. This allows the preservation of resolution, improved information flow and propagation of both high- and low-level features to obtain accurate segmentation maps. The proposed MSRF-Net allows to capture object variabilities and provides improved results on different biomedical datasets. Extensive experiments on MSRF-Net demonstrate that the proposed method outperforms the cutting edge medical image segmentation methods on four publicly available datasets. We achieve the Dice Coefficient (DSC) of 0.9217, 0.9420, and 0.9224, 0.8824 on Kvasir-SEG, CVC-ClinicDB, 2018 Data Science Bowl dataset, and ISIC-2018 skin lesion segmentation challenge dataset respectively. We further conducted generalizability tests that also achieved the highest DSC score with 0.7921 and 0.7575 on CVC-ClinicDB and Kvasir-SEG, respectively.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | IEEE Journal of Biomedical and Health Informatics |
Volume | 26 |
Issue | 5 |
Pagination | 2252-2263 |
Date Published | 12/2021 |
Publisher | IEEE |
ISSN | 2168-2194 |
URL | https://ieeexplore.ieee.org/document/9662196 |
DOI | 10.1109/JBHI.2021.3138024 |
Proceedings, refereed
GMSRF-Net: An Improved generalizability with Global Multi-Scale Residual Fusion Network for Polyp Segmentation
In 26th International Conference on Pattern Recognition. IEEE, 2022.Status: Published
GMSRF-Net: An Improved generalizability with Global Multi-Scale Residual Fusion Network for Polyp Segmentation
Colonoscopy is a gold standard procedure but is highly operator-dependent. Efforts have been made to automate the detection and segmentation of polyps, a precancerous precursor, to effectively minimize missed rate. Widely used computer-aided polyp segmentation systems actuated by encoder-decoder have achieved high performance in terms of accuracy. However, polyp segmentation datasets collected from varied centers can follow different imaging protocols leading to difference in data distribution. As a result, most methods suffer from performance drop when trained and tested on different distributions and therefore, require re-training for each specific dataset. We address this generalizability issue by proposing a global multi-scale residual fusion network (GMSRF-Net). Our proposed network maintains high-resolution representations by performing multi-scale fusion operations across all resolution scales through dense connections while preserving low-level information. To further leverage scale information, we design cross multi-scale attention (CMSA) module that uses multi-scale features to identify, keep, and propagate informative features. Additionally, we introduce multi-scale feature selection (MSFS) modules to perform channel-wise attention that gates irrelevant features gathered through global multi-scale fusion within the GMSRF-Net. The repeated fusion operations gated by CMSA and MSFS demonstrate improved generalizability of our network. Experiments conducted on two different polyp segmentation datasets show that our proposed GMSRF-Net outperforms the previous top-performing state-of-the-art method by 8.34% and 10.31% on unseen CVC-ClinicDB and on unseen Kvasir-SEG, in terms of dice coefficient. Additionally, when tested on unseen CVC-ColonDB, we surpass the state-of-the-art method by 9.38% and 4.04% in terms of dice coefficient, when source dataset is Kvasir-SEG and CVC-ClinicDB, respectively.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 26th International Conference on Pattern Recognition |
Publisher | IEEE |
DOI | 10.1109/ICPR56361.2022.9956726 |
Video Analytics in Elite Soccer: A Distributed Computing Perspective
In IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM). Trondheim, Norway: IEEE, 2022.Status: Published
Video Analytics in Elite Soccer: A Distributed Computing Perspective
Ubiquitous sensors and Internet of Things (IoT) technologies have revolutionized the sports industry, providing new methodologies for planning, effective coordination of training, and match analysis post game. New methods, including machine learning, image and video processing, have been developed for performance evaluation, allowing the analyst to track the performance of a player in real-time. Following FIFA's 2015 approval of electronics performance and tracking system during games, performance data of a single player or the entire team is allowed to be collected using GPS-based wearables. Data from practice sessions outside the sporting arena is being collected in greater numbers than ever before. Realizing the significance of data in professional soccer, this paper presents video analytics, examines recent state-of-the-art literature in elite soccer, and summarizes existing real-time video analytics algorithms. We also discuss real-time crowdsourcing of the obtained data, tactical and technical performance, distributed computing and its importance in video analytics and propose a future research perspective.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM) |
Pagination | 221-225 |
Date Published | 06/2022 |
Publisher | IEEE |
Place Published | Trondheim, Norway |
Keywords | analytics, football, soccer, Video |
URL | https://ieeexplore.ieee.org/document/9827827 |
DOI | 10.1109/SAM53842.2022.9827827 |
PhD Thesis
Machine Learning-based Classification, Detection, and Segmentation of Medical Images
In UiT The Arctic University of Norway. Vol. PhD, 2022.Status: Published
Machine Learning-based Classification, Detection, and Segmentation of Medical Images
Gastointestinal (GI) cancers are among the most common types of cancers worldwide. In particular, CRC is the most lethal in terms of number of incidences and mortality (third most common cause of cancer and the second common cause of cancer-related deaths). Colonoscopy is the gold standard for screening patients for CRC. During the colonoscopy, gastroenterologists examine the large bowel, detect precancerous abnormal tissue growths like polyps and remove them through the scope if necessary. Although colonoscopy is considered the gold standard, it is an operator-dependent procedure. Previous research has shown large missing rates for GI abnormalities, e.g., polyp miss detection is around 22%-28%. Early detection of GI lesions and cancers at the curable stage can help reduce the mortality rate. The development of automated, accurate, and efficient methods for the detection of the GI cancers could benefit both gastroenterologists and patients. In addition, if integrated into screening programs, an automatic analysis could improve overall GI endoscopy quality.
The medical field is becoming more interdisciplinary, and the importance of medical image data is increasing rapidly. Medical image analysis can play a central role in disease detection, diagnosis, and treatment. With the increasing number of medical images, there is enormous potential to improve the screening quality. Deep learning (DL), in particular, CNN based models have tremendous potential to automate and enhance the medical image analysis procedure and provide an accurate diagnosis. The automated analysis of the medical images could reduce the burden of the medical experts and provide quality and accessible healthcare to a larger population. In medical imaging, classification, detection, and semantic segmentation tasks are crucial for clinical practice. The development of accurate and efficient CADx or CADe models can help to identify the abnormalities at an early stage and can act as a third eye for the doctors.
To this end, we have studied and designed ml and dl based architectures for gi tract disease classification, detection, and segmentation. Our designed architectures can classify different types of \gls{gi} tract findings and abnormalities accurately with high performance. Our contribution towards the development of cade models for automated polyp detection showed improved performance. Out of three different medical imaging tasks, semantic segmentation of medical imaging data plays a significant role in extracting meaningful information from images by classifying each pixel and segmenting it by class. Using the GI case scenario, we have mainly worked on polyp segmentation and proposed and evaluated different automated polyp segmentation architectures. We have also built architectures for surgical instrument segmentation that showed high performance and real-time speed.
We have collected, annotated, and released several open-access datasets such as HyperKvasir, KvasirCapsule, PolypGen, Kvasir-SEG, Kvasir-instrument, and KvasirCapsule-SEG in collaboration with hospitals in Norway and abroad to address the lack of datasets in the field. We have devised several medical image segmentation architectures (for example, ResUNet++, DoubleU-Net, and ResUNet + CRF + TTA) that provided improved results with the publicly available datasets. Beside that, we have also designed architectures that have the capability of segmenting polyps in real-time with high frame per second (for example, ColonSegNet, NanoNet, PNS-Net, and DDANet). Moreover, we performed extensive studies on the generalizability of our models on public datasets, and by creating a dataset consisting of data from different hospitals, we allow multi-center cross dataset testing. Our results prove that proposed dl based CADx systems might be of great assistance to clinicians in the future.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | PhD Thesis |
Year of Publication | 2022 |
Degree awarding institution | UiT The Arctic University of Norway |
Degree | PhD |
Number of Pages | 400 |
Date Published | 01/2022 |
Thesis Type | Paper-based PhD thesis |
URL | https://munin.uit.no/handle/10037/23693 |
Journal Article
A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging
Medical Image Analysis 70 (2021): 102007.Status: Published
A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging
Gastrointestinal endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Medical Image Analysis |
Volume | 70 |
Pagination | 102007 |
Publisher | Elsevier |
Keywords | Artificial intelligence, BioMedia 2019 Grand Challenge, Computer-aided detection and diagnosis, Gastrointestinal endoscopy challenges, Medical imaging, Medico Task 2017, Medico Task 2018 |
DOI | 10.1016/j.media.2021.102007 |
A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation
IEEE Journal of Biomedical and Health Informatics 25, no. 6 (2021): 2029-2040.Status: Published
A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation
Colonoscopy is considered the gold standard for detection of colorectal cancer and its precursors. Existing examination methods are, however, hampered by high overall miss-rate, and many abnormalities are left undetected. Computer-Aided Diagnosis systems based on advanced machine learning algorithms are touted as a game-changer that can identify regions in the colon overlooked by the physicians during endoscopic examinations, and help detect and characterize lesions. In previous work, we have proposed the ResUNet++ architecture and demonstrated that it produces more efficient results compared with its counterparts U-Net and ResUNet. In this paper, we demonstrate that further improvements to the overall prediction performance of the ResUNet++ architecture can be achieved by using Conditional Random Field (CRF) and Test-Time Augmentation (TTA). We have performed extensive evaluations and validated the improvements using six publicly available datasets: Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS-Larib PolypDB, ASU-Mayo Clinic Colonoscopy Video Database, and CVC-VideoClinicDB. Moreover, we compare our proposed architecture and resulting model with other State-of-the-art methods. To explore the generalization capability of ResUNet++ on different publicly available polyp datasets, so that it could be used in a real-world setting, we performed an extensive cross-dataset evaluation. The experimental results show that applying CRF andTTA improves the performance on various polyp segmentation datasets both on the same dataset and cross-dataset. To check the model’s performance on difficult to detect polyps, we selected, with the help of an expert gastroenterologist,196sessile or flat polyps that are less than ten millimeters in size. This additional data has been made available as a subset of Kvasir-SEG. Our approaches showed good results for flat or sessile and smaller polyps, which are known to be one of the major reasons for high polyp miss-rates. This is one of the significant strengths of our work and indicates that our methods should be investigated further for use in clinical practice.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | IEEE Journal of Biomedical and Health Informatics |
Volume | 25 |
Issue | 6 |
Pagination | 2029 - 2040 |
Publisher | IEEE |
Keywords | colonoscopy, conditional random field, generalization, Polyp segmentation, ResUNet++, test-time augmentation |
DOI | 10.1109/JBHI.2021.3049304 |
Improving Polyp Segmentation in Colonoscopy using Deep Learning
Nordic Machine Intelligence 1, no. 1 (2021): 35-37.Status: Published
Improving Polyp Segmentation in Colonoscopy using Deep Learning
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Nordic Machine Intelligence |
Volume | 1 |
Issue | 1 |
Pagination | 35 - 37 |
Date Published | Jan-11-2021 |
Publisher | Department of Informatics, University of Oslo |
URL | https://journals.uio.no/NMI/article/view/9136 |
DOI | 10.5617/nmi.9136 |
Improving Polyp Segmentation in Colonoscopy using Deep Learning
Nordic Machine Intelligence 1, no. 1 (2021): 35-37.Status: Published
Improving Polyp Segmentation in Colonoscopy using Deep Learning
Colorectal cancer is one of the major causes of cancer-related deaths globally. Although colonoscopy is considered as the gold standard for examination of colon polyps, there is a significant miss rate of around 22-28%. Deep learning algorithms such as convolutional neural networks can aid in the detection and describe abnormalities in the colon that clinicians might miss during endoscopic examinations. The" MedAI: Transparency in Medical Image Segmentation" competition provides an opportunity to develop accurate and automated polyp segmentation algorithms on the same dataset provided by the challenge organizer. We participate in the polyp segmentation task of the challenge and provide a solution based on the dual decoder attention network (DDANet). The DDANet is an encoder-decoder-based architecture based on a dual decoder attention network. Our experimental results on the organizers' dataset showed a dice coefficient of 0.7967, Jaccard index of 0.7220, a recall of 0.8214, a precision of 0.8359, and an accuracy of 0.9557. Our results on unseen datasets suggest that deep learning and computer vision-based methods can effectively solve automated polyp segmentation tasks.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Nordic Machine Intelligence |
Volume | 1 |
Issue | 1 |
Pagination | 35-37 |
Date Published | 11/2021 |
Publisher | Nordic Machine Intelligence |
Kvasir-Capsule, a video capsule endoscopy dataset
Scientific Data 8, no. 1 (2021): 142.Status: Published
Kvasir-Capsule, a video capsule endoscopy dataset
Artificial intelligence (AI) is predicted to have profound effects on the future of video capsule endoscopy (VCE) technology. The potential lies in improving anomaly detection while reducing manual labour. Existing work demonstrates the promising benefits of AI-based computer-assisted diagnosis systems for VCE. They also show great potential for improvements to achieve even better results. Also, medical data is often sparse and unavailable to the research community, and qualified medical personnel rarely have time for the tedious labelling work. We present Kvasir-Capsule, a large VCE dataset collected from examinations at a Norwegian Hospital. Kvasir-Capsule consists of 117 videos which can be used to extract a total of 4,741,504 image frames. We have labelled and medically verified 47,238 frames with a bounding box around findings from 14 different classes. In addition to these labelled images, there are 4,694,266 unlabelled frames included in the dataset. The Kvasir-Capsule dataset can play a valuable role in developing better algorithms in order to reach true potential of VCE technology.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Scientific Data |
Volume | 8 |
Issue | 1 |
Pagination | 142 |
Publisher | Springer Nature |
URL | http://www.nature.com/articles/s41597-021-00920-z |
DOI | 10.1038/s41597-021-00920-z |
MedAI: Transparency in Medical Image Segmentation
Nordic Machine Intelligence 1, no. 1 (2021): 1-4.Status: Published
MedAI: Transparency in Medical Image Segmentation
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Nordic Machine Intelligence |
Volume | 1 |
Issue | 1 |
Pagination | 1 - 4 |
Date Published | Jan-11-2021 |
Publisher | NMI |
Place Published | Oslo |
URL | https://journals.uio.no/NMI/article/view/9140https://journals.uio.no/NMI... |
DOI | 10.5617/nmi.9140 |
Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning
IEEE Access 9 (2021): 40496-40510.Status: Published
Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning
Computer-aided detection, localization, and segmentation methods can help improve colonoscopy procedures. Even though many methods have been built to tackle automatic detection and segmentation of polyps, benchmarking of state-of-the-art methods still remains an open problem. This is due to the increasing number of researched computer vision methods that can be applied to polyp datasets. Benchmarking of novel methods can provide a direction to the development of automated polyp detection and segmentation tasks. Furthermore, it ensures that the produced results in the community are reproducible and provide a fair comparison of developed methods. In this paper, we benchmark several recent state-of-the-art methods using Kvasir-SEG, an open-access dataset of colonoscopy images for polyp detection, localization, and segmentation evaluating both method accuracy and speed. Whilst, most methods in literature have competitive performance over accuracy, we show that the proposed ColonSegNet achieved a better trade-off between an average precision of 0.8000 and mean IoU of 0.8100, and the fastest speed of 180 frames per second for the detection and localization task. Likewise, the proposed ColonSegNetachieved a competitive dice coefficient of 0.8206 and the best average speed of 182.38 frames per second for the segmentation task. Our comprehensive comparison with various state-of-the-art methods reveals the importance of benchmarking the deep learning methods for automated real-time polyp identification and delineations that can potentially transform current clinical practices and minimize miss-detection rates.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | IEEE Access |
Volume | 9 |
Pagination | 40496-40510 |
Publisher | IEEE |
Keywords | Medical image segmentation, ColonSegNet, colonoscopy, polyps, deep learning, detection, localization, benchmarking, Kvasir-SEG |
DOI | 10.1109/ACCESS.2021.3063716 |
Book Chapter
Artificial Intelligence in Gastroenterology
In Artificial Intelligence in Medicine, 1-20. Cham: Springer International Publishing, 2021.Status: Published
Artificial Intelligence in Gastroenterology
The holy grail in endoscopy examinations has for a long time been assisted diagnosis using Artificial Intelligence (AI). Recent developments in computer hardware are now enabling technology to equip clinicians with promising tools for computer-assisted diagnosis (CAD) systems. However, creating viable models or architectures, training them, and assessing their ability to diagnose at a human level, are complicated tasks. This is currently an active area of research, and many promising methods have been proposed. In this chapter, we give an overview of the topic. This includes a description of current medical challenges followed by a description of the most commonly used methods in the field. We also present example results from research targeting some of these challenges, and a discussion on open issues and ongoing work is provided. Hopefully, this will inspire and enable readers to future develop CAD systems for gastroenterology.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Book Chapter |
Year of Publication | 2021 |
Book Title | Artificial Intelligence in Medicine |
Pagination | 1 - 20 |
Date Published | 09/2021 |
Publisher | Springer International Publishing |
Place Published | Cham |
ISBN Number | 978-3-030-58080-3 |
Keywords | Anomaly detection, artificial intelligence, Gastrointestinal endoscopy, Hand-crafted features, Neural Networks, Performance, Semantic segmentation |
URL | https://link.springer.com/referenceworkentry/10.1007%2F978-3-030-58080-3... |
DOI | 10.1007/978-3-030-58080-3_163-2 |
Proceedings, non-refereed
Automated Polyp Segmentation in Colonoscopy using MSRFNet
In MediaEval medico . CEUR Workshop Proceedings, 2021.Status: Published
Automated Polyp Segmentation in Colonoscopy using MSRFNet
Colorectal cancer is one of the major cause of cancer-related death around the world. High-quality colonoscopy is considered mandatory for resecting and preventing colorectal cancers. In the recent past, various technological advances have been made towards improving the quality of colonoscopy. Despite the technical advancement, some polyps are frequently missed during colonoscopy examinations. Polyp detection ( for example, adenomas) rates are largely influenced by inter-endoscopist variability. Therefore, it is very challenging to standardize a high-quality colonoscopy. A computer-aided detection system could solve the problem with miss-detection. The ``MediaEval 2021'' challenge entails the chance to study and develop accurate automated polyp segmentation algorithms \cite{Hicks2021Medico}. In this paper, we propose our approach based on MSRFNet. Our experimental findings show that the model trained on the Kvasir-SEG dataset and evaluated on a competition test dataset obtains a dice coefficient of 0.7055, Jaccard of 0.6176, a recall of 0.7293, and a precision of 0.7769. In addition to the MediaEval 2021 challenge, we evaluated our approach on the Endotect Challenge Dataset and ``2020 Medico Automatic Polyp Segmentation Challenge Dataset". The results further demonstrate the efficiency of our approach.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2021 |
Conference Name | MediaEval medico |
Publisher | CEUR Workshop Proceedings |
Improving generalizibilty in polyp segmentation using ensemble convolutional neural network
In 3rd International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV2021). Vol. 2886. CEUR Workshop Proceedings, 2021.Status: Published
Improving generalizibilty in polyp segmentation using ensemble convolutional neural network
Medical image segmentation is a crucial task in medical image analysis. Despite near expert-label performance with the application of the deep learning method in medical image segmentation, the generalization of such models in the clinical environment remains a significant challenge. Transfer learning from a large medical dataset from the same domain is a common technique to address generalizability. However, it is difficult to find a similar large medical dataset. To address generalizability in polyp segmentation, we have used an ensemble of four MultiResUNet architectures, each trained on the combination of the different centered datasets provided by the challenge organizers. Our method achieved a decent performance of 0.6172 ± 0.0778 for the multi-centered dataset. Our study shows that significant work needs to be done to develop a computer-aided diagnosis system to detect and localize polyp of the multi-center datasets, which is essential for improving the quality of the colonoscopy.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2021 |
Conference Name | 3rd International Workshop and Challenge on Computer Vision in Endoscopy (EndoCV2021) |
Volume | 2886 |
Publisher | CEUR Workshop Proceedings |
Keywords | colonoscopy, Convolutional neural network, health informatics, Polyp segmentation |
Proceedings, refereed
Automatic Polyp Segmentation using U-Net-ResNet50
In Medico MediaEval 2020. MediaEval, 2021.Status: Published
Automatic Polyp Segmentation using U-Net-ResNet50
Polyps are the predecessors to colorectal cancer which is considered as one of the leading causes of cancer-related deaths worldwide. Colonoscopy is the standard procedure for the identification, localization, and removal of colorectal polyps. Due to variability in shape, size, and surrounding tissue similarity, colorectal polyps are often missed by the clinicians during colonoscopy. With the use of an automatic, accurate, and fast polyp segmentation method during the colonoscopy, many colorectal polyps can be easily detected and removed. The ``Medico automatic polyp segmentation challenge'' provides an opportunity to study polyp segmentation and build an efficient and accurate segmentation algorithm. We use the U-Net with pre-trained ResNet50 as the encoder for the polyp segmentation. The model is trained on Kvasir-SEG dataset provided for the challenge and tested on the organizer's dataset and achieves a dice coefficient of 0.8154, Jaccard of 0.7396, recall of 0.8533, precision of 0.8532, accuracy of 0.9506, and F2 score of 0.8272, demonstrating the generalization ability of our model.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Medico MediaEval 2020 |
Publisher | MediaEval |
DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation
In 25th International Conference on Pattern Recognition. Springer, 2021.Status: Published
DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation
Colonoscopy is the gold standard for examination and detection of colorectal polyps. Localization and delineation of polyps can play a vital role in treatment (e.g., surgical planning) and prognostic decision making. Polyp segmentation can provide detailed boundary information for clinical analysis. Convolutional neural networks have improved the performance in colonoscopy. However, polyps usually possess various challenges, such as intra-and inter-class variation and noise. While manual labeling for polyp assessment requires time from experts and is prone to human error (e.g., missed lesions), an automated, accurate, and fast segmentation can improve the quality of delineated lesion boundaries and reduce missed rate. The Endotect challenge provides an opportunity to benchmark computer vision methods by training on the publicly available Hyperkvasir and testing on a separate unseen dataset. In this paper, we propose a novel architecture called ``DDANet'' based on a dual decoder attention network. Our experiments demonstrate that the model trained on the Kvasir-SEG dataset and tested on an unseen dataset achieves a dice coefficient of 0.7874, mIoU of 0.7010, recall of 0.7987, and a precision of 0.8577, demonstrating the generalization ability of our model.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 25th International Conference on Pattern Recognition |
Pagination | 307-314 |
Publisher | Springer |
Keywords | Benchmarking, Convolutional neural network, deep learning, Polyp segmentation |
Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy
In 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE, 2021.Status: Published
Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy
Minimally Invasive Surgery (MIS) is a surgical intervention used to examine the organs inside the abdomen and has been widely used due to its effectiveness over open surgery. Due to the hardware improvements such as high definition cameras, this procedure has significantly improved and new software methods have open potential for computer-assisted procedures. However, there exists challenges and requirement to improve detection and tracking of the position of the instruments during these surgical procedures. To this end, we evaluate and compare some popular deep learning methods that can potentially be explored for the automated segmentation of surgical instruments in laparoscopy, an important step towards tool tracking. Our experimental results demonstrate that the Dual decoder attention network (DDANet) produces a superior result compared to other recent deep learning methods. DDANet produces a dice coefficient of 0.8739 and mean intersection over union of 0.8183 for the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge 2019 dataset, at a real-time speed of 101.36 frames per second which is critical for such procedures.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) |
Pagination | 1-4 |
Publisher | IEEE |
Keywords | Real-time segmentation, minimally invasive surgery, surgical instruments, laparoscopy, deep learning |
HTAD: A Home-Tasks Activities Dataset with Wrist-accelerometer and Audio Features
In 27th International Conference on Multimedia Modeling. Springer, 2021.Status: Published
HTAD: A Home-Tasks Activities Dataset with Wrist-accelerometer and Audio Features
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 27th International Conference on Multimedia Modeling |
Pagination | 196-205 |
Publisher | Springer |
Keywords | Accelerometer, Activity recognition, Audio, dataset, Sensor fusion |
Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
In 27th International Conference on Multimedia Modeling. Vol. LNCS, volume 12573. Springer, 2021.Status: Published
Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
Gastrointestinal (GI) pathologies are periodically screened, biopsied, and resected using surgical tools. Usually the procedures and the treated or resected areas are not specifically tracked or analysed during or after colonoscopies. Information regarding disease borders, development and amount and size of the resected area get lost. This can lead to poor follow-up and bothersome reassessment difficulties post-treatment. To improve the current standard and also to foster more research on the topic we have released the ``Kvasir Instrument'' dataset which consists of $590$ annotated frames containing GI procedure tools such as snares, balloons and biopsy forceps, etc. Beside of the images, the dataset includes ground truth masks and bounding boxes and has been verified by two expert GI endoscopists. Additionally, we provide a baseline for the segmentation of the GI tools to promote research and algorithm development. We obtained a dice coefficient score of 0.9158 and a Jaccard index of 0.8578 using a classical U-Net architecture. A similar dice coefficient score was observed for DoubleUNet. The qualitative results showed that the model did not work for the images with specularity and the frames with multiple instruments, while the best result for both methods was observed on all other types of images. Both, qualitative and quantitative results show that the model performs reasonably good, but there is a large potential for further improvements. Benchmarking using the dataset provides an opportunity for researchers to contribute to the field of automatic endoscopic diagnostic and therapeutic tool segmentation for GI endoscopy.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 27th International Conference on Multimedia Modeling |
Volume | LNCS, volume 12573 |
Pagination | 218-229 |
Publisher | Springer |
Medico Multimedia Task at MediaEval 2021: Transparency in Medical Image Segmentation
In Mediaeval Medico 2021. Mediaeval 2021, 2021.Status: Published
Medico Multimedia Task at MediaEval 2021: Transparency in Medical Image Segmentation
The Medico Multimedia Task focuses on providing multimedia researchers with the opportunity to contribute to different areas of medicine using multimedia data to solve several subtasks. This year, the task focuses on transparency within machine learning-based medical segmentation systems, where the use case is gastrointestinal endoscopy. In this paper, we motivate the organization of this task, describe the development and test dataset, and present the evaluation process used to assess the participants' submissions.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Mediaeval Medico 2021 |
Publisher | Mediaeval 2021 |
URL | https://2021.multimediaeval.com/ |
NanoNet: Real-Time Polyp Segmentation in Video Capsule Endoscopy and Colonoscopy
In 34th IEEE CBMS International Symposium on Computer-Based Medical Systems. IEEE, 2021.Status: Published
NanoNet: Real-Time Polyp Segmentation in Video Capsule Endoscopy and Colonoscopy
Deep learning in gastrointestinal endoscopy can assist to improve clinical performance and be helpful to assess lesions more accurately. To this extent, semantic segmentation methods that can perform automated real-time delineation of a region-of-interest, e.g., boundary identification of cancer or precancerous lesions, can benefit both diagnosis and interventions. However, accurate and real-time segmentation of endoscopic images is extremely challenging due to its high operator dependence and high-definition image quality. To utilize automated methods in clinical settings, it is crucial to design lightweight models with low latency such that they can be integrated with low-end endoscope hardware devices. In this work, we propose NanoNet, a novel architecture for the segmentation of video capsule endoscopy and colonoscopy images. Our proposed architecture allows real-time performance and has higher segmentation accuracy compared to other more complex ones. We use video capsule endoscopy and standard colonoscopy datasets with polyps, and a dataset consisting of endoscopy biopsies and surgical instruments, to evaluate the effectiveness of our approach. Our experiments demonstrate the increased performance of our architecture in terms of a trade-off between model complexity, speed, model parameters, and metric performances. Moreover, the resulting model size is relatively tiny, with only nearly 36,000 parameters compared to traditional deep learning approaches having millions of parameters.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 34th IEEE CBMS International Symposium on Computer-Based Medical Systems |
Publisher | IEEE |
Keywords | colonoscopy, deep learning, segmentation, tool segmentation, Video capsule endoscopy |
PAANet: Progressive Alternating Attention for Automatic Medical Image Segmentation
In Proc. of 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART). Paris / Créteil, France: IEEE, 2021.Status: Published
PAANet: Progressive Alternating Attention for Automatic Medical Image Segmentation
Medical image segmentation can provide detailed information for clinical analysis which can be useful for scenarios where the detailed location of a finding is important. Knowing the location of a disease can play a vital role in treatment and decision-making. Convolutional neural network (CNN) based encoder-decoder techniques have advanced the performance of automated medical image segmentation systems. Several such CNN-based methodologies utilize techniques such as spatial- and channel-wise attention to enhance performance. Another technique that has drawn attention in recent years is residual dense blocks (RDBs). The successive convolutional layers in densely connected blocks are capable of extracting diverse features with varied receptive fields and thus, enhancing performance. However, consecutive stacked convolutional operators may not necessarily generate features that facilitate the identification of the target structures. In this paper, we propose a progressive alternating attention network (PAANet). We develop progressive alternating attention dense (PAAD) blocks, which construct a guiding attention map (GAM) after every convolutional layer in the dense blocks using features from all scales. The GAM allows the following layers in the dense blocks to focus on the spatial locations relevant to the target region. Every alternatePAAD block inverts the GAM to generate a reverse attention map which guides ensuing layers to extract boundary and edge-related information, refining the segmentation process. Our experiments on three different biomedical image segmentation datasets exhibit that our PAANet achieves favorable performance when compared to other state-of-the-art methods.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proc. of 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART) |
Date Published | 12/2021 |
Publisher | IEEE |
Place Published | Paris / Créteil, France |
URL | https://ieeexplore.ieee.org/document/9677844 |
DOI | 10.1109/BioSMART54244.2021.9677844 |
Progressively Normalized Self-Attention Network for Video Polyp Segmentation
In Medical Image Computing and Computer Assisted Intervention (MICCAI 2021). Vol. LNCS, volume 12901. Springer, 2021.Status: Published
Progressively Normalized Self-Attention Network for Video Polyp Segmentation
Existing video polyp segmentation (VPS) models typically employ convolutional neural networks (CNNs) to extract features. However, due to their limited receptive fields, CNNs can not fully exploit the global temporal and spatial information in successive video frames, resulting in false positive segmentation results. In this paper, we propose the novel PNS-Net(Progressively Normalized Self-attention Net-work), which can efficiently learn representations from polyp videos with real-time speed (∼140fps) on a single RTX 2080 GPU and no post-processing. OurPNS-Netis based solely on a basic normalized self-attention block, equipping with recurrence and CNNs entirely. Experiments on challenging VPS datasets demonstrate that the proposed PMS-Netachieves state-of-the-art performance. We also conduct extensive experiments to study the effectiveness of the channel split, soft-attention, and progressive learning strategy. We find that our PNS-Networks well under different settings, making it a promising solution to the VPS task.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Medical Image Computing and Computer Assisted Intervention (MICCAI 2021) |
Volume | LNCS, volume 12901 |
Pagination | 142-152 |
Publisher | Springer |
Keywords | colonoscopy, Normalized self-attention, Polyp segmentation |
Journal Article
An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification
ACM Transactions on Computing for Healthcare 1 (2020): 1-29.Status: Published
An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification
Precise and efficient automated identification of Gastrointestinal (GI) tract diseases can help doctors treat more patients and improve the rate of disease detection and identification. Currently, automatic analysis of diseases in the GI tract is a hot topic in both computer science and medical-related journals. Nevertheless, the evaluation of such an automatic analysis is often incomplete or simply wrong. Algorithms are often only tested on small and biased datasets, and cross-dataset evaluations are rarely performed. A clear understanding of evaluation metrics and machine learning models with cross datasets is crucial to bring research in the field to a new quality level. Towards this goal, we present comprehensive evaluations of five distinct machine learning models using Global Features and Deep Neural Networks that can classify 16 different key types of GI tract conditions, including pathological findings, anatomical landmarks, polyp removal conditions, and normal findings from images captured by common GI tract examination instruments. Inour evaluation, we introduce performance hexagons using six performance metrics such as recall, precision, specificity, accuracy, F1-score, and Matthews Correlation Coefficient to demonstrate how to determine the real capabilities of models rather than evaluatingthem shallowly. Furthermore, we perform cross-dataset evaluations using different datasets for training and testing. With these cross-dataset evaluations, we demonstrate the challenge of actually building a generalizable model that could be used across different hospitals. Our experiments clearly show that more sophisticated performance metrics and evaluation methods need to be applied to get reliable models rather than depending on evaluations of the splits of the same dataset, i.e., the performance metrics should always be interpreted together rather than relying on a single metric.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | ACM Transactions on Computing for Healthcare |
Volume | 1 |
Number | 3 |
Pagination | 1-29 |
Publisher | ACM |
Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge
Medical Image Analysis (2020).Status: Published
Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge
Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. While numerous methods for detecting, segmenting and tracking of medical instruments based on endoscopic video im-ages have been proposed in the literature, key limitations remain to be addressed: Firstly, robustness, that is, the reliable performance of state-of-the-art methods when run on challenging images (e.g. in the presence of blood, smoke or motion artifacts). Secondly, generalization; algorithms trained for a specific intervention in a specific hospital should generalize to other interventions or institutions. In an effort to promote solutions for these limitations, we organized the Robust Medical Instrument Segmentation (ROBUST-MIS) challenge as an international benchmarking competition with a specific focus on the robustness and generalization capabilities of algorithms. For the first time in the field of endoscopic image processing, our challenge included a task on binary segmentation and also addressed multi-instance detection and segmentation. The challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures from three different types of surgery. The validation of the competing methods for the three tasks (binary segmentation, multi-instance detection and multi-instance segmentation) was performed in three different stages with an increasing domain gap between the training and the test data. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap. While the average detection and segmentation quality of the best-performing algorithms is high, future research should concentrate on detection and segmentation of small, crossing, moving and transparent instrument(s) (parts).
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Medical Image Analysis |
Date Published | 11/2020 |
Publisher | Elsevier |
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Scientific Data 7, no. 1 (2020): 1-14.Status: Published
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Artificial intelligence is currently a hot topic in medicine. However, medical data is often sparse and hard to obtain due to legal restrictions and lack of medical personnel for the cumbersome and tedious process to manually label training data. These constraints make it difficult to develop systems for automatic analysis, like detecting disease or other lesions. In this respect, this article presents HyperKvasir, the largest image and video dataset of the gastrointestinal tract available today. The data is collected during real gastro- and colonoscopy examinations at Bærum Hospital in Norway and partly labeled by experienced gastrointestinal endoscopists. The dataset contains 110,079 images and 374 videos, and represents anatomical landmarks as well as pathological and normal findings. The total number of images and video frames together is around 1 million. Initial experiments demonstrate the potential benefits of artificial intelligence-based computer-assisted diagnosis systems. The HyperKvasir dataset can play a valuable role in developing better algorithms and computer-assisted examination systems not only for gastro- and colonoscopy, but also for other fields in medicine.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Scientific Data |
Volume | 7 |
Issue | 1 |
Pagination | 1-14 |
Date Published | 08/2020 |
Publisher | Springer Nature |
Keywords | dataset, GI, Machine learning |
URL | http://www.nature.com/articles/s41597-020-00622-y |
DOI | 10.1038/s41597-020-00622-y |
Proceedings, refereed
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation
In CBMS 2020: International Symposium on Computer-Based Medical Systems. IEEE, 2020.Status: Published
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | CBMS 2020: International Symposium on Computer-Based Medical Systems |
Publisher | IEEE |
Notes | This paper was nominated for the best paper award at CBMS 2020. |
Kvasir-instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
In International Conference on Multimedia Modeling. Springer, 2020.Status: Published
Kvasir-instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
Gastrointestinal pathologies are periodically screened, biopsied, and resected using surgical tools. Usually, the procedures and the treated or resected areas are not specifically tracked or analysed during or after colonoscopies. Information regarding disease borders, development, amount, and size of the resected area get lost. This can lead to poor follow-up and bothersome reassessment difficulties post-treatment. To improve the current standard and also to foster more research on the topic, we have released the ``Kvasir-Instrument'' dataset, which consists of 590 annotated frames containing GI procedure tools such as snares, balloons, and biopsy forceps, etc. Besides the images, the dataset includes ground truth masks and bounding boxes and has been verified by two expert GI endoscopists. Additionally, we provide a baseline for the segmentation of the GI tools to promote research and algorithm development. We obtained a dice coefficient score of 0.9158 and a Jaccard index of 0.8578 using a classical U-Net architecture. A similar dice coefficient score was observed for DoubleUNet. The qualitative results showed that the model did not work for the images with specularity and the frames with multiple tools, while the best result for both methods was observed on all other types of images. Both qualitative and quantitative results show that the model performs reasonably good, but there is potential for further improvements. Benchmarking using the dataset provides an opportunity for researchers to contribute to the field of automatic endoscopic diagnostic and therapeutic tool segmentation for GI endoscopy.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Conference on Multimedia Modeling |
Publisher | Springer |
Keywords | Benchmarking, Convolutional neural network, Gastrointestinal endoscopy, Tool segmentation |
URL | https://www.springerprofessional.de/en/kvasir-instrument-diagnostic-and-... |
Kvasir-SEG: A Segmented Polyp Dataset
In International Conference on Multimedia Modeling. Daejeon, Korea: Springer, 2020.Status: Published
Kvasir-SEG: A Segmented Polyp Dataset
Pixel-wise image segmentation is a highly demanding task in medical image analysis. In practice, it is difficult to find annotated medical images with corresponding segmentation masks. In this paper, we present Kvasir-SEG: an open-access dataset of gastrointestinal polyp images and corresponding segmentation masks, manually annotated and verified by an experienced gastroenterologist. We demonstrate the use of our dataset with a traditional segmentation approach and a modern deep learning based CNN approach. This work will be valuable for researchers to reproduce results and compare their methods in the future. By adding segmentation masks to the Kvasir dataset, which until today only consisted of framewise annotations, we enable multimedia and computer vision researchers to contribute in the field of polyp segmentation and automatic analysis of colonoscopy videos.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Conference on Multimedia Modeling |
Pagination | 451-462 |
Publisher | Springer |
Place Published | Daejeon, Korea |
Keywords | Kvasir-SEG dataset, Medical images, Polyp segmentation, ResUNet Fuzzy c-mean clustering, Semantic segmentation |
LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classification
In The Joint International Conference PDCAT-PAAP 2020. Springer, 2020.Status: Published
LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classification
Deep Neural Networks (DNNs) have become the de-facto standard in computer vision, as well as in many other pattern recognition tasks. A key drawback of DNNs is that the training phase can be very computationally expensive. Organizations or individuals that cannot afford purchasing state-of-the-art hardware or tapping into cloud-hosted infrastructures may face a long waiting time before the training completes or might not be able to train a model at all. Investigating novel ways to reduce the training time could be a potential solution to alleviate this drawback, and thus enabling more rapid development of new algorithms and models. In this paper, we propose LightLayers, a method for reducing the number of trainable parameters in deep neural networks (DNN). The proposed LightLayers consists of LightDense andLightConv2D layer that are as efficient as regular Conv2D and Dense layers, but uses less parameters. We resort to Matrix Factorization to reduce the complexity of the DNN models resulting into lightweight DNNmodels that require less computational power, without much loss in the accuracy. We have tested LightLayers on MNIST, Fashion MNIST, CI-FAR 10, and CIFAR 100 datasets. Promising results are obtained for MNIST, Fashion MNIST, CIFAR-10 datasets whereas CIFAR 100 shows acceptable performance by using fewer parameters.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The Joint International Conference PDCAT-PAAP 2020 |
Publisher | Springer |
Keywords | CIFAR-10, Convolutional neural network, Deep Learning, Fashion MNIST, Lightweight model, MNIST, Weight decomposition |
Medico Multimedia Task at MediaEval 2020: Automatic Polyp Segmentation
In Medico MediaEval 2020. CEUR, 2020.Status: Published
Medico Multimedia Task at MediaEval 2020: Automatic Polyp Segmentation
Colorectal cancer is the third most common cause of cancer worldwide. According to Global cancer statistics 2018, the incidence of colorectal cancer is increasing in both developing and developed countries. Early detection of colon anomalies such as polyps is important for cancer prevention, and automatic polyp segmentation can play a crucial role for this. Regardless of the recent advancement in early detection and treatment options, the estimated polyp miss rate is still around 20\%. Support via an automated computer-aided diagnosis system could be one of the potential solutions for the overlooked polyps. Such detection systems can help low-cost design solutions and save doctors time, which they could for example use to perform more patient examinations. In this paper, we introduce the 2020 Medico challenge, provide some information on related work and the dataset, describe the task and evaluation metrics, and discuss the necessity of organizing the Medico challenge.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Medico MediaEval 2020 |
Publisher | CEUR |
PMData: a sports logging dataset
In Proceedings of the 11th ACM Multimedia Systems Conference. ACM, 2020.Status: Published
PMData: a sports logging dataset
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Proceedings of the 11th ACM Multimedia Systems Conference |
Pagination | 231-236 |
Publisher | ACM |
The EndoTect 2020 Challenge: Evaluation and Comparison of Classification, Segmentation and Inference Time for Endoscopy
In 25th International Conference on Pattern Recognition (ICPR). IEEE, 2020.Status: Published
The EndoTect 2020 Challenge: Evaluation and Comparison of Classification, Segmentation and Inference Time for Endoscopy
The EndoTect challenge at the International Conference on Pattern Recognition 2020 aims to motivate the development of algorithms that aid medical experts in finding anomalies that commonly occur in the gastrointestinal tract. Using HyperKvasir, a large dataset containing images taken from several endoscopies, the participants competed in three tasks. Each task focuses on a specific requirement for making it useful in a real-world medical scenario. The tasks are (i) high classification performance in terms of prediction accuracy, (ii) efficient classification measured by the number of images classified per second, and (iii) pixel-level segmentation of specific anomalies. Hopefully, this can motivate different computer science researchers to help benchmark a crucial component of a future computer-aided diagnosis system, which in turn, could potentially save human lives.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 25th International Conference on Pattern Recognition (ICPR) |
Publisher | IEEE |
Proceedings, refereed
ResUNet++: An Advanced Architecture for Medical Image Segmentation
In 2019 IEEE International Symposium on Multimedia (ISM). San Diego, California, USA: IEEE, 2019.Status: Published
ResUNet++: An Advanced Architecture for Medical Image Segmentation
Accurate computer-aided polyp detection and segmentation during colonoscopy examinations can help endoscopists resect abnormal tissue and thereby decrease chances of polyps growing into cancer. Towards developing a fully automated model for pixel-wise polyp segmentation, we propose ResUNet++, which is an improved ResUNet architecture for colonoscopic image segmentation. Our experimental evaluations show that the suggested architecture produces good segmentation results on publicly available datasets. Furthermore, ResUNet++ significantly outperforms U-Net and ResUNet, two key state-of-the-art deep learning architectures, by achieving high evaluation scores with a dice coefficient of 81.33%, and a mean Intersection over Union (mIoU) of 79.27% for the Kvasir-SEG dataset and a dice coefficient of 79.55%, and a mIoU of 79.62% with CVC-612 dataset.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 IEEE International Symposium on Multimedia (ISM) |
Publisher | IEEE |
Place Published | San Diego, California, USA |
Keywords | colonoscopy, deep learning, health informatics, Medical image segmentation, Polyp segmentation, Semantic segmentation |
Proceedings, refereed
The Medico-Task 2018: Disease Detection in the Gastrointestinal Tract using Global Features and Deep Learning
In MediaEval 2018. Nice, France: MediaEval, 2018.Status: Published
The Medico-Task 2018: Disease Detection in the Gastrointestinal Tract using Global Features and Deep Learning
In this paper, we present our approach for the 2018 Medico Task classifying diseases in the gastrointestinal tract. We have proposed a system based on global features and deep neural networks. The best approach combines two neural networks and the reproducible experimental results signify the efficiency of the proposed model with an accuracy rate of 95.80%, a precision of 95.87%, and an F1-score of 95.80%.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | MediaEval 2018 |
Date Published | 10/2018 |
Publisher | MediaEval |
Place Published | Nice, France |
Keywords | CNN, deep learning, Gastrointestinal Disease Detection, Global Features, Medico-Task 2018, Transfer Learning |