Publications
Proceedings, refereed
A Streaming System for Large-scale Temporal Graph Mining of Reddit Data
In 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). Lyon, France: IEEE, 2022.Status: Published
A Streaming System for Large-scale Temporal Graph Mining of Reddit Data
Afilliation | Scientific Computing, Machine Learning |
Project(s) | Department of High Performance Computing , Enabling Graph Neural Networks at Exascale |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) |
Pagination | 1153-1162 |
Publisher | IEEE |
Place Published | Lyon, France |
URL | https://ieeexplore.ieee.org/document/9835250/http://xplorestaging.ieee.o... |
DOI | 10.1109/IPDPSW55747.2022.00189 |
Implementing Spatio-Temporal Graph Convolutional Networks on Graphcore IPUs
In 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). Lyon, France: IEEE, 2022.Status: Published
Implementing Spatio-Temporal Graph Convolutional Networks on Graphcore IPUs
Afilliation | Machine Learning |
Project(s) | Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) |
Pagination | 45-54 |
Publisher | IEEE |
Place Published | Lyon, France |
URL | https://ieeexplore.ieee.org/document/9835385/http://xplorestaging.ieee.o... |
DOI | 10.1109/IPDPSW55747.2022.00016 |
Journal Article
COVID-19 and 5G conspiracy theories: Long term observation of a digital wildfire
International Journal of Data Science and Analytics (2022).Status: Published
COVID-19 and 5G conspiracy theories: Long term observation of a digital wildfire
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | International Journal of Data Science and Analytics |
Publisher | Springer |
The connectivity network underlying the German’s Twittersphere: a testbed for investigating information spreading phenomena
Scientific Reports 12, no. 1 (2022).Status: Published
The connectivity network underlying the German’s Twittersphere: a testbed for investigating information spreading phenomena
Online social networks are ubiquitous, have billions of users, and produce large amounts of data. While platforms like Reddit are based on a forum-like organization where users gather around topics, Facebook and Twitter implement a concept in which individuals represent the primary entity of interest. This makes them natural testbeds for exploring individual behavior in large social networks. Underlying these individual-based platforms is a network whose “friend” or “follower” edges are of binary nature only and therefore do not necessarily reflect the level of acquaintance between pairs of users. In this paper,we present the network of acquaintance “strengths” underlying the German Twittersphere. To that end, we make use of the full non-verbal information contained in tweet–retweet actions to uncover the graph of social acquaintances among users, beyond pure binary edges. The social connectivity between pairs of users is weighted by keeping track of the frequency of shared content and the time elapsed between publication and sharing. Moreover, we also present a preliminary topological analysis of the German Twitter network. Finally, making the data describing the weighted German Twitter network of acquaintances, we discuss how to apply this framework as a ground basis for investigating spreading phenomena of particular contents.
Afilliation | Communication Systems |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Enabling Graph Neural Networks at Exascale |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Scientific Reports |
Volume | 12 |
Issue | 1 |
Date Published | Jan-12-2022 |
Publisher | Nature Publishing Group |
URL | https://www.nature.com/articles/s41598-022-07961-3 |
DOI | 10.1038/s41598-022-07961-3 |
Journal Article
A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging
Medical Image Analysis 70 (2021): 102007.Status: Published
A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging
Gastrointestinal endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Medical Image Analysis |
Volume | 70 |
Pagination | 102007 |
Publisher | Elsevier |
Keywords | Artificial intelligence, BioMedia 2019 Grand Challenge, Computer-aided detection and diagnosis, Gastrointestinal endoscopy challenges, Medical imaging, Medico Task 2017, Medico Task 2018 |
DOI | 10.1016/j.media.2021.102007 |
Don't Trust Your Eyes: Image Manipulation in the Age of DeepFakes
Frontiers in Communication 6 (2021).Status: Published
Don't Trust Your Eyes: Image Manipulation in the Age of DeepFakes
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Frontiers in Communication |
Volume | 6 |
Publisher | Frontiers Media SA |
Place Published | Lausanne, Switzerland |
URL | https://www.frontiersin.org/articles/10.3389/fcomm.2021.632317/full |
DOI | 10.3389/fcomm.2021.632317 |
Proceedings, refereed
Explaining the Performance of Supervised and Semi-Supervised Methods for Automated Sparse Matrix Format Selection
In 50th International Conference on Parallel Processing Workshop. Chicago, Illinois, USA: ACM, 2021.Status: Published
Explaining the Performance of Supervised and Semi-Supervised Methods for Automated Sparse Matrix Format Selection
Afilliation | Scientific Computing, Machine Learning |
Project(s) | Department of High Performance Computing , SparCity: An Optimization and Co-design Framework for Sparse Computation |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 50th International Conference on Parallel Processing Workshop |
Pagination | 1-10 |
Date Published | 08/2021 |
Publisher | ACM |
Place Published | Chicago, Illinois, USA |
iPUG: Accelerating Breadth-First Graph Traversals Using Manycore Graphcore IPUs
In High Performance Computing. ISC High Performance 2021. Vol. LNCS, volume 12728. Cham: Springer International Publishing, 2021.Status: Published
iPUG: Accelerating Breadth-First Graph Traversals Using Manycore Graphcore IPUs
The Graphcore Intelligence Processing Unit (IPU) is a newly developed processor type whose architecture does not rely on the traditional caching hierarchies. Developed to meet the need for more and more data-centric applications, such as machine learning, IPUs combine a dedicated portion of SRAM with each of its numerous cores, resulting in high memory bandwidth at the price of capacity. The proximity of processor cores and memory makes the IPU a promising field of experimentation for graph algorithms since it is the unpredictable, irregular memory accesses that lead to performance losses in traditional processors with pre-caching.
This paper aims to test the IPU’s suitability for algorithms with hard-to-predict memory accesses by implementing a breadth-first search (BFS) that complies with the Graph500 specifications. Precisely because of its apparent simplicity, BFS is an established benchmark that is not only subroutine for a variety of more complex graph algorithms, but also allows comparability across a wide range of architectures.
We benchmark our IPU code on a wide range of instances and compare its performance to state-of-the-art CPU and GPU codes. The results indicate that the IPU delivers speedups of up to 4×4× over the fastest competing result on an NVIDIA V100 GPU, with typical speedups of about 1.5×1.5× on most test instances.
Afilliation | Scientific Computing |
Project(s) | Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | High Performance Computing. ISC High Performance 2021 |
Volume | LNCS, volume 12728 |
Pagination | 291-309 |
Publisher | Springer International Publishing |
Place Published | Cham |
ISBN Number | 978-3-030-78712-7 |
ISSN Number | 0302-9743 |
Keywords | BFS, Graph500, IPU, Performance optimization |
URL | https://link.springer.com/10.1007/978-3-030-78713-4 |
DOI | 10.1007/978-3-030-78713-4 |
WICO Graph: a Labeled Dataset of Twitter Subgraphs based on Conspiracy Theory and 5G-Corona Misinformation Tweets
In Proceedings of the 13th International Conference on Agents and Artificial Intelligence (ICAART 2021). Vol. 2. SCITEPRESS, 2021.Status: Published
WICO Graph: a Labeled Dataset of Twitter Subgraphs based on Conspiracy Theory and 5G-Corona Misinformation Tweets
In the wake of the COVID-19 pandemic, a surge of misinformation has flooded social media and other internet channels, and some of it has the potential to cause real-world harm. To counteract this misinformation, reliably identifying it is a principal problem to be solved. However, the identification of misinformation poses a formidable challenge for language processing systems since the texts containing misinformation are short, work with insinuation rather than explicitly stating a false claim, or resemble other postings that deal with the same topic ironically. Accordingly, for the development of better detection systems, it is not only essential to use hand-labeled ground truth data and extend the analysis with methods beyond Natural Language Processing to consider the characteristics of the participant's relationships and the diffusion of misinformation. This paper presents a novel dataset that deals with a specific piece of misinformation: the idea that the 5G wireless network is causally connected to the COVID-19 pandemic. We have extracted the subgraphs of 3,000 manually classified Tweets from Twitter's follower network and distinguished them into three categories. First, subgraphs of Tweets that propagate the specific 5G misinformation, those that spread other conspiracy theories, and Tweets that do neither. We created the WICO (Wireless Networks and Coronavirus Conspiracy) dataset to support experts in machine learning experts, graph processing, and related fields in studying the spread of misinformation. Furthermore, we provide a series of baseline experiments using both Graph Neural Networks and other established classifiers that use simple graph metrics as features. The dataset is available at https://datasets.simula.no/wico-graph
Afilliation | Scientific Computing |
Project(s) | Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proceedings of the 13th International Conference on Agents and Artificial Intelligence (ICAART 2021) |
Volume | 2 |
Pagination | 257-266 |
Publisher | SCITEPRESS |
ISBN Number | 978-989-758-484-8 |
DOI | 10.5220/0010262802570266 |
WICO Text: A Labeled Dataset of Conspiracy Theory and 5G-Corona Misinformation Tweets
In Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks (OASIS '21). ACM, 2021.Status: Published
WICO Text: A Labeled Dataset of Conspiracy Theory and 5G-Corona Misinformation Tweets
Afilliation | Machine Learning |
Project(s) | Department of High Performance Computing , Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks (OASIS '21) |
Pagination | 21-25 |
Publisher | ACM |
Talks, contributed
Will technical means help in preventing digital wildfires?
In 6th World Conference on Media and Mass Communication, Cagliari, Italy, 2021.Status: Published
Will technical means help in preventing digital wildfires?
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Talks, contributed |
Year of Publication | 2021 |
Location of Talk | 6th World Conference on Media and Mass Communication, Cagliari, Italy |
Poster
A Framework for Interaction-based Propagation Analysis in Online Social Networks
Complex Networks, 2020.Status: Published
A Framework for Interaction-based Propagation Analysis in Online Social Networks
Afilliation | Scientific Computing |
Project(s) | Department of High Performance Computing , Department of Holistic Systems, UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Poster |
Year of Publication | 2020 |
Place Published | Complex Networks |
Graph Structure Based Monitoring of Digital Wildfires
6th International Conference on Computational Social Science, Boston, MA, USA, 2020.Status: Published
Graph Structure Based Monitoring of Digital Wildfires
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Poster |
Year of Publication | 2020 |
Date Published | 07/2020 |
Place Published | 6th International Conference on Computational Social Science, Boston, MA, USA |
Type of Work | Poster |
Keywords | Digital wildfires, Graph neural networks, Large scale infrastructure, Misinformation, Social network analysis |
URL | http://2020.ic2s2.org/program |
Proceedings, refereed
A Framework for Interaction-based Propagation Analysis in Online Social Networks
In Complex Networks. Springer, 2020.Status: Accepted
A Framework for Interaction-based Propagation Analysis in Online Social Networks
Online social networks create a digital footprint of human interaction naturally by the way they function. Thus, they allow a large-scale analysis of human behavior which was previously infeasible for social scientists. Consequently, social networks have been studied intensely in the last decade. The core of most social networks is the relationship between users which can be described as a graph. The graph can be either undirected, as is the case for the friendship relation of Facebook, or directed, which is the case of the follower relation on Twitter. The relationship is readily visible, e.g. on the user interface of the social networks themselves. However, these edges are unweighted expressions of interest and reflect how individuals have chosen to relate to each other rather than how they actually interact with each other. For studying information propagation, comparing interaction properties is crucial and, therefore, using models based on connections that reflect different dimensions and strengths of acquaintance seems appropriate. Thus, there is a need for obtaining weighted edges from the communication that occurs on the social network. In this paper, we present a novel method to calculate an acquaintance score between pairs of Twitter users and use the resulting networks to enable the analysis of interaction based information propagation. By understanding the frequency and velocity with which individuals share content as a measure of acquaintance, it becomes possible to predict, compare communication patterns, and detect unusual communication. In contrast to previous work which assigns edge weights based on tie strength, our score considers the response time as a crucial factor and, therefore, enables time-based spreading comparisons.
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Complex Networks |
Publisher | Springer |
Notes | Extended abstract |
A Scalable System for Bundling Online Social Network Mining Research
In 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS). IEEE, 2020.Status: Published
A Scalable System for Bundling Online Social Network Mining Research
Online social networks such as Facebook and Twitter are part of the everyday life of millions of people. They are not only used for interaction but play an essential role when it comes to information acquisition and knowledge gain. The abundance and detail of the accumulated data in these online social networks open up new possibilities for social researchers and psychologists, allowing them to study behavior in a large test population. However, complex application programming interfaces (API) and data scraping restrictions are, in many cases, a limiting factor when accessing this data. Furthermore, research projects are typically granted restricted access based on quotas. Thus, research tools such as scrapers that access social network data through an API must manage these quotas. While this is generally feasible, it becomes a problem when more than one tool, or multiple instances of the same tool, is being used in the same research group. Since different tools typically cannot balance access to a shared quota on their own, additional software is needed to prevent the individual tools from overusing the shared quota. In this paper, we present a proxy server that manages several researchers' data contingents in a cooperative research environment and thus enables a transparent view of a subset of Twitter's API. Our proxy scales linearly with the number of clients in use and incurs almost no performance penalties or implementation overhead to further layer or applications that need to work with the Twitter API. Thus, it allows seamless integration of multiple API accessing programs within the same research group.
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS) |
Pagination | 1-6 |
Publisher | IEEE |
A System for High Performance Mining on GDELT Data
In 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2020.Status: Published
A System for High Performance Mining on GDELT Data
We design a system for efficient in-memory analysis of data from the GDELT database of news events. The specialization of the system allows us to avoid the inefficiencies of existing alternatives, and make full use of modern parallel high-performance computing hardware. We then present a series of experiments showcasing the system’s ability to analyze correlations in the entire GDELT 2.0 database containing more than a billion news items. The results reveal large scale trends in the world of today’s online news.
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) |
Date Published | 05/2020 |
Publisher | IEEE |
Keywords | Data mining, GDELT, High Performance Computing, Misinformation, Publishing |
Evaluating Standard Classifiers for Detecting COVID-19 related Misinformation
In MediaEval 2020. CEUR, 2020.Status: Published
Evaluating Standard Classifiers for Detecting COVID-19 related Misinformation
Afilliation | Machine Learning |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of High Performance Computing , Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | MediaEval 2020 |
Publisher | CEUR |
FakeNews: Corona Virus and 5G Conspiracy Task at MediaEval 2020
In Media Eval Challange 2020. CEUR, 2020.Status: Published
FakeNews: Corona Virus and 5G Conspiracy Task at MediaEval 2020
Afilliation | Scientific Computing, Machine Learning |
Project(s) | Department of High Performance Computing , Department of Holistic Systems, UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Media Eval Challange 2020 |
Publisher | CEUR |
Journal Article
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Scientific Data 7, no. 1 (2020): 1-14.Status: Published
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Artificial intelligence is currently a hot topic in medicine. However, medical data is often sparse and hard to obtain due to legal restrictions and lack of medical personnel for the cumbersome and tedious process to manually label training data. These constraints make it difficult to develop systems for automatic analysis, like detecting disease or other lesions. In this respect, this article presents HyperKvasir, the largest image and video dataset of the gastrointestinal tract available today. The data is collected during real gastro- and colonoscopy examinations at Bærum Hospital in Norway and partly labeled by experienced gastrointestinal endoscopists. The dataset contains 110,079 images and 374 videos, and represents anatomical landmarks as well as pathological and normal findings. The total number of images and video frames together is around 1 million. Initial experiments demonstrate the potential benefits of artificial intelligence-based computer-assisted diagnosis systems. The HyperKvasir dataset can play a valuable role in developing better algorithms and computer-assisted examination systems not only for gastro- and colonoscopy, but also for other fields in medicine.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Scientific Data |
Volume | 7 |
Issue | 1 |
Pagination | 1-14 |
Date Published | 08/2020 |
Publisher | Springer Nature |
Keywords | dataset, GI, Machine learning |
URL | http://www.nature.com/articles/s41597-020-00622-y |
DOI | 10.1038/s41597-020-00622-y |
Proceedings, refereed
ACM Multimedia BioMedia 2019 Grand Challenge Overview
In The ACM International Conference on Multimedia (ACM MM). New York, New York, USA: ACM Press, 2019.Status: Published
ACM Multimedia BioMedia 2019 Grand Challenge Overview
The BioMedia 2019 ACM Multimedia Grand Challenge is the first in a series of competitions focusing on the use of multimedia for different medical use-cases. In this year’s challenge, the participants are asked to develop efficient algorithms which automatically detect a variety of findings commonly identified in the gastrointestinal (GI) tract (a part of the human digestive system). The purpose of this task is to develop methods to aid medical doctors performing routine endoscopy inspections of the GI tract. In this paper, we give a detailed description of the four different tasks of this year’s challenge, present the datasets used for training and testing, and discuss how each submission is evaluated both qualitatively and quantitatively.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | The ACM International Conference on Multimedia (ACM MM) |
Pagination | 2563-2567 |
Date Published | 10/2019 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450368896 |
URL | http://dl.acm.org/citation.cfm?doid=3343031http://dl.acm.org/citation.cf... |
DOI | 10.1145/334303110.1145/3343031.3356058 |
FACT: a Framework for Analysis and Capture of Twitter Graphs
In The Sixth IEEE International Conference on Social Networks Analysis, Management and Security (SNAMS-2019) . IEEE, 2019.Status: Published
FACT: a Framework for Analysis and Capture of Twitter Graphs
In the recent years, online social networks have become an important source of news and the primary place for political debates for a growing part of the population. At the same time, the spread of fake news and digital wildfires (fast- spreading and harmful misinformation) has become a growing concern worldwide, and online social networks the problem is most prevalent. Thus, the study of social networks is an essential component in the understanding of the fake news phenomenon. Of particular interest is the network connectivity between participants, since it makes communication patterns visible. These patterns are hidden in the offline world, but they have a profound impact on the spread of ideas, opinions and news. Among the major social networks, Twitter is of special interest. Because of its public nature, Twitter offers the possibility to perform research without the risk of breaching the expectation of privacy. However, obtaining sufficient amounts of data from Twitter is a fundamental challenge for many researchers. Thus, in this paper, we present a scalable framework for gathering the graph structure of follower networks, posts and profiles. We also show how to use the collected data for high-performance social network analysis.
Afilliation | Software Engineering |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | The Sixth IEEE International Conference on Social Networks Analysis, Management and Security (SNAMS-2019) |
Pagination | 134-141 |
Publisher | IEEE |
DOI | 10.1109/SNAMS.2019.8931870 |
Medico Multimedia Task at MediaEval 2019
In MediaEval. CEUR Workshop Proceedings, 2019.Status: Published
Medico Multimedia Task at MediaEval 2019
The Medico: Multimedia for Medicine Task is running for the third time as part of MediaEval 2019. This year, we have changed the task from anomaly detection in images of the gastrointestinal tract to focus on the automatic prediction of human semen quality based on videos. The purpose of this task is to aid in the assessment of male reproductive health by providing a quick and consisted method of analyzing human semen. In this paper, we describe the task in detail, give a brief description of the provided dataset, and discuss the evaluation process and the metrics used to rank the submissions of the participants.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Publisher | CEUR Workshop Proceedings |
Multi-Modal Machine Learning for Flood Detection in News, Social Media and Satellite Sequences
In Multimediaeval Benchmark 2019. CEUR Workshop Proceedings, 2019.Status: Published
Multi-Modal Machine Learning for Flood Detection in News, Social Media and Satellite Sequences
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Multimediaeval Benchmark 2019 |
Date Published | 10/2019 |
Publisher | CEUR Workshop Proceedings |
Predicting Sperm Motility and Morphology using Deep Learning and Handcrafted Features
In MediaEval. ceur ws org, 2019.Status: Published
Predicting Sperm Motility and Morphology using Deep Learning and Handcrafted Features
This paper presents the approach proposed by the organizer team (SimulaMet) for MediaEval 2019 Multimedia for Medicine: The Medico Task. The approach uses a data preparation method which is based on global features extracted from multiple frames within each video and then combines this with information about the patient in order to create a compressed representation of each video. The goal is to create a less hardware expensive data representation that still retains the temporal information of the video and related patient data. Overall, the results need some improvement before being a viable option for clinical use.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Publisher | ceur ws org |
The Multimedia Satellite Task at MediaEval 2019: Estimation of Flood Severity
In MediaEval. CEUR Workshop Proceedings, 2019.Status: Published
The Multimedia Satellite Task at MediaEval 2019: Estimation of Flood Severity
This paper provides a description of the Multimedia Satellite Task at MediaEval 2019. The main objective of the task is to extract complementary information associated with events which are present in Satellite Imagery and news articles.
Due to their high socio-economic impact, we focus on flooding events and built upon the last two years of the Multimedia Satellite Task. Our task focuses this year on flood severity estimation and consists of three subtasks: (1) Image-based News Topic Disambiguation, (2) Multimodal Flood Level Estimation from news, (3) Classification of city-centered satellite sequences. The task moves forward the state of the art in flood impact assessment by concentrating on aspects that are important but are not generally studied by multimedia researchers.
Afilliation | Scientific Computing |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Publisher | CEUR Workshop Proceedings |
Journal Article
Automatic detection of passable roads after floods in remote sensed and social media data
Signal Processing: Image Communication 74 (2019): 110-118.Status: Published
Automatic detection of passable roads after floods in remote sensed and social media data
This paper addresses the problem of floods classification and floods aftermath detection based onboth social media and satellite imagery. Automatic detection of disasters such as floods is still a very challenging task. The focus lies on identifying passable routes or roads during floods. Two novel solutions are presented, which were developed for two corresponding tasks at the MediaEval 2018 benchmarking challenge. The tasks are (i) identification of images providing evidence for road passability and (ii) differentiation and detection of passable and non-passable roads in images from two complementary sources of information. For the first challenge, we mainly rely on object and scene-level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are then combined using early, late and double fusion techniques. To identify whether or not it is possible for a vehicle to pass a road in satellite images, we rely on Convolutional Neural Networks and a transfer learning-based classification approach. The evaluation of the proposed methods is carried out on the large-scale datasets provided for the benchmark competition. The results demonstrate significant improvement in the performance over the recent state-of-art approaches.
Afilliation | Communication Systems |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Signal Processing: Image Communication |
Volume | 74 |
Pagination | 110-118 |
Publisher | Elsevier |
Keywords | convolutional neural networks, Flood detection, Multimedia Indexing and Retrieval, Natural Disasters, Satellite Imagery, Social Media |
DOI | 10.1016/j.image.2019.02.002 |
Bleeding detection in wireless capsule endoscopy videos—Color versus texture features
Journal of applied clinical medical physics 20, no. 8 (2019): 141-154.Status: Published
Bleeding detection in wireless capsule endoscopy videos—Color versus texture features
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Journal of applied clinical medical physics |
Volume | 20 |
Issue | 8 |
Pagination | 141-154 |
Publisher | Wiley Online Library |
Deep Learning for Automatic Generation of Endoscopy Reports
Gastrointestinal Endoscopy 89, no. 6 (2019).Status: Published
Deep Learning for Automatic Generation of Endoscopy Reports
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Gastrointestinal Endoscopy |
Volume | 89 |
Issue | 6 |
Date Published | 06/2019 |
Publisher | Elsevier |
Place Published | Gastrointestinal Endoscopy |
DOI | 10.1016/j.gie.2019.04.053 |
Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming
International Journal of Semantic Computing 13, no. 3 (2019): 367-391.Status: Published
Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming
360 panorama video displayed through Virtual reality (VR) glasses or large screens o®ers immersive user experiences, but as such technology becomes commonplace, the need for e±cient streaming methods of such high-bitrate videos arises. In this respect, the attention that 360panorama video has received lately is huge. Many methods have already been proposed, and in this paper, we shed more light on the di®erent trade-o®s in order to save bandwidth while preserving the video quality in the user's ̄eld-of-view (FoV). Using 360 VR content delivered to a Gear VR head-mounted display with a Samsung Galaxy S7 and to a Huawei Q22 set-top- box, we have tested various tiling schemes analyzing the tile layout, the tiling and encoding overheads, mechanisms for faster quality switching beyond the DASH segment boundaries and quality selection con ̄gurations. In this paper, we present an e±cient end-to-end design and real-world implementation of such a 360 streaming system. Furthermore, in addition to researching an on-demand system, we also go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-o® between bandwidth usage and the video quality in the user's FoV. We have created an architecture that combines RTP and DASH, and our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-o®s through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the FoV changes.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | International Journal of Semantic Computing |
Volume | 13 |
Issue | 3 |
Number | 3 |
Pagination | 367-391 |
Publisher | World Scientific |
Natural disasters detection in social media and satellite imagery: a survey
Multimedia Tools and Applications 78, no. 22 (2019): 31267-31302.Status: Published
Natural disasters detection in social media and satellite imagery: a survey
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Multimedia Tools and Applications |
Volume | 78 |
Issue | 22 |
Pagination | 31267 - 31302 |
Date Published | Jan-11-2019 |
Publisher | Springer |
ISSN | 1380-7501 |
DOI | 10.1007/s11042-019-07942-1 |
Social media and satellites: Disaster event detection, linking and summarization
Multimedia Tools and Applications 78 (2019): 2837-2875.Status: Published
Social media and satellites: Disaster event detection, linking and summarization
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Multimedia Tools and Applications |
Volume | 78 |
Number | 3 |
Pagination | 2837–2875 |
Publisher | Springer |
Place Published | Netherlands |
PhD Thesis
DeepEIR: A Holistic Medical Multimedia System for Gastrointestinal Tract Disease Detection and Localization
In The University of Oslo. Vol. PhD. Department of Informatics, University of Oslo, 2019.Status: Published
DeepEIR: A Holistic Medical Multimedia System for Gastrointestinal Tract Disease Detection and Localization
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | PhD Thesis |
Year of Publication | 2019 |
Degree awarding institution | The University of Oslo |
Degree | PhD |
Publisher | Department of Informatics, University of Oslo |
Poster
Efficient Processing of Medical Videos in a Multi-auditory Environment Using Gpu Lending
NVIDIA's GPU Technology Conference (GTC), 2019.Status: Published
Efficient Processing of Medical Videos in a Multi-auditory Environment Using Gpu Lending
Afilliation | Software Engineering |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Poster |
Year of Publication | 2019 |
Place Published | NVIDIA's GPU Technology Conference (GTC) |
Talk, keynote
Automatic Detection of Angiectasia: Evaluation of Deep Learning and Handcrafted Approaches
In IEEE Conference on Biomedical and Health Informatics (BHI) 2018, 2018.Status: Published
Automatic Detection of Angiectasia: Evaluation of Deep Learning and Handcrafted Approaches
Angiectasia, formerly called angiodysplasia, is one of the most frequent vascular lesions and often the cause of gastrointestinal bleedings. Medical specialists assessing videos of examinations reach a detection performance of 16% for the detection of bleeding to 69% for the detection of angiectasia. In this paper, we present several machine-learning-based approaches for angiectasia detection in wireless video capsule endoscopy images. The most promising results for pixel-wise localization and frame-wise detection are obtained by the proposed deep learning approach using generative adversarial networks (GANs) with a sensitivity of 88% and specificity of 99.9% for pixel-wise localization and a sensitivity of 98% and a specificity of 100% for frame-wise detection, which fits the requirements for automatic angiectasia detection in real clinical settings.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | IEEE Conference on Biomedical and Health Informatics (BHI) 2018 |
Proceedings, refereed
Comprehensible Reasoning and Automated Reporting of Medical Examinations Based on Deep Learning Analysis
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
Comprehensible Reasoning and Automated Reporting of Medical Examinations Based on Deep Learning Analysis
Afilliation | Communication Systems |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 490-493 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208113 |
Deep Learning and Hand-crafted Feature Based Approaches for Polyp Detection in Medical Videos
In 31st IEEE CBMS International Symposium on Computer-Based Medical Systems. Karlstad, Sweden: IEEE, 2018.Status: Published
Deep Learning and Hand-crafted Feature Based Approaches for Polyp Detection in Medical Videos
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 31st IEEE CBMS International Symposium on Computer-Based Medical Systems |
Pagination | 381-386 |
Publisher | IEEE |
Place Published | Karlstad, Sweden |
ISSN Number | 2372-9198 |
DOI | 10.1109/CBMS.2018.00073 |
Deep Learning and Handcrafted Feature Based Approaches for Automatic Detection of Angiectasia
In 2018 IEEE Conference on Biomedical and Health Informatics (BHI). IEEE, 2018.Status: Published
Deep Learning and Handcrafted Feature Based Approaches for Automatic Detection of Angiectasia
Angiectasia, formerly called angiodysplasia, is one of the most frequent vascular lesions and often the cause of gastrointestinal bleedings. Medical specialists assessing videos or images of examinations reach a detection performance of 16% for the detection of bleeding to 69% for the detection of angiectasia. This shows that automatic detection to support medical experts can be useful. In this paper, we present several machine learning-based approaches for angiectasia detection in wireless video capsule endoscopy frames. In summary, the most promising results for pixel-wise localization and framewise detection are obtained by the proposed deep learning method using generative adversarial networks (GANs). Using this approach, we achieve a sensitivity of 88% and specificity of 99.9% for pixel-wise localization, and a sensitivity of 98% and a specificity of 100% for frame-wise detection. Thus, the results demonstrate the capability of using deep learning for automatic angiectasia detection in real clinical settings.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE Conference on Biomedical and Health Informatics (BHI) |
Pagination | 365-368 |
Publisher | IEEE |
Keywords | Angiectasia, computer aided diagnosis, deep learning, Machine learning, video capsular endoscopy |
DOI | 10.1109/BHI.2018.8333444 |
Deep learning approaches for flood classification and flood aftermath detection
In Working Notes Proceedings of the MediaEval 2018 Workshop. Vol. 2283. Sophia Antipolis, France: CEUR-WS.org, 2018.Status: Published
Deep learning approaches for flood classification and flood aftermath detection
This paper presents the method proposed by team UTAOS for MediaEval 2018 Multimedia Satellite Task: Emergency Response for Flooding Events. In the first challenge, we mainly rely on object and scene level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are combined using early, late and double fusion techniques achieving an average F1-score of 65.03%, 60.59% and 63.58%, respectively. For the second challenge, we rely on a convolutional neural networks (CNNs) and a transfer learning-based classification approach achieving an average F1-score of 62.30% and 61.02% for run 1 and run 2, respectively.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Volume | 2283 |
Publisher | CEUR-WS.org |
Place Published | Sophia Antipolis, France |
Dissecting Deep Neural Networks for Better Medical Image Classification and Classification Understanding
In 31st IEEE CBMS International Symposium on Computer-Based Medical Systems. Karlstad, Sweden: IEEE, 2018.Status: Published
Dissecting Deep Neural Networks for Better Medical Image Classification and Classification Understanding
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 31st IEEE CBMS International Symposium on Computer-Based Medical Systems |
Publisher | IEEE |
Place Published | Karlstad, Sweden |
ISSN Number | 2372-9198 |
DOI | 10.1109/CBMS.2018.00070 |
Efficient Live and on-Demand Tiled HEVC 360 VR Video Streaming
In 2018 IEEE International Symposium on Multimedia (ISM). Taichung, Taiwan: IEEE, 2018.Status: Published
Efficient Live and on-Demand Tiled HEVC 360 VR Video Streaming
With 360◦ panorama video technology becoming commonplace, the need for efficient streaming methods for such videos arises. We go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-off between bandwidth usage and the video quality in the user’s field-of-view. We have created an architecture that combines RTP and DASH to deliver 360◦ VR content to a Huawei set-top-box and a Samsung Galaxy S7. Our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-offs through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the field-of- view changes.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE International Symposium on Multimedia (ISM) |
Pagination | 81-88 |
Date Published | 12/2018 |
Publisher | IEEE |
Place Published | Taichung, Taiwan |
DOI | 10.1109/ISM.2018.00022 |
Medico Multimedia Task at MediaEval 2018
In Working Notes Proceedings of the MediaEval 2018 Workshop. CEUR Workshop Proceedings, 2018.Status: Published
Medico Multimedia Task at MediaEval 2018
The Medico: Multimedia for Medicine Task, running for the second time as part of MediaEval 2018, focuses on detecting abnormalities, diseases, anatomical landmarks and other findings in images captured by medical devices in the gastrointestinal tract. The task is described, including the use case and its challenges, the dataset with ground truth, the required participant runs and the evaluation metrics.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Publisher | CEUR Workshop Proceedings |
Mimir: An Automatic Reporting and Reasoning System for Deep Learning based Analysis in the Medical Domain
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
Mimir: An Automatic Reporting and Reasoning System for Deep Learning based Analysis in the Medical Domain
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 369-374 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208129 |
OpenSea - Open Search Based Classification Tool
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
OpenSea - Open Search Based Classification Tool
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 363-368 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208128 |
Tradeoffs using Binary and Multiclass Neural Network Classification for Medical Multidisease Detection
In 2018 IEEE International Symposium on Multimedia (ISM). IEEE, 2018.Status: Published
Tradeoffs using Binary and Multiclass Neural Network Classification for Medical Multidisease Detection
The interest in neural networks has increased sig- nificantly, and the application of this type of machine learning is vast, ranging from natural image classification to medical image segmentation. However, many users of neural networks tend to use them as a black box tool. They do not access all of the possible variations, nor take into account the respective classification accuracies and costs. In our work, we focus on multiclass image classification, and in this research, we shed light on the trade-offs between systems using a single multiclass classification and multiple binary classifiers, respectively. We have tested the these classifiers on several modern neural network architectures, including DenseNet, Inception v3, Inception ResNet v2, Xception, NASNet and MobileNet. We have compared several aspects of the performance of these architectures during training and testing using both classification styles. We have compared classification speed and several classification accuracy metrics. Here, we present the results from experiments on a total of 99 networks: 11 multiclass and 88 individual binary networks, for an 8-class classification of medical images. In short, using multiple binary classification networks resulted in a 7% increase in the average F1 score, a 1% increase in average accuracy, a 1% increase in precision, and a 4% increase in average recall. However, on average, such a multi-network style performed the classification 7.6 times slower compared to a single network multiclass implementation. These collective findings show that both approaches can be applied to modern neural network structures. Several binary networks will often give increased classification accuracy, but at the cost of classification speed and resource consumption.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE International Symposium on Multimedia (ISM) |
Pagination | 1-8 |
Date Published | 12/2018 |
Publisher | IEEE |
DOI | 10.1109/ISM.2018.00009 |
Transfer learning with prioritized classification and training dataset equalization for medical objects detection
In Working Notes Proceedings of the MediaEval 2018 Workshop. CEUR Workshop Proceedings, 2018.Status: Published
Transfer learning with prioritized classification and training dataset equalization for medical objects detection
This paper presents the method proposed by the organizer team (SIMULA) for MediaEval 2018 Multimedia for Medicine: the Medico Task. We utilized the recent transfer-learning-based image classification methodology and focused on how easy it is to implement multi-class image classifiers in general and how to improve the classification performance without deep neural network model redesign. The goal for this was both to provide a baseline for the Medico task and to show the performance of out-of-the-box classifiers for the medical use-case scenario.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Publisher | CEUR Workshop Proceedings |
Journal Article
Social Media and Satellites. Disaster event detection, linking and summarization
Multimedia Tools and Applications 78, no. 3 (2018): 2837-2875.Status: Published
Social Media and Satellites. Disaster event detection, linking and summarization
Being able to automatically link social media and satellite imagery holds large opportunities for research, with a potentially considerable impact on society. The possibility of integrating different information sources opens in fact to new scenarios where the wide coverage of satellite imaging can be used as a collector of the fine-grained details provided by the social media. Remote-sensed data and social media data can well complement each other, integrating the wide perspective provided by the satellite view with the information collected locally, being it textual, audio, or visual. Among the possible applications, natural disasters are certainly one of the most interesting scenarios, where global and local perspectives are needed at the same time.
In this paper, we present a system called JORD that is able to autonomously collect social media data (including the text analysis in local languages) about technological and environmental disasters, and link it automatically to remote-sensed data. Moreover, in order to ensure the quality of retrieved information, JORD is equipped with a hierarchical filtering mechanism relying on the temporal information and the content analysis of retrieved multimedia data.
To show the capabilities of the system, we present a large number of disaster events detected by the system, and we evaluate both the quality of the provided information about the events and the usefulness of JORD from potential users viewpoint, using crowdsourcing.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Multimedia Tools and Applications |
Volume | 78 |
Issue | 3 |
Pagination | 2837–2875 |
Publisher | Springer |
Place Published | US |
Keywords | Event Detection, Information retrieval, Natural Disaster, Social Media |
DOI | 10.1007/s11042-018-5982-9 |
Top-Down Saliency Detection Driven by Visual Classification
Computer Vision and Image Understanding 172 (2018): 67-76.Status: Published
Top-Down Saliency Detection Driven by Visual Classification
This paper presents an approach for saliency detection able to emulate the integration of the top-down (task-controlled) and bottom-up (sensory information) processes involved in human visual attention. In particular, we first learn how to generate saliency when a specific visual task has to be accomplished. Afterwards, we investigate if and to what extent
the learned saliency maps can support visual classification in nontrivial cases. To achieve this, we propose SalClass-
Net, a CNN framework consisting of two networks jointly trained: a) the first one computing top-down saliency maps
from input images, and b) the second one exploiting the computed saliency maps for visual classification.
To test our approach, we collected a dataset of eye-gaze maps, using a Tobii T60 eye tracker, by asking several subjects
to look at images from the Stanford Dogs dataset, with the objective of distinguishing dog breeds.
Performance analysis on our dataset and other saliency benchmarking datasets, such as POET, showed that Sal-
ClassNet outperforms state-of-the-art saliency detectors, such as SalNet and SALICON. Finally, we also analyzed
the performance of SalClassNet in a fine-grained recognition task and found out that it yields enhanced classification
accuracy compared to Inception and VGG-19 classifiers. The achieved results, thus, demonstrate that 1) condition-
ing saliency detectors with object classes reaches state-of-the-art performance, and 2) explicitly providing top-down
saliency maps to visual classifiers enhances accuracy.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Computer Vision and Image Understanding |
Volume | 172 |
Pagination | 67-76 |
Publisher | Elsevier |
DOI | 10.1016/j.cviu.2018.03.005 |
Proceedings, refereed
A comparison of deep learning with global features for gastrointestinal disease detection
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
A comparison of deep learning with global features for gastrointestinal disease detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
A Holistic Multimedia System for Gastrointestinal Tract Disease Detection
In 8th annual ACM conference on Multimedia Systems (MMSys). ACM, 2017.Status: Published
A Holistic Multimedia System for Gastrointestinal Tract Disease Detection
Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.
In this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | 8th annual ACM conference on Multimedia Systems (MMSys) |
Pagination | 112-123 |
Date Published | 06/2017 |
Publisher | ACM |
ISBN Number | 978-1-4503-5002-0 |
URL | http://dl.acm.org/citation.cfm?id=3083189 |
DOI | 10.1145/3083187.3083189 |
ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections
In ACM International Conference on Multimedia Retrieval. Bucharest: ACM, 2017.Status: Published
ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Conference on Multimedia Retrieval |
Date Published | 06/2017 |
Publisher | ACM |
Place Published | Bucharest |
CNN and GAN Based Satellite and Social Media Data Fusion for Disaster Detection
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
CNN and GAN Based Satellite and Social Media Data Fusion for Disaster Detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
Detection and Classification of Bleeding Region in WCE Images Using Color Feature
In Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing. New York, NY, USA: ACM, 2017.Status: Published
Detection and Classification of Bleeding Region in WCE Images Using Color Feature
Wireless capsule endoscopy (WCE) is a modern and efficient technology to diagnose complete gastrointestinal tract (GIT) for various abnormalities. Due to long recording time of WCE, it acquires a huge amount of images, which is very tedious for clinical expertise to inspect each and every frame of a complete video footage. In this paper, an automated color feature based technique of bleeding detection is proposed. In case of bleeding, color is a very important feature for an efficient information extraction. Our algorithm is based on statistical color feature analysis and we use support vector machine (SVM) to classify WCE video frames into bleeding and non-bleeding classes with a high processing speed. An experimental evaluation shows that our method has promising bleeding detection performance with sensitivity and specificity higher than existing approaches.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing |
Pagination | 17:1–17:6 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-5333-5 |
Keywords | Bleeding detection, Color feature, Support vector machine, Wireless Capsule Endoscopy |
URL | http://doi.acm.org/10.1145/3095713.3095731 |
DOI | 10.1145/3095713.3095731 |
EIR: changing the scene of automatic detection software for gastrointestinal endoscopy
In World Congress of GI Endoscopy. Hyderabad, India: World Endoscopic Organisation, 2017.Status: Published
EIR: changing the scene of automatic detection software for gastrointestinal endoscopy
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | World Congress of GI Endoscopy |
Date Published | 02/2017 |
Publisher | World Endoscopic Organisation |
Place Published | Hyderabad, India |
JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery
In Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing - CBMI '17. New York, USA: ACM Press, 2017.Status: Published
JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery
Gathering information, and continuously monitoring the affected areas after a natural disaster can be crucial to assess the damage, and speed up the recovery process. Satellite imagery is being considered as one of the most productive sources to monitor the after effects of a natural disaster; however, it also comes with a lot of challenges and limitations, due to slow update. It would be beneficiary to link remote sensed data with social media for the damage assessment, and obtaining detailed information about a disaster. The additional information, which are obtainable by social media, can enrich remote-sensed data, and overcome its limitations. To tackle this, we present a system called JORD that is able to autonomously collect social media data about natural disasters, and link it automatically to remote-sensed data. In addition, we demonstrate that queries in local languages that are relevant to the exact position of natural disasters retrieve more accurate information about a disaster event. We also provide content based analysis along with temporal and geo-location based filtering to provide more accurate information to the users. To show the capabilities of the system, we demonstrate that a large number of disaster events can be detected by the system. In addition, we use crowdsourcing to demonstrate the quality of the provided information about the disasters, and usefulness of JORD from potential users point of view.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing - CBMI '17 |
Pagination | 12:1--12:6 |
Publisher | ACM Press |
Place Published | New York, USA |
ISBN Number | 9781450353335 |
URL | http://dl.acm.org/citation.cfm?doid=3095713 |
DOI | 10.1145/3095713.3095726 |
Kvasir: A Multi-Class Image-Dataset for Computer Aided Gastrointestinal Disease Detection
In ACM Multimedia Systems. Taiwan: ACM, 2017.Status: Published
Kvasir: A Multi-Class Image-Dataset for Computer Aided Gastrointestinal Disease Detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM Multimedia Systems |
Date Published | 07/2017 |
Publisher | ACM |
Place Published | Taiwan |
Multimedia for medicine: the medico Task at mediaEval 2017
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
Multimedia for medicine: the medico Task at mediaEval 2017
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
Nerthus: A Bowel Preparation Quality Video Dataset
In ACM Multimedia Systems. Taipei: ACM, 2017.Status: Published
Nerthus: A Bowel Preparation Quality Video Dataset
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM Multimedia Systems |
Publisher | ACM |
Place Published | Taipei |
Journal Article
Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge
IEEE Transactions on Medical Imaging (2017): 1-19.Status: Published
Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | IEEE Transactions on Medical Imaging |
Pagination | 1-19 |
Publisher | IEEE |
ISSN | 0278-0062 |
Keywords | Endoscopic vision, Handcrafted features, Machine learning, Polyp Detection, Validation Framework |
DOI | 10.1109/TMI.2017.2664042 |
Efficient disease detection in gastrointestinal videos – global features versus neural networks
Multimedia Tools and Applications 76, no. 21 (2017): 22493-22525.Status: Published
Efficient disease detection in gastrointestinal videos – global features versus neural networks
Analysis of medical videos from the human gastrointestinal (GI) tract for detection and localization of abnormalities like lesions and diseases requires both high precision and recall. Additionally, it is important to support efficient, real-time processing for live feedback during (i) standard colonoscopies and (ii) scalability for massive population-based screening, which we conjecture can be done using a wireless video capsule endoscope (camera-pill). Existing related work in this field does neither provide the necessary combination of accuracy and performance for detecting multiple classes of abnormalities simultaneously nor for particular disease localization tasks. In this paper, a complete end-to-end multimedia system is presented where the aim is to tackle automatic analysis of GI tract videos. The system includes an entire pipeline ranging from data collection, processing and analysis, to visualization. The system combines deep learning neural networks, information retrieval, and analysis of global and local image features in order to implement multi-class classification, detection and localization. Furthermore, it is built in a modular way, so that it can be easily extended to deal with other types of abnormalities. Simultaneously, the system is developed for efficient processing in order to provide real-time feedback to the doctors and for scalability reasons when potentially applied for massive population-based algorithmic screenings in the future. Initial experiments show that our system has multiclass detection accuracy and polyp localization precision at least as good as state-of-the-art systems, and provides additional novelty in terms of real-time performance, low resource consumption and ability to extend with support for new classes of diseases.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | Multimedia Tools and Applications |
Volume | 76 |
Issue | 21 |
Pagination | 22493–22525 |
Date Published | 11/2017 |
Publisher | ACM/Springer |
ISSN | 1380-7501 |
Keywords | Algorithmic screening, Automatic disease detection, Deep learning neural networks, Global and local image features, Information retrieval, Medical, performance evaluation |
URL | https://link.springer.com/article/10.1007%2Fs11042-017-4989-y |
DOI | 10.1007/s11042-017-4989-y |
From Annotation to Computer-Aided Diagnosis: Detailed Evaluation of a Medical Multimedia System
ACM Transactions on Multimedia Computing, Communications, and Applications 13, no. 3 (2017).Status: Published
From Annotation to Computer-Aided Diagnosis: Detailed Evaluation of a Medical Multimedia System
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | ACM Transactions on Multimedia Computing, Communications, and Applications |
Volume | 13 |
Issue | 3 |
Publisher | ACM |
Proceedings, refereed
Computer Aided Disease Detection System for Gastrointestinal Examinations
In Multimedia Systems Conference 2016. New York: ACM, 2016.Status: Published
Computer Aided Disease Detection System for Gastrointestinal Examinations
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia Systems Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Efficient Processing of Videos in a Multi Auditory Environment Using Device Lending of GPUs
In The 7th International Conference on Multimedia Systems (MMSys). ACM, 2016.Status: Published
Efficient Processing of Videos in a Multi Auditory Environment Using Device Lending of GPUs
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | The 7th International Conference on Multimedia Systems (MMSys) |
Pagination | 36:1-36:4 |
Date Published | 05/2016 |
Publisher | ACM |
ISBN Number | 978-1-4503-4297-1 |
DOI | 10.1145/2910017.2910636 |
EIR - Efficient Computer Aided Diagnosis Framework for Gastrointestinal Endoscopies
In International Workshop on Content-based Multimedia Indexing. IEEE / ACM, 2016.Status: Published
EIR - Efficient Computer Aided Diagnosis Framework for Gastrointestinal Endoscopies
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Workshop on Content-based Multimedia Indexing |
Date Published | 06/2016 |
Publisher | IEEE / ACM |
Explorative Hyperbolic-Tree-Based Clustering Tool for Unsupervised Knowledge Discovery
In International Workshop on Content-based Multimedia Indexing. IEEE / ACM, 2016.Status: Published
Explorative Hyperbolic-Tree-Based Clustering Tool for Unsupervised Knowledge Discovery
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Workshop on Content-based Multimedia Indexing |
Date Published | 06/2016 |
Publisher | IEEE / ACM |
GPU-accelerated Real-time Gastrointestinal Diseases Detection
In CBMS 2016 : The 29th International Symposium on Computer-Based Medical Systems. IEEE, 2016.Status: Published
GPU-accelerated Real-time Gastrointestinal Diseases Detection
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | CBMS 2016 : The 29th International Symposium on Computer-Based Medical Systems |
Date Published | 07/2016 |
Publisher | IEEE |
Heimdallr: a dataset for sport analysis
In ACM Multimedia System. ACM, 2016.Status: Published
Heimdallr: a dataset for sport analysis
In this paper, we present Heimdallr, a dataset that aims to serve two different purposes. The first purpose is action recognition and pose estimation, which requires a dataset of annotated sequences of athlete skeletons. We employed a crowdsourcing platform where people around the world were asked to annotate frames and obtained more than 3000 fully annotated frames for 42 different sequences with a variety of poses and actions. The second purpose is an improved understanding of crowdworkers, and for this purpose, we collected over 10000 written feedbacks from 592 crowdworkers. This is valuable information for crowdsourcing researchers who explore algorithms for worker quality assessment. In addition to the complete dataset, we also provide the code for the application that has been used to collect the data as an open source software.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ACM Multimedia System |
Date Published | 05/2016 |
Publisher | ACM |
DOI |
LIRE - Open Source Visual Information Retrieval
In Multimedia System Conference 2016. New York: ACM, 2016.Status: Published
LIRE - Open Source Visual Information Retrieval
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia System Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Multimedia and Medicine: Teammates for Better Disease Detection and Survival
In ACM Multimedia. Amsterdam, The Netherlands, The Netherlands: ACM, 2016.Status: Published
Multimedia and Medicine: Teammates for Better Disease Detection and Survival
Afilliation | Communication Systems, Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ACM Multimedia |
Date Published | 10/2016 |
Publisher | ACM |
Place Published | Amsterdam, The Netherlands, The Netherlands |
DOI | 10.1145/2964284.2976760 |
Right inflight? A dataset for exploring the automatic prediction of movies suitable for a watching situation
In Multimedia Systems Conference 2016. New York: ACM, 2016.Status: Published
Right inflight? A dataset for exploring the automatic prediction of movies suitable for a watching situation
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia Systems Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Simula @ MediaEval 2016 Context of Experience Task
In MediaEval 2016 Workshop. Hilversum, Netherlands: CEUR Workshop Proceedings, 2016.Status: Published
Simula @ MediaEval 2016 Context of Experience Task
This paper presents our approach for the Context of Multimedia Experience Task of the MediaEval 2016 Benchmark. We present different analyses of the given data using different subsets of data sources and combinations of it. Our approach gives a baseline evaluation indicating that metadata approaches work well but that also visual features can provide useful information for the given problem to solve.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | MediaEval 2016 Workshop |
Date Published | 10/2016 |
Publisher | CEUR Workshop Proceedings |
Place Published | Hilversum, Netherlands |