Publications
Journal Article
A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation
IEEE Journal of Biomedical and Health Informatics (2021).Status: Published
A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation
Afilliation | Machine Learning |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | IEEE Journal of Biomedical and Health Informatics |
Publisher | IEEE |
Keywords | colonoscopy, conditional random field, generalization, Polyp segmentation, ResUNet++, test-time augmentation |
Proceedings, refereed
DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation
In 25th International Conference on Pattern Recognition , 2021.Status: Accepted
DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation
Afilliation | Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 25th International Conference on Pattern Recognition |
Keywords | Benchmarking, Convolutional neural network, deep learning, Polyp segmentation |
Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
In 27th International Conference on Multimedia Modeling, 2021.Status: Accepted
Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
Afilliation | Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 27th International Conference on Multimedia Modeling |
Book Chapter
HTAD: A Home-Tasks Activities Dataset with Wrist-accelerometer and Audio Features
In 27th International Conference on Multimedia Modeling. Springer, 2021.Status: Accepted
HTAD: A Home-Tasks Activities Dataset with Wrist-accelerometer and Audio Features
Afilliation | Machine Learning |
Publication Type | Book Chapter |
Year of Publication | 2021 |
Book Title | 27th International Conference on Multimedia Modeling |
Publisher | Springer |
Keywords | Accelerometer, Activity recognition, Audio, dataset, Sensor fusion |
Proceedings, refereed
ACM Multimedia BioMedia 2020 Grand Challenge Overview
In Proceedings of the 28th ACM International Conference on Multimedia. New York, NY, USA: Association for Computing Machinery, 2020.Status: Published
ACM Multimedia BioMedia 2020 Grand Challenge Overview
The BioMedia 2020 ACM Multimedia Grand Challenge is the second in a series of competitions focusing on the use of multimedia for different medical use-cases. In this year's challenge, participants are asked to develop algorithms that automatically predict the quality of a given human semen sample using a combination of visual, patient-related, and laboratory-analysis-related data. Compared to last year's challenge, participants are provided with a fully multimodal dataset (videos, analysis data, study participant data) from the field of assisted human reproduction. The tasks encourage the use of the different modalities contained within the dataset and finding smart ways of how they may be combined to further improve prediction accuracy. For example, using only video data or combining video data and patient-related data. The ground truth was developed through a preliminary analysis done by medical experts following the World Health Organization's standard for semen quality assessment. The task lays the basis for automatic, real-time support systems for artificial reproduction. We hope that this challenge motivates multimedia researchers to explore more medical-related applications and use their vast knowledge to make a real impact on people's lives.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Proceedings of the 28th ACM International Conference on Multimedia |
Pagination | 4655–4658 |
Publisher | Association for Computing Machinery |
Place Published | New York, NY, USA |
ISBN Number | 9781450379885 |
Keywords | artificial intelligence, Machine learning, male fertility, semen analysis, spermatozoa |
URL | https://doi.org/10.1145/3394171.3416287 |
DOI | 10.1145/3394171.3416287 |
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation
In CBMS 2020: International Symposium on Computer-Based Medical Systems. IEEE, 2020.Status: Published
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation
Afilliation | Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | CBMS 2020: International Symposium on Computer-Based Medical Systems |
Publisher | IEEE |
Notes | This paper was nominated for the best paper award at CBMS 2020. |
ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications
In European Conference on Information Retrieval. Cham: Springer International Publishing, 2020.Status: Published
ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications
This paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum—CLEF Labs 2020 in Thessaloniki, Greece. ImageCLEF is an ongoing evaluation initiative (run since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF will organize four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data and adapted tasks, (iii) a Coral task about segmenting and labeling collections of coral images for 3D modeling, and a new (iv) Web user interface task addressing the problems of detecting and recognizing hand drawn website UIs (User Interfaces) for generating automatic code. The strong participation, with over 235 research groups registering and 63 submitting over 359 runs for the tasks in 2019 shows an important interest in this benchmarking campaign. We expect the new tasks to attract at least as many researchers for 2020.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | European Conference on Information Retrieval |
Pagination | 533 - 541 |
Publisher | Springer International Publishing |
Place Published | Cham |
ISBN Number | 978-3-030-45441-8 |
ISSN Number | 0302-9743 |
URL | https://link.springer.com/chapter/10.1007/978-3-030-45442-5_69 |
DOI | 10.1007/978-3-030-45442-5_69 |
Kvasir-SEG: A Segmented Polyp Dataset
In MMM2020. Daejeon, Korea: MMM 2020 26TH INTERNATIONAL CONFERENCE ON MULTIMEDIA MODELING, 2020.Status: Published
Kvasir-SEG: A Segmented Polyp Dataset
Pixel-wise image segmentation is a highly demanding task in medical image analysis. In practice, it is difficult to find annotated medical images with corresponding segmentation masks. In this paper, we present Kvasir-SEG: an open-access dataset of gastrointestinal polyp images and corresponding segmentation masks, manually annotated and verified by an experienced gastroenterologist. We demonstrate the use of our dataset with a traditional segmentation approach and a modern deep learning based CNN approach. This work will be valuable for researchers to reproduce results and compare their methods in the future. By adding segmentation masks to the Kvasir dataset, which until today only consisted of framewise annotations, we enable multimedia and computer vision researchers to contribute in the field of polyp segmentation and automatic analysis of colonoscopy videos.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | MMM2020 |
Publisher | MMM 2020 26TH INTERNATIONAL CONFERENCE ON MULTIMEDIA MODELING |
Place Published | Daejeon, Korea |
Keywords | Kvasir-SEG dataset, Medical images, Polyp segmentation, ResUNet Fuzzy c-mean clustering, Semantic segmentation |
LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classification
In PDCAT-PAAP2020, 2020.Status: Accepted
LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classification
Afilliation | Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | PDCAT-PAAP2020 |
Keywords | CIFAR-10, Convolutional neural network, Deep Learning, Fashion MNIST, Lightweight model, MNIST, Weight decomposition |
Medico Multimedia Task at MediaEval 2020: Automatic Polyp Segmentation
In Medico MediaEval 2020, 2020.Status: Accepted
Medico Multimedia Task at MediaEval 2020: Automatic Polyp Segmentation
Afilliation | Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | Medico MediaEval 2020 |
Organiser Team at ImageCLEFlifelog 2020: A Baseline Approach for Moment Retrieval and Athlete Performance Prediction using Lifelog Data
In CLEF2020. CEUR Workshop Proceedings, 2020.Status: Published
Organiser Team at ImageCLEFlifelog 2020: A Baseline Approach for Moment Retrieval and Athlete Performance Prediction using Lifelog Data
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | CLEF2020 |
Publisher | CEUR Workshop Proceedings |
Overview of ImageCLEF lifelog 2020: lifelog moment retrieval and sport performance lifelog
In CLEF2020 . CEUR Workshop Proceedings, 2020.Status: Published
Overview of ImageCLEF lifelog 2020: lifelog moment retrieval and sport performance lifelog
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | CLEF2020 |
Publisher | CEUR Workshop Proceedings |
Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications
Vol. 122603238842399811512840. Cham: Springer International Publishing, 2020.Status: Published
Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications
This paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum—CLEF Labs 2020 in Thessaloniki, Greece. ImageCLEF is an ongoing evaluation initiative (run since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF will organize four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data and adapted tasks, (iii) a Coral task about segmenting and labeling collections of coral images for 3D modeling, and a new (iv) Web user interface task addressing the problems of detecting and recognizing hand drawn website UIs (User Interfaces) for generating automatic code. The strong participation, with over 235 research groups registering and 63 submitting over 359 runs for the tasks in 2019 shows an important interest in this benchmarking campaign. We expect the new tasks to attract at least as many researchers for 2020.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Volume | 122603238842399811512840 |
Pagination | 311 - 341 |
Publisher | Springer International Publishing |
Place Published | Cham |
ISBN Number | 978-3-030-58218-0 |
ISSN Number | 0302-9743 |
URL | https://doi.org/10.1007/978-3-030-58219-7_22 |
DOI | 10.1007/978-3-030-58219-710.1007/978-3-030-58219-7_22 |
PMData: a sports logging dataset
In ACM MMSys 2020, 2020.Status: Published
PMData: a sports logging dataset
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ACM MMSys 2020 |
PSYKOSE: A Motor Activity Database of Patients with Schizophrenia
In 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS)2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS). Rochester, MN, USA: IEEE, 2020.Status: Published
PSYKOSE: A Motor Activity Database of Patients with Schizophrenia
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS)2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) |
Publisher | IEEE |
Place Published | Rochester, MN, USA |
URL | https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=9169740http... |
DOI | 10.1109/CBMS49503.202010.1109/CBMS49503.2020.00064 |
Pyramid-Focus-Augmentation: Medical Image Segmentation with Step-Wise Focus
In MediaEval 2020, 2020.Status: Published
Pyramid-Focus-Augmentation: Medical Image Segmentation with Step-Wise Focus
Segmentation of findings in the gastrointestinal tract is a challenging but also an important task which is an important building stone for sufficient automatic decision support systems. In this work, we present our solution for the Medico 2020 task, which focused on the problem of colon polyp segmentation. We present our simple but efficient idea of using an augmentation method that uses grids in a pyramid-like manner (large to small) for segmentation. Our results show that the proposed methods work as indented and can also lead to comparable results when competing with other methods.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | MediaEval 2020 |
Real-Time Detection of Events in Soccer Videos using 3D Convolutional Neural Networks
In ISM. IEEE, 2020.Status: Published
Real-Time Detection of Events in Soccer Videos using 3D Convolutional Neural Networks
In this paper, we present an algorithm for automatically detecting events in soccer videos using 3D convolutional neural networks. The algorithm uses a sliding window approach to scan over a given video to detect events such as goals, yellow/red cards, and player substitutions. We test the method on three different datasets from SoccerNet, the Swedish Allsvenskan, and the Norwegian Eliteserien. Overall, the results show that we can detect events with high recall, low latency, and accurate time estimation. The trade-off is a slightly lower precision compared to the current state-of-the-art, which has higher latency and performs better when a less accurate time estimation can be accepted. In addition to the presented algorithm, we perform an extensive ablation study on how the different parts of the training pipeline affect the final results.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ISM |
Publisher | IEEE |
DOI | 10.1109/ISM.2020.00030 |
Scalable Infrastructure for Efficient Real-Time Sports Analytics
In ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTIONCompanion Publication of the 2020 International Conference on Multimodal Interaction. Virtual Event NetherlandsNew York, NY, USA: ACM, 2020.Status: Published
Scalable Infrastructure for Efficient Real-Time Sports Analytics
Recent technological advances are adapted in sports to improve performance, avoid injuries, and make advantageous decisions. In this paper, we describe our ongoing efforts to develop and deploy PMSys, our smartphone-based athlete monitoring and reporting system. We describe our first attempts to gain insight into some of the data we have collected. Experiences so far are promising, both on the technical side and for athlete performance development. Our initial application of artificial-intelligence methods for prediction is encouraging and indicative.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTIONCompanion Publication of the 2020 International Conference on Multimodal Interaction |
Publisher | ACM |
Place Published | Virtual Event NetherlandsNew York, NY, USA |
ISBN Number | 9781450380027 |
Keywords | algorithmic analysis, artificial intelligence, Machine learning, privacy-preserving data collection, Sports performance logging |
URL | https://dl.acm.org/doi/proceedings/10.1145/3395035https://dl.acm.org/doi... |
DOI | 10.1145/339503510.1145/3395035.3425300 |
The EndoTect 2020 Challenge: Evaluation and Comparison of Classification, Segmentation and Inference Time for Endoscopy
In ICPR, 2020.Status: Accepted
The EndoTect 2020 Challenge: Evaluation and Comparison of Classification, Segmentation and Inference Time for Endoscopy
The EndoTect challenge at the International Conference on Pattern Recognition 2020 aims to motivate the development of algorithms that aid medical experts in finding anomalies that commonly occur in the gastrointestinal tract. Using HyperKvasir, a large dataset containing images taken from several endoscopies, the participants competed in three tasks. Each task focuses on a specific requirement for making it useful in a real-world medical scenario. The tasks are (i) high classification performance in terms of prediction accuracy, (ii) efficient classification measured by the number of images classified per second, and (iii) pixel-level segmentation of specific anomalies. Hopefully, this can motivate different computer science researchers to help benchmark a crucial component of a future computer-aided diagnosis system, which in turn, could potentially save human lives.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ICPR |
Toadstool: A Dataset for Training Emotional IntelligentMachines Playing Super Mario Bros
In The ACM Multimedia Systems Conference (MMSys). The ACM Multimedia Systems Conference (MMSys): ACM, 2020.Status: Published
Toadstool: A Dataset for Training Emotional IntelligentMachines Playing Super Mario Bros
Games are often defined as engines of experience, and they are heavily relying on emotions, they arouse in players. In this paper, we present a dataset called Toadstool as well as a reproducible methodology to extend on the dataset. The dataset consists of video, sensor, and demographic data collected from ten participants playing Super Mario Bros, an iconic and famous video game. The sensor data is collected through an Empatica E4 wristband, which provides high-quality measurements and is graded as a medical device. In addition to the dataset and the methodology for data collection, we present a set of baseline experiments which show that we can use video game frames together with the facial expressions to predict the blood volume pulse of the person playing Super Mario Bros. With the dataset and the collection methodology we aim to contribute to research on emotionally aware machine learning algorithms, focusing on reinforcement learning and multimodal data fusion. We believe that the presented dataset can be interesting for a manifold of researchers to explore exciting new interdisciplinary questions.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The ACM Multimedia Systems Conference (MMSys) |
Publisher | ACM |
Place Published | The ACM Multimedia Systems Conference (MMSys) |
URL | https://dl.acm.org/doi/10.1145/3339825.3394939 |
DOI | 10.1145/3339825.3394939 |
Vid2Pix - A Framework for Generating High-Quality Synthetic Videos
In IEEE ISM, 2020.Status: Published
Vid2Pix - A Framework for Generating High-Quality Synthetic Videos
Data is arguably the most important resource today as it fuels the algorithms powering services we use every day. However, in fields like medicine, publicly available datasets are few, and labeling medical datasets require tedious efforts from trained specialists. Generated synthetic data can be to future successful healthcare clinical intelligence. Here, we present a GAN-based video generator demonstrating promising results.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | IEEE ISM |
Journal Article
An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification
ACM Transactions on Computing for Healthcare (2020).Status: Published
An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification
Afilliation | Machine Learning |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | ACM Transactions on Computing for Healthcare |
Publisher | ACM Transactions on Computing for Healthcare |
Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge
Medical Image Analysis (2020).Status: Published
Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge
Afilliation | Machine Learning |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Medical Image Analysis |
Date Published | 11/2020 |
Publisher | Elsevier |
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Scientific Data (2020).Status: Published
HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy
Artificial intelligence is currently a hot topic in medicine. However, medical data is often sparse and hard to obtain due to legal restrictions and lack of medical personnel for the cumbersome and tedious process to manually label training data. These constraints make it difficult to develop systems for automatic analysis, like detecting disease or other lesions. In this respect, this article presents HyperKvasir, the largest image and video dataset of the gastrointestinal tract available today. The data is collected during real gastro- and colonoscopy examinations at Bærum Hospital in Norway and partly labeled by experienced gastrointestinal endoscopists. The dataset contains 110,079 images and 374 videos, and represents anatomical landmarks as well as pathological and normal findings. The total number of images and video frames together is around 1 million. Initial experiments demonstrate the potential benefits of artificial intelligence-based computer-assisted diagnosis systems. The HyperKvasir dataset can play a valuable role in developing better algorithms and computer-assisted examination systems not only for gastro- and colonoscopy, but also for other fields in medicine.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Scientific Data |
Date Published | 08/2020 |
Publisher | Springer Nature |
Keywords | dataset, GI, Machine learning |
URL | http://www.nature.com/articles/s41597-020-00622-y |
DOI | 10.1038/s41597-020-00622-y |
Technological and Clinical Challenges in Lead Placement for Cardiac Rhythm Management Devices
Annals of Biomedical Engineering 48 (2020): 26-46.Status: Published
Technological and Clinical Challenges in Lead Placement for Cardiac Rhythm Management Devices
Afilliation | Scientific Computing |
Project(s) | Department of Computational Physiology |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Annals of Biomedical Engineering |
Volume | 48 |
Pagination | 26 - 46 |
Date Published | 01/2020 |
Publisher | Springer Link |
ISSN | 0090-6964 |
DOI | 10.1007/s10439-019-02376-0 |
Proceedings, refereed
A Web-Based Software for Training and Quality Assessment in the Image Analysis Workflow for Cardiac T1 Mapping MRI
In 2019 IEEE International Symposium on Multimedia (ISM). IEEE, 2019.Status: Published
A Web-Based Software for Training and Quality Assessment in the Image Analysis Workflow for Cardiac T1 Mapping MRI
Medical practice makes significant use of imaging scans such as Ultrasound or MRI as a diagnostic tool. They are used in the visual inspection or quantification of medical parameters computed from the images in post-processing. However, the value of such parameters depends much on the user's variability, device, and algorithmic differences. In this paper, we focus on quantifying the variability due to the human factor, which can be primarily addressed by the structured training of a human operator. We focus on a specific emerging cardiovascular \gls{mri} methodology, the T1 mapping, that has proven useful to identify a range of pathological alterations of the myocardial tissue structure. Training, especially in emerging techniques, is typically not standardized, varying dramatically across medical centers and research teams. Additionally, training assessment is mostly based on qualitative approaches. Our work aims to provide a software tool combining traditional clinical metrics and convolutional neural networks to aid the training process by gathering contours from multiple trainees, quantifying discrepancy from local gold standard or standardized guidelines, classifying trainees output based on critical parameters that affect contours variability.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 IEEE International Symposium on Multimedia (ISM) |
Publisher | IEEE |
DOI | 10.1109/ISM46123.2019.00047 |
ACM Multimedia BioMedia 2019 Grand Challenge Overview
In The ACM International Conference on Multimedia (ACM MM). New York, New York, USA: ACM Press, 2019.Status: Published
ACM Multimedia BioMedia 2019 Grand Challenge Overview
The BioMedia 2019 ACM Multimedia Grand Challenge is the first in a series of competitions focusing on the use of multimedia for different medical use-cases. In this year’s challenge, the participants are asked to develop efficient algorithms which automatically detect a variety of findings commonly identified in the gastrointestinal (GI) tract (a part of the human digestive system). The purpose of this task is to develop methods to aid medical doctors performing routine endoscopy inspections of the GI tract. In this paper, we give a detailed description of the four different tasks of this year’s challenge, present the datasets used for training and testing, and discuss how each submission is evaluated both qualitatively and quantitatively.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | The ACM International Conference on Multimedia (ACM MM) |
Pagination | 2563-2567 |
Date Published | 10/2019 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450368896 |
URL | http://dl.acm.org/citation.cfm?doid=3343031http://dl.acm.org/citation.cf... |
DOI | 10.1145/334303110.1145/3343031.3356058 |
Automatic Hyperparameter Optimization for Transfer Learning on Medical Image Datasets Using Bayesian Optimization
In 13th International Symposium on Medical Information and Communication Technology (ISMICT). IEEE, 2019.Status: Published
Automatic Hyperparameter Optimization for Transfer Learning on Medical Image Datasets Using Bayesian Optimization
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 13th International Symposium on Medical Information and Communication Technology (ISMICT) |
Pagination | 1-6 |
Publisher | IEEE |
DOI | 10.1109/ISMICT.2019.8743779 |
Extracting temporal features into a spatial domain using autoencoders for sperm video analysis
In MediaEval 2019. CEUR Workshop Proceedings, 2019.Status: Published
Extracting temporal features into a spatial domain using autoencoders for sperm video analysis
In this paper, we present a two-step deep learning method that is used to predict sperm motility and morphology based on video recordings of human spermatozoa. First, we use an autoencoder to extract temporal features from a given semen video and plot these into image-space, which we call feature-images. Second, these feature-images are used to perform transfer learning to predict the motility and morphology values of human sperm. The presented method shows it's capability to extract temporal information into spatial domain feature-images which can be used with traditional convolutional neural networks. Furthermore, the accuracy of the predicted motility of a given semen sample shows that a deep learning-based model can capture the temporal information of microscopic recordings of human semen.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval 2019 |
Date Published | 10/2019 |
Publisher | CEUR Workshop Proceedings |
GameStory Task at MediaEval 2019
In Proceedings of MediaEval 2019. CEUR Workshop Proceedings (CEUR-WS.org), 2019.Status: Published
GameStory Task at MediaEval 2019
Game video streams are watched by millions, so that, meanwhile, one can make a living from broadcasting and commenting video games, whereas some have become professional e-sports athletes. E-sports leagues and tournaments have emerged worldwide, where players compete in controlled environments, streaming the matches online, and allowing the audience to discuss and criticize the game- play. In the GameStory task, held for the second time at MediaEval, we foster research into this exciting domain. Our focus is on an- alyzing and summarizing video game streams. With the help of ZNIPE.tv, we compiled a high-quality dataset of a Counter-Strike: Global Offensive tournament alongside ground truth labels for two analysis tasks, forming a basis for summarization.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of MediaEval 2019 |
Date Published | 10/2019 |
Publisher | CEUR Workshop Proceedings (CEUR-WS.org) |
GANEx: A complete pipeline of training, inference and benchmarking GAN experiments
In 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 2019.Status: Published
GANEx: A complete pipeline of training, inference and benchmarking GAN experiments
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 International Conference on Content-Based Multimedia Indexing (CBMI) |
Pagination | 1-4 |
Publisher | IEEE |
Keywords | GAN, GANEx, Generative Adversarial Network |
Medical Multimedia Systems and Applications
In Proceedings of the 27th ACM International Conference on Multimedia - MM '19. New York, NY, USA: ACM Press, 2019.Status: Published
Medical Multimedia Systems and Applications
In recent years, we have observed a rise of interest in the multimedia community towards research topics related to health. It can be observed that this goes into two interesting directions. One is personal health with a larger focus on well-being and everyday healthy living. The other direction focuses more on multimedia challenges within the health-care systems, for example, how can multimedia content produced in hospitals be used efficiently but also on the user perspective of patients and health-care personal. Challenges and requirements in this interesting and challenging direction are similar to classic multimedia research, but with some additional pitfalls and challenges. This tutorial aims to give a general introduction to the research area; to provide an overview of specific requirements, pitfalls and challenges; to discuss existing and possible future work; and to elaborate on how machine learning approaches can help in multimedia-related challenges to improve the health-care quality for patients and support medical experts in their daily work.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the 27th ACM International Conference on Multimedia - MM '19 |
Pagination | 2711-2713 |
Date Published | 1072019 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450368896 |
URL | http://dl.acm.org/citation.cfm?doid=3343031http://dl.acm.org/citation.cf... |
DOI | 10.1145/3343031.3351319 |
Medico Multimedia Task at MediaEval 2019
In MediaEval. CEUR Workshop Proceedings, 2019.Status: Published
Medico Multimedia Task at MediaEval 2019
The Medico: Multimedia for Medicine Task is running for the third time as part of MediaEval 2019. This year, we have changed the task from anomaly detection in images of the gastrointestinal tract to focus on the automatic prediction of human semen quality based on videos. The purpose of this task is to aid in the assessment of male reproductive health by providing a quick and consisted method of analyzing human semen. In this paper, we describe the task in detail, give a brief description of the provided dataset, and discuss the evaluation process and the metrics used to rank the submissions of the participants.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Publisher | CEUR Workshop Proceedings |
One-Dimensional Convolutional Neural Networks on Motor Activity Measurements in Detection of Depression
In Proceedings of the 4th International Workshop on Multimedia for Personal Health & Health Care - HealthMedia '19. New York, NY, USA: ACM Press, 2019.Status: Published
One-Dimensional Convolutional Neural Networks on Motor Activity Measurements in Detection of Depression
Nowadays, it has become possible to measure different human activities using wearable devices. Besides measuring the number of daily steps or calories burned, these datasets have much more potential since different activity levels are also collected. Such data would be helpful in the field of psychology because it can relate to various mental health issues such as changes in mood and stress. In this paper, we present a machine learning approach to detect depression using a dataset with motor activity recordings of one group of people with depression and one group without, i.e., the condition group includes 23 unipolar and bipolar persons, and the control group includes 32 persons without depression. We use convolutional neural networks to classify the depressed and nondepressed patients. Moreover, different levels of depression were classified. Finally, we trained a model that predicts MontgomeryÅsberg Depression Rating Scale scores. We achieved an average F1-score of 0.70 for detecting the control and condition groups. The mean squared error for score prediction was approximately 4.0.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the 4th International Workshop on Multimedia for Personal Health & Health Care - HealthMedia '19 |
Pagination | 9-15 |
Date Published | 10/2019 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450369145 |
URL | http://dl.acm.org/citation.cfm?doid=3347444http://dl.acm.org/citation.cf... |
DOI | 10.1145/334744410.1145/3347444.3356238 |
Performance of Data Enhancements and Training Optimization for Neural Network – A Polyp Detection Case Study
In IEEE CBMS International Symposium on Computer-Based Medical Systems (CBMS). IEEE, 2019.Status: Published
Performance of Data Enhancements and Training Optimization for Neural Network – A Polyp Detection Case Study
Deep learning using neural networks is becoming more and more popular. It is frequently used in areas like video analysis, image retrieval, traffic forecast and speech recognition. In this respect, the learning and training process usually requires a lot of data. However, in many areas, data is scarce which is definitely the case in our medical application scenario, i.e., polyp detection in the gastrointestinal tract. Here, colorectal cancer is on the list of most common cancer types, and often, the cancer arises from benign, adenomatous polyps containing dysplastic cells. Detection and removal of polyps can therefore prevent the development of cancer. Due to high cost, time consumption, patient discomfort and in-accuracy of existing procedures, researchers have started to explore systems for automatic polyp detection to assist and automate current examination procedures. Following the current gained traction for neural networks, and the typical lack of medical data, we explore how data enhancements affect the training and evaluation of the networks in terms of polyp detection accuracy and particularly if it can be used to increase the detection rate. We also experiment with how various training techniques can be used to increase performance. Our experimental results show how data enhancement and training optimization can be used to increase different aspects of the performance, but we also point out mechanisms that have no and even a negative effect.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | IEEE CBMS International Symposium on Computer-Based Medical Systems (CBMS) |
Publisher | IEEE |
Predicting Peek Readiness-to-Train of Soccer Players Using Long Short-Term Memory Recurrent Neural Networks
In 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 2019.Status: Published
Predicting Peek Readiness-to-Train of Soccer Players Using Long Short-Term Memory Recurrent Neural Networks
We are witnessing the emergence of a myriad of hardware and software systems that quantifies sport and physical activities. These are frequently touted as game changers and important for future sport developments. The vast amount of generated data is often visualized in graphs and dashboards, for use by coaches and other sports professionals to make decisions on training and match strategies. Modern machine- learning methods have the potential to further fuel this process by deriving useful insights that are not easily observable in the raw data streams.
This paper tackles the problem of deriving peaks in soccer players’ ability to perform from subjective self-reported wellness data collected using the PMSys system. For this, we train a long short-term memory recurrent neural network model using data from two professional Norwegian soccer teams. We show that our model can predict performance peaks in most scenarios with a precision and recall of at least 90%. Equipped with such insight, coaches and trainers can better plan individual and team training sessions, and perhaps avoid over training and injuries.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 International Conference on Content-Based Multimedia Indexing (CBMI) |
Publisher | IEEE |
DOI | 10.1109/CBMI.2019.8877406 |
Predicting Sperm Motility and Morphology using Deep Learning and Handcrafted Features
In MediaEval, 2019.Status: Accepted
Predicting Sperm Motility and Morphology using Deep Learning and Handcrafted Features
This paper presents the approach proposed by the organizer team (SimulaMet) for MediaEval 2019 Multimedia for Medicine: The Medico Task. The approach uses a data preparation method which is based on global features extracted from multiple frames within each video and then combines this with information about the patient in order to create a compressed representation of each video. The goal is to create a less hardware expensive data representation that still retains the temporal information of the video and related patient data. Overall, the results need some improvement before being a viable option for clinical use.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Real-time Analysis of Physical Performance Parameters in Elite Soccer
In 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 2019.Status: Published
Real-time Analysis of Physical Performance Parameters in Elite Soccer
Technology is having vast impact on the sports in- dustry, and in particular soccer. All over the world, soccer teams are adapting digital information systems to quantify performance metrics. The goal is to assess strengths and weaknesses of indi- vidual players, training regimes, and play strategies; to improve performance and win games. However, most existing methods rely on post-game analytic. This limits coaches to review games in retrospect without any means to do changes during sessions. In collaboration with an elite soccer club, we have developed Metrix which is a computerized toolkit for coaches to perform real- time monitoring and analysis of the players’ performance. Using sensor technology to track movement, performance parameters are instantly available to coaches through a mobile phone client. Metrix provides coaches with a toolkit to individualize training load to different playing positions on the field, or to the player himself. Our results show that Metrix is able to quantify player performance and propagate it to coaches in real-time during a match or practice, i.e., latency is below 100 ms on the field. In our initial user evaluation, the coaches express that this is a valuable asset in day-to-day work.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 International Conference on Content-Based Multimedia Indexing (CBMI) |
Publisher | IEEE |
DOI | 10.1109/CBMI.2019.8877422 |
ResUNet++: An Advanced Architecture for Medical Image Segmentation
In 2019 IEEE International Symposium on Multimedia (ISM). San Diego, California, USA: IEEE, 2019.Status: Published
ResUNet++: An Advanced Architecture for Medical Image Segmentation
Accurate computer-aided polyp detection and segmentation during colonoscopy examinations can help endoscopists resect abnormal tissue and thereby decrease chances of polyps growing into cancer. Towards developing a fully automated model for pixel-wise polyp segmentation, we propose ResUNet++, which is an improved ResUNet architecture for colonoscopic image segmentation. Our experimental evaluations show that the suggested architecture produces good segmentation results on publicly available datasets. Furthermore, ResUNet++ significantly outperforms U-Net and ResUNet, two key state-of-the-art deep learning architectures, by achieving high evaluation scores with a dice coefficient of 81.33%, and a mean Intersection over Union (mIoU) of 79.27% for the Kvasir-SEG dataset and a dice coefficient of 79.55%, and a mIoU of 79.62% with CVC-612 dataset.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 IEEE International Symposium on Multimedia (ISM) |
Publisher | IEEE |
Place Published | San Diego, California, USA |
Keywords | colonoscopy, deep learning, health informatics, Medical image segmentation, Polyp segmentation, Semantic segmentation |
Saga: An Open Source Platform for Training Machine Learning Models and Community-driven Sharing of Techniques
In International Conference on Content-Based Multimedia Indexing (CBMI 2019). IEEE, 2019.Status: Published
Saga: An Open Source Platform for Training Machine Learning Models and Community-driven Sharing of Techniques
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | International Conference on Content-Based Multimedia Indexing (CBMI 2019) |
Pagination | 1-4 |
Publisher | IEEE |
DOI | 10.1109/CBMI.2019.8877455 |
Semantic Analysis of Soccer News for Automatic Game Event Classification
In 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 2019.Status: Published
Semantic Analysis of Soccer News for Automatic Game Event Classification
We are today overwhelmed with information, of which an important part is news. Sports news, in particular, has become very popular, where soccer makes up a big part of this coverage. For sports fans, it can be a time consuming and tedious to keep up with the news that they really care about. In this paper, we present different machine learning methods applied to soccer news from a Norwegian newspaper and a TV station's news site to summarize the content in a short and digestible manner. We present a system to collect, index, label, analyze, and present the collected news articles based on the content. We perform a thorough comparison between deep learning and traditional machine learning algorithms on text classification. Furthermore, we present a dataset of soccer news which was collected from two different Norwegian news sites and shared online.
Afilliation | Machine Learning |
Project(s) | Simula Metropolitan Center for Digital Engineering, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 International Conference on Content-Based Multimedia Indexing (CBMI) |
Publisher | IEEE |
Stacked dense optical flows and dropout layers to predict sperm motility and morphology
In MediaEval 2019, 27-29 October 2019, Sophia Antipolis, France, 2019.Status: Published
Stacked dense optical flows and dropout layers to predict sperm motility and morphology
In this paper, we analyse two deep learning methods to predict sperm motility and sperm morphology from sperm videos. We use two different inputs: stacked pure frames of videos and dense optical flows of video frames. To solve this regression task of predicting motility and morphology, stacked dense optical flows and extracted original frames from sperm videos were used with the modified state of the art convolution neural networks. For modifications of the selected models, we have introduced an additional multi-layer perceptron to overcome the problem of over-fitting. The method which had an additional multi-layer perceptron with dropout layers, shows the best results when the inputs consist of both dense optical flows and an original frame of videos.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval 2019, 27-29 October 2019, Sophia Antipolis, France |
Date Published | 10/2019 |
Summarizing E-Sports Matches and Tournaments: The Example of Counter-Strike: Global Offensive
In International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE). ACM, 2019.Status: Published
Summarizing E-Sports Matches and Tournaments: The Example of Counter-Strike: Global Offensive
That video and computer games have reached the masses is a well known fact. Furthermore, game streaming and watching other people play video games is another phenomenon that has outgrown its small beginning by far, and game streams, be it live or recorded, are today viewed by millions. E-sports is the result of organized leagues and tournaments in which players can compete in controlled environments and viewers can experience the matches, discuss and criticize, just like in physical sports. However, as traditional sports, e-sports matches may be long and contain less interesting parts, introducing the challenge of producing well directed summaries and highlights. In this paper, we describe our efforts to approach the game streaming and e-sports phenomena from a multimedia research point of view. We focus on the challenge of summarizing matches from specific relevant game, Counter-Strike: Global Offensive (CS:GO). We survey related work, describe the rules and structure of the game and identify the main challenges for summarizing e-sports matches. With this contribution, we aim to foster multimedia research in the area of e-sports and game streaming.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE) |
Publisher | ACM |
Unsupervised Preprocessing to Improve Generalisation for Medical Image Classification
In IEEE 13th International Symposium on Medical Information and Communication Technology (ISMICT). IEEE, 2019.Status: Published
Unsupervised Preprocessing to Improve Generalisation for Medical Image Classification
Automated disease detection in videos and images from the gastrointestinal (GI) tract has received much attention in the last years. However, the quality of image data is often reduced due to overlays of text and positional data.
In this paper, we present different methods of preprocessing such images and we describe our approach to GI disease classification for the Kvasir v2 dataset.
We propose multiple approaches to inpaint problematic areas in the images to improve the anomaly classification, and we discuss the effect that such preprocessing does to the input data.
In short, our experiments show that the proposed methods improve the Matthews correlation coefficient by approximately 7% in terms of better classification of GI anomalies.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems, Simula Metropolitan Center for Digital Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | IEEE 13th International Symposium on Medical Information and Communication Technology (ISMICT) |
Publisher | IEEE |
DOI | 10.1109/ISMICT.2019.8743979 |
Using 2D and 3D Convolutional Neural Networks to Predict Semen Quality
In MediaEval. CEUR Workshop Proceedings, 2019.Status: Published
Using 2D and 3D Convolutional Neural Networks to Predict Semen Quality
In this paper, we present the approach of team Jmag to solve this year's Medico Multimedia Task as part of the MediaEval 2019 Benchmark. This year, the task focuses on automatically determining quality characteristics of human sperm through the analysis of microscopic videos of human semen and associated patient data. Our approach is based on deep convolutional neural networks (CNNs) of varying sizes and dimensions. Here, we aim to analyze both the spatial and temporal information present in the videos. The results show that the method holds promise for predicting the motility of sperm, but predicting morphology appears to be more difficult.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval |
Publisher | CEUR Workshop Proceedings |
Using Deep Learning to Predict Motility and Morphology of Human Sperm
In MediaEval 2019. CEUR Workshop Proceedings, 2019.Status: Published
Using Deep Learning to Predict Motility and Morphology of Human Sperm
In the Medico Task 2019, the main focus is to predict sperm quality based on videos and other related data. In this paper, we present the approach of team LesCats which is based on deep convolution neural networks, where we experiment with different data preprocessing methods to predict the morphology and motility of human sperm. The achieved results show that deep learning is a promising method for human sperm analysis. Out best method achieves a mean absolute error of 8.962 for the motility task and a mean absolute error of 5.303 for the morphology task.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MediaEval 2019 |
Publisher | CEUR Workshop Proceedings |
VISEM: a multimodal video dataset of human spermatozoa
In Proceedings of the 10th ACM Multimedia Systems Conference. ACM, 2019.Status: Published
VISEM: a multimodal video dataset of human spermatozoa
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the 10th ACM Multimedia Systems Conference |
Pagination | 261–266 |
Publisher | ACM |
Journal Article
Automatic detection of passable roads after floods in remote sensed and social media data
Signal Processing: Image Communication 74 (2019): 110-118.Status: Published
Automatic detection of passable roads after floods in remote sensed and social media data
This paper addresses the problem of floods classification and floods aftermath detection based onboth social media and satellite imagery. Automatic detection of disasters such as floods is still a very challenging task. The focus lies on identifying passable routes or roads during floods. Two novel solutions are presented, which were developed for two corresponding tasks at the MediaEval 2018 benchmarking challenge. The tasks are (i) identification of images providing evidence for road passability and (ii) differentiation and detection of passable and non-passable roads in images from two complementary sources of information. For the first challenge, we mainly rely on object and scene-level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are then combined using early, late and double fusion techniques. To identify whether or not it is possible for a vehicle to pass a road in satellite images, we rely on Convolutional Neural Networks and a transfer learning-based classification approach. The evaluation of the proposed methods is carried out on the large-scale datasets provided for the benchmark competition. The results demonstrate significant improvement in the performance over the recent state-of-art approaches.
Afilliation | Communication Systems |
Project(s) | UMOD: Understanding and Monitoring Digital Wildfires, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Signal Processing: Image Communication |
Volume | 74 |
Pagination | 110-118 |
Publisher | Elsevier |
Keywords | convolutional neural networks, Flood detection, Multimedia Indexing and Retrieval, Natural Disasters, Satellite Imagery, Social Media |
DOI | 10.1016/j.image.2019.02.002 |
Bleeding detection in wireless capsule endoscopy videos—Color versus texture features
Journal of applied clinical medical physics 20, no. 8 (2019): 141-154.Status: Published
Bleeding detection in wireless capsule endoscopy videos—Color versus texture features
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Journal of applied clinical medical physics |
Volume | 20 |
Issue | 8 |
Pagination | 141-154 |
Publisher | Wiley Online Library |
Deep Learning for Automatic Generation of Endoscopy Reports
Gastrointestinal Endoscopy 89, no. 6 (2019).Status: Published
Deep Learning for Automatic Generation of Endoscopy Reports
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Gastrointestinal Endoscopy |
Volume | 89 |
Issue | 6 |
Date Published | 06/2019 |
Publisher | Elsevier |
Place Published | Gastrointestinal Endoscopy |
DOI | 10.1016/j.gie.2019.04.053 |
Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming
International Journal of Semantic Computing 13, no. 3 (2019): 367-391.Status: Published
Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming
360 panorama video displayed through Virtual reality (VR) glasses or large screens o®ers immersive user experiences, but as such technology becomes commonplace, the need for e±cient streaming methods of such high-bitrate videos arises. In this respect, the attention that 360panorama video has received lately is huge. Many methods have already been proposed, and in this paper, we shed more light on the di®erent trade-o®s in order to save bandwidth while preserving the video quality in the user's ̄eld-of-view (FoV). Using 360 VR content delivered to a Gear VR head-mounted display with a Samsung Galaxy S7 and to a Huawei Q22 set-top- box, we have tested various tiling schemes analyzing the tile layout, the tiling and encoding overheads, mechanisms for faster quality switching beyond the DASH segment boundaries and quality selection con ̄gurations. In this paper, we present an e±cient end-to-end design and real-world implementation of such a 360 streaming system. Furthermore, in addition to researching an on-demand system, we also go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-o® between bandwidth usage and the video quality in the user's FoV. We have created an architecture that combines RTP and DASH, and our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-o®s through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the FoV changes.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | International Journal of Semantic Computing |
Volume | 13 |
Issue | 3 |
Number | 3 |
Pagination | 367-391 |
Publisher | World Scientific |
Flexible device compositions and dynamic resource sharing in PCIe interconnected clusters using Device Lending
Cluster Computing 22, no. 86 (2019): 1-24.Status: Published
Flexible device compositions and dynamic resource sharing in PCIe interconnected clusters using Device Lending
Modern workloads often exceed the processing and I/O capabilities provided by resource virtualization, requiring direct access to the physical hardware in order to reduce latency and computing overhead. For computers interconnected in a cluster, access to remote hardware resources often requires facilitation both in hardware and specialized drivers with virtualization support. This limits the availability of resources to specific devices and drivers that are supported by the virtualization technology being used, as well as what the interconnection technology supports. For PCI Express (PCIe) clusters, we have previously proposed Device Lending as a solution for enabling direct low latency access to remote devices. The method has extremely low computing overhead and does not require any application- or device-specific distribution mechanisms. Any PCIe device, such as network cards disks, and GPUs, can easily be shared among the connected hosts. In this work, we have extended our solution with support for a virtual machine (VM) hypervisor. Physical remote devices can be “passed through” to VM guests, enabling direct access to physical resources while still retaining the flexibility of virtualization. Additionally, we have also implemented multi-device support, enabling shortest-path peer-to-peer transfers between remote devices residing in different hosts. Our experimental results prove that multiple remote devices can be used, achieving bandwidth and latency close to native PCIe, and without requiring any additional support in device drivers. I/O intensive workloads run seamlessly using both local and remote resources. With our added VM and multi-device support, Device Lending offers highly customizable configurations of remote devices that can be dynamically reassigned and shared to optimize resource utilization, thus enabling a flexible composable I/O infrastructure for VMs as well as bare-metal machines.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization, LADIO: Live Action Data Input/Output, Department of Holistic Systems, Department of High Performance Computing |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Cluster Computing |
Volume | 22 |
Issue | 86 |
Pagination | 1-24 |
Date Published | 09/2019 |
Publisher | Springer |
ISSN | 1573-7543 |
URL | https://link.springer.com/article/10.1007/s10586-019-02988-0 |
DOI | 10.1007/s10586-019-02988-0 |
Machine Learning-Based Analysis of Sperm Videos and Participant Data for Male Fertility Prediction
Nature Scientific Reports 9, no. 1 (2019).Status: Published
Machine Learning-Based Analysis of Sperm Videos and Participant Data for Male Fertility Prediction
Methods for automatic analysis of clinical data are usually targeted towards a specific modality and do not make use of all relevant data available. In the field of male human reproduction, clinical and biological data
are not used to its fullest potential. Manual evaluation of a semen sample using a microscope is time-consuming and requires extensive training. Furthermore, the validity of manual semen analysis has been questioned due to limited reproducibility, and often high inter-personnel variation. The existing computer-aided sperm analyzer systems are not recommended for routine clinical use due to methodological challenges caused by the consistency of the semen sample. Thus, there is a need for an improved methodology.
We use modern and classical machine learning techniques together with a dataset consisting of 85 videos of human semen samples and related participant data to automatically predict sperm motility. Used techniques include simple linear regression and more sophisticated methods using convolutional neural networks. Our results indicate that sperm motility prediction based on deep learning using sperm motility videos is rapid to perform and consistent. The algorithms performed worse when participant data was added. In conclusion, machine learning-based automatic analysis may become a valuable tool in male infertility investigation and research.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Nature Scientific Reports |
Volume | 9 |
Issue | 1 |
Number | 16770 |
Publisher | Springer Nature |
Maskinlæringssystemer for gastrointestinale endoskopier
Best Practice Nordic - Gastroenterologi (2019).Status: Published
Maskinlæringssystemer for gastrointestinale endoskopier
Assistert diagnostikk med hjelp av kunstig intelligens (KI) har vært etterspurt lenge og kan bli et viktig hjelpemiddel innen medisin, godt hjulpet av den raske utviklingen innen maskinvare. Denne har gjort innføringen av slike hjelpemidler mulig på relativt kort sikt. Sikrere påvisning og klassifisering av funn og lesjoner innen radiologi og endoskopi er i ferd med å bli et viktig forskningsområde innen KI, og det fokuseres spesielt på maskinlæring. Imidlertid krever vellykket utvikling et komplett system som kan brukes i sanntid i daglig praksis, og som begrenser seg til utvikling av algoritmer. Det kreves også store randomiserte studier for å fastslå om kvaliteten og påliteligheten til systemene er god. Vi deler i denne artikkelen våre erfaringer fra utviklingen av et system for gastrointestinale endoskopier og belyser viktige utfordringer for å skape en effektiv digital assistent.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Best Practice Nordic - Gastroenterologi |
Date Published | 07/2019 |
Publisher | BestPracticeNordic |
URL | https://bestprac.no/maskinlaeringssystemer-for-gastrointestinale-endosko... |
Social media and satellites: Disaster event detection, linking and summarization
Multimedia Tools and Applications 78 (2019): 2837-2875.Status: Published
Social media and satellites: Disaster event detection, linking and summarization
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Multimedia Tools and Applications |
Volume | 78 |
Number | 3 |
Pagination | 2837–2875 |
Publisher | Springer |
Place Published | Netherlands |
Book Chapter
Challenges for Multimedia Research in E-Sports Using Counter-Strike Global Offensive as an Example
In Savegame, 197-206. Vol. 4. Wiesbaden: Springer Fachmedien Wiesbaden, 2019.Status: Published
Challenges for Multimedia Research in E-Sports Using Counter-Strike Global Offensive as an Example
That video and computer games have reached the masses is a well-known fact. However, game streaming and, therefore, watching other people play videogames has also outgrown its humble beginnings by far. Game streams, be it live or recorded, are viewed by millions. Many of the streams are broadcasting competitive multiplayer games. This is called e-sports and it is very similar to sports broadcasting. E-sports is organized in leagues and tournaments in which players can compete in controlled environments and viewers can experience the matches, discuss and criticize just like in physical sports. In this paper, we look into the challenges for computer science in general and multimedia research in particular. The multimedia research community has done a lot of work on video streaming, broadcasting and analyzing the audience, but has missed the opportunity to investigate e-sports in detail. We focus on one particular game we deem representative for e-sports, Counter-Strike: Global Offensive, and investigate how the audience consumes game streams from competitive tournaments.
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Book Chapter |
Year of Publication | 2019 |
Book Title | Savegame |
Volume | 4 |
Pagination | 197 - 206 |
Publisher | Springer Fachmedien Wiesbaden |
Place Published | Wiesbaden |
ISBN Number | 978-3-658-27394-1 |
ISBN | 2524-3241 |
URL | http://link.springer.com/10.1007/978-3-658-27395-8_13 |
DOI | 10.1007/978-3-658-27395-8_13 |
Image Retrieval Evaluation in Specific Domains
In Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, 275-305. Vol. INRE, volume 41. Springer, 2019.Status: Published
Image Retrieval Evaluation in Specific Domains
Image retrieval was, and still is, a hot topic in research. It comes with many challenges that changed over the years with the emergence of more advanced methods for analysis and enormous growth of images created, shared and consumed. This chapter gives an overview of domain-specific image retrieval evaluation approaches, which were part of the ImageCLEF evaluation campaign . Specifically, the robot vision, photo retrieval, scalable image annotation and lifelogging tasks are presented. The ImageCLEF medical activity is described in a separate chapter in this volume. Some of the presented tasks have been available for several years, whereas others are quite new (like lifelogging). This mix of new and old topics has been chosen to give the reader an idea about the development and trends within image retrieval. For each of the tasks, the datasets, participants, techniques used and lessons learned are presented and discussed leading to a comprehensive summary.
Afilliation | Machine Learning |
Project(s) | Simula Metropolitan Center for Digital Engineering, Department of Holistic Systems |
Publication Type | Book Chapter |
Year of Publication | 2019 |
Book Title | Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF |
Volume | INRE, volume 41 |
Pagination | 275-305 |
Publisher | Springer |
DOI | 10.1007/978-3-030-22948-1_12 |
Poster
Efficient Processing of Medical Videos in a Multi-auditory Environment Using Gpu Lending
NVIDIA's GPU Technology Conference (GTC), 2019.Status: Published
Efficient Processing of Medical Videos in a Multi-auditory Environment Using Gpu Lending
Afilliation | Software Engineering |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Poster |
Year of Publication | 2019 |
Place Published | NVIDIA's GPU Technology Conference (GTC) |
Miscellaneous
Kunstig intelligens for endoskopi – Automatisk deteksjon av lesjoner i sanntid
NGF Nytt, Vol. 26, No 1, March 2019, p. 34: Norsk Gastroenterologisk Forening, 2019.Status: Published
Kunstig intelligens for endoskopi – Automatisk deteksjon av lesjoner i sanntid
BAKGRUNN: I krysningspunktet mellom matematikk, informatikk og statistikk finner vi den vitenskapelige disiplinen kunstig intelligens (KI). Sammen med de siste års eksplosive utvikling innen teknologi har KI muliggjort nye algoritmer, modeller og systemer for maskinassistert diagnostikk. Resultater fra KI basert på dype nevrale nettverk har vist spesielt stort potensiale, også for automatisk deteksjon av lesjoner og anatomiske landemerker i gastrointestinaltraktus under endoskopi. Med sensitivitet og spesifisitet for deteksjon av polypper i tykktarm
på over 90% møter slike metoder nødvendige kliniske krav, men mange eksperimenter er utført på begrensede datasett, eller analysert på feilaktig grunnlag grunnet manglende tilgang og forståelse hos informatikere. For å oppnå best mulig resultat er
et interdisiplinært samarbeid mellom klinikere og informatikere
en forutsetning. Informatikerne trenger medisinske innspill for å lage effektive systemer som fungerer ute i klinikken, og klinikerne trenger forståelse av systemet for å kunne stole på resultatet og stille pålitelige diagnoser. En stor utfordring for denne tilliten er
at fremgangsmåten til en KI-algoritme sees på som en svart boks hvor ingen nøyaktig kan dechiffrere hvordan systemet kom frem
til sin konklusjon.
METODE: Vi har gjennom mange år samlet en stor bilde- database fra endoskopier utført ved Bærum Sykehus, Vestre Viken HF. Bildene er gjennomgått og annotert av tre erfarne endoskopører og fordelt på 16 klasser, inkludert normal Z-linje, øsofagitt, normal cøkum, polypper og ulcerøs colitt. Deretter er bildene brukt til å utvikle, trene og teste KI-modeller. Modellene er basert på maskinlæring og dyp læring, en gren innen KI. Med vårt system Mimir, som kombinerer KI med informasjonssøk og
-gjenfinning, søker vi å lage et helhetlig beslutningsstøttesystem for endoskopører. Algoritmene analyserer videoer i sanntid, finner lesjoner, klassifiserer disse og gir skopøren live feedback om funn under undersøkelsen, slik at funnene kan undersøkes nærmere. Mimir presenterer deretter resultatene i egen programvare, og bruker blant annet “heatmaps” til å forklare hvordan konklusjonen er nådd, og er på den måten et bidrag på veien til å forstå hvordan KI-algoritmene fungerer. Videre jobber vi med å videreutvikle Mimirs støtte for automatisk rapportgenerering, med bilder
og standardtekst basert på funn fra undersøkelsen.
RESULTATER: Deteksjon og klassifisering for de 16 gruppene har vist en sensitivitet på 0,939 og en spesifisitet på 0,996. Algoritmene våre klarer å prosessere bildene i hastigheter på mellom 30 - 1000 bilder per sekund, raskt nok til å kjøre deteksjon i sanntid. En prototype av systemet er i samråd med klinikere testet ved å koble til et koloskopisystem ute i klinikken, og kan
nå analysere videoer i sanntid.
KONKLUSJON: Tester av våre system viser at KI kan bli et viktig hjelpemiddel for å bedre oppdage GI-forandringer, og generere automatiske rapporter i løpet av nærmeste fremtid. Dette kan fungere som viktig beslutningsstøtte for endoskopører, og kan brukes i opplæring av nye endoskopører. Den største begrensningen med KI er at vi per i dag ikke vet hvordan systemet kommer frem til sin konklusjon, som kan påvirke i hvor stor grad vi stoler på resultatet. Vi arbeider derfor med et helhetlig system som ikke bare hjelper legen med diagnostikk, men også forklarer hvordan konklusjonen er nådd, samt å generere automatiske rapporter fra undersøkelsen.
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Miscellaneous |
Year of Publication | 2019 |
Publisher | Norsk Gastroenterologisk Forening |
Place Published | NGF Nytt, Vol. 26, No 1, March 2019, p. 34 |
Talk, keynote
Automatic Detection of Angiectasia: Evaluation of Deep Learning and Handcrafted Approaches
In IEEE Conference on Biomedical and Health Informatics (BHI) 2018, 2018.Status: Published
Automatic Detection of Angiectasia: Evaluation of Deep Learning and Handcrafted Approaches
Angiectasia, formerly called angiodysplasia, is one of the most frequent vascular lesions and often the cause of gastrointestinal bleedings. Medical specialists assessing videos of examinations reach a detection performance of 16% for the detection of bleeding to 69% for the detection of angiectasia. In this paper, we present several machine-learning-based approaches for angiectasia detection in wireless video capsule endoscopy images. The most promising results for pixel-wise localization and frame-wise detection are obtained by the proposed deep learning approach using generative adversarial networks (GANs) with a sensitivity of 88% and specificity of 99.9% for pixel-wise localization and a sensitivity of 98% and a specificity of 100% for frame-wise detection, which fits the requirements for automatic angiectasia detection in real clinical settings.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | IEEE Conference on Biomedical and Health Informatics (BHI) 2018 |
Proceedings, refereed
Automatic Hyperparameter Optimization in Keras for the MediaEval 2018 Medico Multimedia Task
In Working Notes Proceedings of the MediaEval 2018 Workshop. CEUR Workshop Proceedings (CEUR-WS.org), 2018.Status: Published
Automatic Hyperparameter Optimization in Keras for the MediaEval 2018 Medico Multimedia Task
This paper details the approach to the MediaEval 2018 Medico Multimedia Task made by the Rune team. The decided upon approach uses a work-in-progress hyperparameter optimization system called Saga. Saga is a system for creating the best hyperparameter finding in Keras, a popular machine learning framework, using Bayesian optimization and transfer learning. In addition to optimizing the Keras classifier configuration, we try manipulating the dataset by adding extra images in a class lacking in images and splitting a commonly misclassified class into two classes.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Publisher | CEUR Workshop Proceedings (CEUR-WS.org) |
Keywords | automatic hyperparameter optimization, Bayesian optimization, CNN, convolutional neural networks, dataset manipulation, gpyopt, hyperparameter optimization, keras, saga, tensorflow, Transfer Learning |
Autonomic Adaptation of Multimedia Content Adhering to Application Mobility
In Distributed Applications and Interoperable Systems (DAIS 2018). Lecture Notes in Computer Science ed. Vol. 10853. Springer, Cham, 2018.Status: Published
Autonomic Adaptation of Multimedia Content Adhering to Application Mobility
Today,manyusersofmultimediaapplicationsaresurrounded by a changing set of multimedia-capable devices. However, users can move their running multimedia applications only to a pre-defined set of devices. Application mobility is the paradigm where users can move their running applications (or parts of) to heterogeneous devices in a seamless manner. In order to continue multimedia processing under the implied context changes in application mobility, applications need to adapt the presentation of multimedia content and their internal configuration. We propose the system DAMPAT that implements an adaptation control loop to adapt multimedia pipelines. Exponential combinatorial growth of possible pipeline configurations is controlled by architectural constraints specified as high-level goals by application developers. Our evaluation shows that the pipeline only needs to be interrupted a few tens of milliseconds to perform the reconfiguration. Thus, production or consumption of multimedia content can continue across heterogeneous devices and user context changes in a seamless manner.
Afilliation | Communication Systems |
Project(s) | Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization, Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Distributed Applications and Interoperable Systems (DAIS 2018) |
Volume | 10853 |
Edition | Lecture Notes in Computer Science |
Pagination | 153-168 |
Date Published | 06/2018 |
Publisher | Springer, Cham |
ISBN Number | 978-3-319-93766-3 |
DOI | 10.1007/978-3-319-93767-0_11 |
Comprehensible Reasoning and Automated Reporting of Medical Examinations Based on Deep Learning Analysis
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
Comprehensible Reasoning and Automated Reporting of Medical Examinations Based on Deep Learning Analysis
Afilliation | Communication Systems |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 490-493 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208113 |
Deep Learning and Hand-crafted Feature Based Approaches for Polyp Detection in Medical Videos
In 31st IEEE CBMS International Symposium on Computer-Based Medical Systems. Karlstad, Sweden: IEEE, 2018.Status: Published
Deep Learning and Hand-crafted Feature Based Approaches for Polyp Detection in Medical Videos
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 31st IEEE CBMS International Symposium on Computer-Based Medical Systems |
Pagination | 381-386 |
Publisher | IEEE |
Place Published | Karlstad, Sweden |
ISSN Number | 2372-9198 |
DOI | 10.1109/CBMS.2018.00073 |
Deep Learning and Handcrafted Feature Based Approaches for Automatic Detection of Angiectasia
In 2018 IEEE Conference on Biomedical and Health Informatics (BHI). IEEE, 2018.Status: Published
Deep Learning and Handcrafted Feature Based Approaches for Automatic Detection of Angiectasia
Angiectasia, formerly called angiodysplasia, is one of the most frequent vascular lesions and often the cause of gastrointestinal bleedings. Medical specialists assessing videos or images of examinations reach a detection performance of 16% for the detection of bleeding to 69% for the detection of angiectasia. This shows that automatic detection to support medical experts can be useful. In this paper, we present several machine learning-based approaches for angiectasia detection in wireless video capsule endoscopy frames. In summary, the most promising results for pixel-wise localization and framewise detection are obtained by the proposed deep learning method using generative adversarial networks (GANs). Using this approach, we achieve a sensitivity of 88% and specificity of 99.9% for pixel-wise localization, and a sensitivity of 98% and a specificity of 100% for frame-wise detection. Thus, the results demonstrate the capability of using deep learning for automatic angiectasia detection in real clinical settings.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE Conference on Biomedical and Health Informatics (BHI) |
Pagination | 365-368 |
Publisher | IEEE |
Keywords | Angiectasia, computer aided diagnosis, deep learning, Machine learning, video capsular endoscopy |
DOI | 10.1109/BHI.2018.8333444 |
Deep learning approaches for flood classification and flood aftermath detection
In Working Notes Proceedings of the MediaEval 2018 Workshop. Vol. 2283. Sophia Antipolis, France: CEUR-WS.org, 2018.Status: Published
Deep learning approaches for flood classification and flood aftermath detection
This paper presents the method proposed by team UTAOS for MediaEval 2018 Multimedia Satellite Task: Emergency Response for Flooding Events. In the first challenge, we mainly rely on object and scene level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are combined using early, late and double fusion techniques achieving an average F1-score of 65.03%, 60.59% and 63.58%, respectively. For the second challenge, we rely on a convolutional neural networks (CNNs) and a transfer learning-based classification approach achieving an average F1-score of 62.30% and 61.02% for run 1 and run 2, respectively.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Volume | 2283 |
Publisher | CEUR-WS.org |
Place Published | Sophia Antipolis, France |
Deep Learning Based Disease Detection Using Domain Specific Transfer Learning
In MediaEval 2018. MediaEval, 2018.Status: Published
Deep Learning Based Disease Detection Using Domain Specific Transfer Learning
In this paper, we present our approach for the Medico Multimedia Task as part of the MediaEval 2018 Benchmark. Our method is based on convolutional neural networks (CNNs), where we compare how fine-tuning, in the context of transfer learning, from different source domains (general versus medical domain) affect classification performance. The preliminary results show that fine-tuning models trained on large and diverse datasets is favorable, even when the model’s source domain has little to no resemblance to the new target.
Afilliation | Machine Learning |
Project(s) | Department of Machine Intelligence |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | MediaEval 2018 |
Publisher | MediaEval |
Keywords | convolutional neural networks, deep learning, Gastrointestinal Disease Detection |
Dissecting Deep Neural Networks for Better Medical Image Classification and Classification Understanding
In 31st IEEE CBMS International Symposium on Computer-Based Medical Systems. Karlstad, Sweden: IEEE, 2018.Status: Published
Dissecting Deep Neural Networks for Better Medical Image Classification and Classification Understanding
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 31st IEEE CBMS International Symposium on Computer-Based Medical Systems |
Publisher | IEEE |
Place Published | Karlstad, Sweden |
ISSN Number | 2372-9198 |
DOI | 10.1109/CBMS.2018.00070 |
Dynamic Adaptation of Multimedia Presentations for Videoconferencing in Application Mobility
In IEEE International Conference on Multimedia and Expo (ICME). San Diego, CA, USA: IEEE, 2018.Status: Published
Dynamic Adaptation of Multimedia Presentations for Videoconferencing in Application Mobility
Application mobility is the paradigm where users can move their running applications to heterogeneous devices in a seamless manner. This mobility involves dynamic context changes of hardware, network resources, user environment, and user preferences. In order to continue multimedia processing under these context changes, applications need to adapt not only the collection of media streams, i.e., multimedia presentation, but also their internal configuration to work on different hardware. We present the performance analysis to adapt a video-conferencing prototype application in a proposed adaptation control loop to autonomously adapt multimedia pipelines. Results show that the time spent to create an adaptation plan and execute it is in the order of hundreds of milliseconds. The reconfiguration of pipelines, compared to building them from scratch, is approximately 1000 times faster when re-utilizing already instantiated hardware-dependent components. Therefore, we conclude that the adaptation of multimedia pipelines is a feasible approach for multimedia applications that adhere to application mobility.
Afilliation | Communication Systems |
Project(s) | Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization, Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | IEEE International Conference on Multimedia and Expo (ICME) |
Date Published | 07/2018 |
Publisher | IEEE |
Place Published | San Diego, CA, USA |
ISSN Number | 1945-7871 |
DOI | 10.1109/ICME.2018.8486565 |
Efficient Live and on-Demand Tiled HEVC 360 VR Video Streaming
In 2018 IEEE International Symposium on Multimedia (ISM). Taichung, Taiwan: IEEE, 2018.Status: Published
Efficient Live and on-Demand Tiled HEVC 360 VR Video Streaming
With 360◦ panorama video technology becoming commonplace, the need for efficient streaming methods for such videos arises. We go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-off between bandwidth usage and the video quality in the user’s field-of-view. We have created an architecture that combines RTP and DASH to deliver 360◦ VR content to a Huawei set-top-box and a Samsung Galaxy S7. Our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-offs through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the field-of- view changes.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE International Symposium on Multimedia (ISM) |
Pagination | 81-88 |
Date Published | 12/2018 |
Publisher | IEEE |
Place Published | Taichung, Taiwan |
DOI | 10.1109/ISM.2018.00022 |
Flexible Device Sharing in PCIe Clusters using Device Lending
In International Conference on Parallel Processing Companion (ICPP'18 Comp). ACM, 2018.Status: Published
Flexible Device Sharing in PCIe Clusters using Device Lending
Processing workloads may have very high IO demands, exceeding the capabilities provided by resource virtualization and requiring direct access to the physical hardware. For computers that are interconnected in PCI Express (PCIe) networks, we have previously proposed Device Lending as a solution for assigning devices to remote hosts. In this paper, we explain how we have extended our implementation with support for the Linux Kernel-based Virtual Machine (KVM) hypervisor. Using our extended Device Lending, it becomes possible to dynamically “pass through” physical remote devices to VM guests while still retaining the flexibility of virtualization, something that previously required extensive facilitation in both hypervisor and device drivers in the form of paravirtualization.
We have also improved our original implementation with sup- port for interoperability between remote devices. We show that it is possible to use multiple devices residing in different hosts, while still achieving the same bandwidth and latency as native PCIe, and without requiring any additional support in device drivers.
Afilliation | Communication Systems |
Project(s) | Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization, LADIO: Live Action Data Input/Output, Department of Holistic Systems, Department of High Performance Computing |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | International Conference on Parallel Processing Companion (ICPP'18 Comp) |
Date Published | 08/2018 |
Publisher | ACM |
ISBN Number | 978-1-4503-6523-9/18/08 |
DOI | 10.1145/3229710.3229759 |
GameStory Task at MediaEval 2018
In Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation. CEUR Workshop Proceedings, 2018.Status: Published
GameStory Task at MediaEval 2018
That video games have reached the masses is well known. Moreover, game streaming and watching other people play video games is a phenomenon that has outgrown its small beginnings. Game video streams, be it live or recorded, are viewed by millions. E-sports is the result of organized leagues and tournaments in which players can compete in controlled environments and viewers can experience the matches, discuss and criticize, just like in physical sports. In the GameStory task, taking place the first time in 2018, we approach the game streaming and e-sports phenomena from a multimedia research side. We focus on the task of summarizing matches using a specific relevant game, Counter-Strike: Global Offensive, as a case study. With the help of ZNIPE.tv, we provide a data set of high quality data and meta data from competitive tournaments and aim to foster research in the area of e-sports and game streaming.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation |
Date Published | 10/2018 |
Publisher | CEUR Workshop Proceedings |
Medico Multimedia Task at MediaEval 2018
In Working Notes Proceedings of the MediaEval 2018 Workshop. CEUR Workshop Proceedings, 2018.Status: Published
Medico Multimedia Task at MediaEval 2018
The Medico: Multimedia for Medicine Task, running for the second time as part of MediaEval 2018, focuses on detecting abnormalities, diseases, anatomical landmarks and other findings in images captured by medical devices in the gastrointestinal tract. The task is described, including the use case and its challenges, the dataset with ground truth, the required participant runs and the evaluation metrics.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Publisher | CEUR Workshop Proceedings |
Mimir: An Automatic Reporting and Reasoning System for Deep Learning based Analysis in the Medical Domain
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
Mimir: An Automatic Reporting and Reasoning System for Deep Learning based Analysis in the Medical Domain
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 369-374 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208129 |
OpenSea - Open Search Based Classification Tool
In Proceedings of the 9th ACM Multimedia Systems Conference. Amsterdam, Netherlands: ACM, 2018.Status: Published
OpenSea - Open Search Based Classification Tool
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 363-368 |
Publisher | ACM |
Place Published | Amsterdam, Netherlands |
ISBN Number | 978-1-4503-5192-8 |
DOI | 10.1145/3204949.3208128 |
Popsift: a faithful SIFT implementation for real-time applications
In Proceedings of the 9th ACM Multimedia Systems Conference. New York, NY, USA: ACM Press, 2018.Status: Published
Popsift: a faithful SIFT implementation for real-time applications
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 415-420 |
Date Published | 06/2018 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450351928 |
URL | http://dl.acm.org/citation.cfm?doid=3204949http://dl.acm.org/citation.cf... |
DOI | 10.1145/320494910.1145/3204949.3208136 |
Popsift: a faithful SIFT implementation for real-time applications
In Proceedings of the 9th ACM Multimedia Systems Conference. New York, NY, USA: ACM Press, 2018.Status: Published
Popsift: a faithful SIFT implementation for real-time applications
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th ACM Multimedia Systems Conference |
Pagination | 415-420 |
Date Published | 06/2018 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450351928 |
URL | http://dl.acm.org/citation.cfm?doid=3204949http://dl.acm.org/citation.cf... |
DOI | 10.1145/320494910.1145/3204949.3208136 |
Team ORG @ GameStory Task 2018
In Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation. MediaEval, 2018.Status: Published
Team ORG @ GameStory Task 2018
This paper describes the approach of the organizers' team for a submission to the GameStory task at MediaEval 2018. Goal of the task is to provide a summary of a match of Counter Strike: Global Offensive (CS:GO), a popular e-sports game, that boils down a long game to it's most important events and delivers a story on the progress of the match. Our approach was to provide match statistics and overlay them with events and highlights of the game. We focused on ends of critical rounds, i.e. the rounds where one team took the lead over the other one, and kill streaks, where one player eliminated a substantial number of other players in short time.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation |
Date Published | 10/2018 |
Publisher | MediaEval |
The 2018 Medico Multimedia Task Submission of Team NOAT using Neural Network Features and Search-based Classification
In Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation. CEUR Workshop Proceedings, 2018.Status: Published
The 2018 Medico Multimedia Task Submission of Team NOAT using Neural Network Features and Search-based Classification
In this paper, we describe our approach for the classification of medical images depicting the human gastrointestinal tract. Search-based classification is performed in three stages. In the first stage, we extract deep features for each image using pre-trained deep-learning models. In the second stage, we use LIRE to index the generated features, so that we can then, in the final stage, search the index for similar images and make our predictions based on the results. With this approach, we achieved a MCC score of 0,54 and a accuracy of 0,94, which shows that deep features combined with search-based classification are a viable option for medical image analysis.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceeding of the MediaEval Benchmarking Initiative for Multimedia Evaluation |
Date Published | 10/2018 |
Publisher | CEUR Workshop Proceedings |
The Importance of Medical Multimedia
In 2018 ACM Multimedia Conference (MM '18). New York, NY, USA: ACM Press, 2018.Status: Published
The Importance of Medical Multimedia
Multimedia research is becoming more and more important for the medical domain, where an increasing number of videos and images are integrated in the daily routine of surgical and diagnostic work. While the collection of medical multimedia data is not an issue, appropriate tools for efficient use of this data are missing. This includes management and inspection of the data, visual analytics, as well as learning relevant semantics and using recognition results for optimizing surgical and diagnostic processes. The characteristics and requirements in this interesting but challenging field are different than the ones in classic multimedia domains. Therefore, this tutorial gives a general introduction to the field, provides a broad overview of specific requirements and challenges, discusses existing work and open challenges, and elaborates in detail how machine learning approaches can help in multimedia-related fields to improve the performance of surgeons/clinicians.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 ACM Multimedia Conference (MM '18) |
Pagination | 2106-2108 |
Date Published | 10/2018 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450356657 |
URL | http://dl.acm.org/citation.cfm?doid=3240508http://dl.acm.org/citation.cf... |
DOI | 10.1145/3240508.3241475 |
The Medico-Task 2018: Disease Detection in the Gastrointestinal Tract using Global Features and Deep Learning
In MediaEval 2018. Nice, France: MediaEval, 2018.Status: Published
The Medico-Task 2018: Disease Detection in the Gastrointestinal Tract using Global Features and Deep Learning
In this paper, we present our approach for the 2018 Medico Task classifying diseases in the gastrointestinal tract. We have proposed a system based on global features and deep neural networks. The best approach combines two neural networks and the reproducible experimental results signify the efficiency of the proposed model with an accuracy rate of 95.80%, a precision of 95.87%, and an F1-score of 95.80%.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | MediaEval 2018 |
Date Published | 10/2018 |
Publisher | MediaEval |
Place Published | Nice, France |
Keywords | CNN, deep learning, Gastrointestinal Disease Detection, Global Features, Medico-Task 2018, Transfer Learning |
Tradeoffs using Binary and Multiclass Neural Network Classification for Medical Multidisease Detection
In 2018 IEEE International Symposium on Multimedia (ISM). IEEE, 2018.Status: Published
Tradeoffs using Binary and Multiclass Neural Network Classification for Medical Multidisease Detection
The interest in neural networks has increased sig- nificantly, and the application of this type of machine learning is vast, ranging from natural image classification to medical image segmentation. However, many users of neural networks tend to use them as a black box tool. They do not access all of the possible variations, nor take into account the respective classification accuracies and costs. In our work, we focus on multiclass image classification, and in this research, we shed light on the trade-offs between systems using a single multiclass classification and multiple binary classifiers, respectively. We have tested the these classifiers on several modern neural network architectures, including DenseNet, Inception v3, Inception ResNet v2, Xception, NASNet and MobileNet. We have compared several aspects of the performance of these architectures during training and testing using both classification styles. We have compared classification speed and several classification accuracy metrics. Here, we present the results from experiments on a total of 99 networks: 11 multiclass and 88 individual binary networks, for an 8-class classification of medical images. In short, using multiple binary classification networks resulted in a 7% increase in the average F1 score, a 1% increase in average accuracy, a 1% increase in precision, and a 4% increase in average recall. However, on average, such a multi-network style performed the classification 7.6 times slower compared to a single network multiclass implementation. These collective findings show that both approaches can be applied to modern neural network structures. Several binary networks will often give increased classification accuracy, but at the cost of classification speed and resource consumption.
Afilliation | Communication Systems, Machine Learning |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE International Symposium on Multimedia (ISM) |
Pagination | 1-8 |
Date Published | 12/2018 |
Publisher | IEEE |
DOI | 10.1109/ISM.2018.00009 |
Transfer learning with prioritized classification and training dataset equalization for medical objects detection
In Working Notes Proceedings of the MediaEval 2018 Workshop. CEUR Workshop Proceedings, 2018.Status: Published
Transfer learning with prioritized classification and training dataset equalization for medical objects detection
This paper presents the method proposed by the organizer team (SIMULA) for MediaEval 2018 Multimedia for Medicine: the Medico Task. We utilized the recent transfer-learning-based image classification methodology and focused on how easy it is to implement multi-class image classifiers in general and how to improve the classification performance without deep neural network model redesign. The goal for this was both to provide a baseline for the Medico task and to show the performance of out-of-the-box classifiers for the medical use-case scenario.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Working Notes Proceedings of the MediaEval 2018 Workshop |
Publisher | CEUR Workshop Proceedings |
Using preprocessing as a tool in medical image detection
In MediaEval 2018. Nice, France: MediaEval, 2018.Status: Published
Using preprocessing as a tool in medical image detection
In this paper, we describe our approach to gastrointestinal disease classification for the medico task at MediaEval 2018. We propose multiple ways to inpaint problematic areas in the test and training set to help with classification. We discuss the effect that preprocessing does to the input data with respect to removing regions with sparse information. We also discuss how preprocessing affects the training and evaluation of a dataset that is limited in size. We will also compare the different inpainting methods with transfer learning using a convolutional neural network.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | MediaEval 2018 |
Date Published | 10/2018 |
Publisher | MediaEval |
Place Published | Nice, France |
Keywords | classification, Image processing, Machine Leanring |
Book Chapter
Camera Synchronization for Panoramic Videos
In MediaSync, 565-592. Springer, 2018.Status: Published
Camera Synchronization for Panoramic Videos
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Book Chapter |
Year of Publication | 2018 |
Book Title | MediaSync |
Pagination | 565-592 |
Date Published | 03/2018 |
Publisher | Springer |
URL | https://doi.org/10.1007/978-3-319-65840-7_20 |
DOI | 10.1007/978-3-319-65840-7_20 |
Journal Article
Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy
World Journal of Gastroenterology 45, no. 24 (2018): 5057-5062.Status: Published
Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy
Assisted diagnosis using artificial intelligence has been a holy grail in medical research for many years, and recent developments in computer hardware have enabled the narrower area of machine learning to equip clinicians with potentially useful tools for computer assisted diagnosis (CAD) systems. However, training and assessing a computer’s ability to diagnose like a human are complex tasks, and successful outcomes depend on various factors. We have focused our work on gastrointestinal (GI) endoscopy because it is a cornerstone for diagnosis and treatment of diseases of the GI tract. About 2.8 million luminal GI (esophageal, stomach, colorectal) cancers are detected globally every year, and although substantial technical improvements in endoscopes have been made over the last 10-15 years, a major limitation of endoscopic examinations remains operator variation. This translates into a substantial inter-observer variation in the detection and assessment of mucosal lesions, causing among other things an average polyp miss-rate of 20% in the colon and thus the subsequent development of a number of post-colonoscopy colorectal cancers. CAD systems might eliminate this variation and lead to more accurate diagnoses. In this editorial, we point out some of the current challenges in the development of efficient computer-based digital assistants. We give examples of proposed tools using various techniques, identify current challenges, and give suggestions for the development and assessment of future CAD systems.
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | World Journal of Gastroenterology |
Volume | 45 |
Issue | 24 |
Pagination | 5057-5062 |
Date Published | 12/2018 |
Publisher | Baishideng Publishing Group Inc |
URL | https://www.wjgnet.com/1007-9327/abstract/v24/i45/5057.htm |
DOI | 10.3748/wjg.v24.i45.5057 |
Multimodal analysis of user behavior and browsed content under different image search intents
International Journal of Multimedia Information Retrieval 7 (2018): 29-41.Status: Published
Multimodal analysis of user behavior and browsed content under different image search intents
The motivation or intent of a search for content may vary between users and use-cases. Knowledge and understanding of these underlying objectives may therefore be important in order to return appropriate search results, and studies of user search intent are emerging in information retrieval to understand why a user is searching for a particular type of content. In the context of image search, our work targets automatic recognition of users’ intent in an early stage of a search session. We have designed seven different search scenarios under the intent conditions of finding items, re-finding items and entertainment. Moreover, we have collected facial expressions, physiological responses, eye gaze and implicit user interactions from 51 participants who performed seven different search tasks on a custom-built image retrieval platform, and we have analyzed the users’ spontaneous and explicit reactions under different intent conditions. Finally, we trained different machine learning models to predict users’ search intent from the visual content of the visited images, the user interactions and the spontaneous responses. Our experimental results show that after fusing the visual and user interaction features, our system achieved the F-1 score of 0.722 for classifying three classes in a user-independent cross-validation. Eye gaze and implicit user interactions, including mouse movements and keystrokes are the most informative features for intent recognition. In summary, the most promising results are obtained by modalities that can be captured unobtrusively and online, and the results therefore demonstrate the potential of including intent-based methods in multimedia retrieval platforms.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | International Journal of Multimedia Information Retrieval |
Volume | 7 |
Pagination | 29 - 41 |
Date Published | Jan-03-2018 |
Publisher | Springer |
ISSN | 2192-6611 |
URL | http://link.springer.com/10.1007/s13735-018-0150-6http://link.springer.c... |
DOI | 10.1007/s13735-018-0150-6 |
Quantified Soccer Using Positional Data: A Case Study
Frontiers in Physiology 9 (2018): 866.Status: Published
Quantified Soccer Using Positional Data: A Case Study
Performance development in international soccer is undergoing a silent revolution fueled by the rapidly increasing availability of athlete quantification data and advanced analytics. Objective performance data from teams and individual players are increasingly being collected automatically during practices and more recently also in matches after FIFA's 2015 approval of wearables in electronic performance and tracking systems. Some clubs have even started collecting data from players outside of the sport arenas. Further algorithmic analysis of these data might provide vital insights for individual training personalization and injury prevention, and also provide a foundation for evidence-based decisions for team performance improvements. This paper presents our experiences from using a detailed radio-based wearable positioning data system in an elite soccer club. We demonstrate how such a system can detect and find anomalies, trends, and insights vital for individual athletic and soccer team performance development. As an example, during a normal microcycle (6 days) full backs only covered 26% of the sprint distance they covered in the next match. This indicates that practitioners must carefully consider to proximity size and physical work pattern in microcycles to better resemble match performance. We also compare and discuss the accuracy between radio waves and GPS in sampling tracking data. Finally, we present how we are extending the radio-based positional system with a novel soccer analytics annotation system, and a real-time video processing system using a video camera array. This provides a novel toolkit for modern forward-looking soccer coaches that we hope to integrate in future studies.
Afilliation | Machine Learning |
Project(s) | No Simula project, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Frontiers in Physiology |
Volume | 9 |
Pagination | 866 |
Date Published | Jun-07-2018 |
Publisher | Frontiers |
URL | https://www.frontiersin.org/article/10.3389/fphys.2018.00866/fullhttps:/... |
DOI | 10.3389/fphys.2018.00866 |
Social Media and Satellites. Disaster event detection, linking and summarization
Multimedia Tools and Applications 78, no. 3 (2018): 2837-2875.Status: Published
Social Media and Satellites. Disaster event detection, linking and summarization
Being able to automatically link social media and satellite imagery holds large opportunities for research, with a potentially considerable impact on society. The possibility of integrating different information sources opens in fact to new scenarios where the wide coverage of satellite imaging can be used as a collector of the fine-grained details provided by the social media. Remote-sensed data and social media data can well complement each other, integrating the wide perspective provided by the satellite view with the information collected locally, being it textual, audio, or visual. Among the possible applications, natural disasters are certainly one of the most interesting scenarios, where global and local perspectives are needed at the same time.
In this paper, we present a system called JORD that is able to autonomously collect social media data (including the text analysis in local languages) about technological and environmental disasters, and link it automatically to remote-sensed data. Moreover, in order to ensure the quality of retrieved information, JORD is equipped with a hierarchical filtering mechanism relying on the temporal information and the content analysis of retrieved multimedia data.
To show the capabilities of the system, we present a large number of disaster events detected by the system, and we evaluate both the quality of the provided information about the events and the usefulness of JORD from potential users viewpoint, using crowdsourcing.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Multimedia Tools and Applications |
Volume | 78 |
Issue | 3 |
Pagination | 2837–2875 |
Publisher | Springer |
Place Published | US |
Keywords | Event Detection, Information retrieval, Natural Disaster, Social Media |
DOI | 10.1007/s11042-018-5982-9 |
Proceedings, refereed
A comparison of deep learning with global features for gastrointestinal disease detection
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
A comparison of deep learning with global features for gastrointestinal disease detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
A Holistic Multimedia System for Gastrointestinal Tract Disease Detection
In 8th annual ACM conference on Multimedia Systems (MMSys). ACM, 2017.Status: Published
A Holistic Multimedia System for Gastrointestinal Tract Disease Detection
Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.
In this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources, Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | 8th annual ACM conference on Multimedia Systems (MMSys) |
Pagination | 112-123 |
Date Published | 06/2017 |
Publisher | ACM |
ISBN Number | 978-1-4503-5002-0 |
URL | http://dl.acm.org/citation.cfm?id=3083189 |
DOI | 10.1145/3083187.3083189 |
ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections
In ACM International Conference on Multimedia Retrieval. Bucharest: ACM, 2017.Status: Published
ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Conference on Multimedia Retrieval |
Date Published | 06/2017 |
Publisher | ACM |
Place Published | Bucharest |
CNN and GAN Based Satellite and Social Media Data Fusion for Disaster Detection
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
CNN and GAN Based Satellite and Social Media Data Fusion for Disaster Detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
EIR: changing the scene of automatic detection software for gastrointestinal endoscopy
In World Congress of GI Endoscopy. Hyderabad, India: World Endoscopic Organisation, 2017.Status: Published
EIR: changing the scene of automatic detection software for gastrointestinal endoscopy
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | World Congress of GI Endoscopy |
Date Published | 02/2017 |
Publisher | World Endoscopic Organisation |
Place Published | Hyderabad, India |
Finding equilibrium for gym ownership distribution based on game dynamics in Pokémon GO game
In ACM International Workshop on Massively Multiuser Virtual Environments. Taiwan: ACM, 2017.Status: Published
Finding equilibrium for gym ownership distribution based on game dynamics in Pokémon GO game
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Workshop on Massively Multiuser Virtual Environments |
Pagination | 1-6 |
Date Published | 07/2017 |
Publisher | ACM |
Place Published | Taiwan |
ISBN Number | 978-1-4503-5006-8 |
DOI | 10.1145/3083207.3083208 |
JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery
In Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing - CBMI '17. New York, USA: ACM Press, 2017.Status: Published
JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery
Gathering information, and continuously monitoring the affected areas after a natural disaster can be crucial to assess the damage, and speed up the recovery process. Satellite imagery is being considered as one of the most productive sources to monitor the after effects of a natural disaster; however, it also comes with a lot of challenges and limitations, due to slow update. It would be beneficiary to link remote sensed data with social media for the damage assessment, and obtaining detailed information about a disaster. The additional information, which are obtainable by social media, can enrich remote-sensed data, and overcome its limitations. To tackle this, we present a system called JORD that is able to autonomously collect social media data about natural disasters, and link it automatically to remote-sensed data. In addition, we demonstrate that queries in local languages that are relevant to the exact position of natural disasters retrieve more accurate information about a disaster event. We also provide content based analysis along with temporal and geo-location based filtering to provide more accurate information to the users. To show the capabilities of the system, we demonstrate that a large number of disaster events can be detected by the system. In addition, we use crowdsourcing to demonstrate the quality of the provided information about the disasters, and usefulness of JORD from potential users point of view.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing - CBMI '17 |
Pagination | 12:1--12:6 |
Publisher | ACM Press |
Place Published | New York, USA |
ISBN Number | 9781450353335 |
URL | http://dl.acm.org/citation.cfm?doid=3095713 |
DOI | 10.1145/3095713.3095726 |
Kvasir: A Multi-Class Image-Dataset for Computer Aided Gastrointestinal Disease Detection
In ACM Multimedia Systems. Taiwan: ACM, 2017.Status: Published
Kvasir: A Multi-Class Image-Dataset for Computer Aided Gastrointestinal Disease Detection
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM Multimedia Systems |
Date Published | 07/2017 |
Publisher | ACM |
Place Published | Taiwan |
LireSolr - A Visual Information Retrieval Server
In ACM International Conference on Multimedia Retrieval. Bucharest: ACM, 2017.Status: Published
LireSolr - A Visual Information Retrieval Server
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Conference on Multimedia Retrieval |
Date Published | 06/2017 |
Publisher | ACM |
Place Published | Bucharest |
ISBN Number | 978-1-4503-4701-3 |
DOI | 10.1145/3078971.3079014 |
Load Balancing of Multimedia Workloads for Energy Efficiency on the Tegra K1 Multicore Architecture
In 8th annual ACM conference on Multimedia Systems (MMSys). ACM, 2017.Status: Published
Load Balancing of Multimedia Workloads for Energy Efficiency on the Tegra K1 Multicore Architecture
Energy efficiency is a timely topic for modern mobile computing. Reducing the energy consumption of devices not only increases their battery lifetime, but also reduces the risk of hardware failure. Many researchers strive to
understand the relationship between software activity and hardware power usage. A recurring strategy for saving power is to reduce operating frequencies. It is widely acknowledged that standard frequency scaling algorithms generally overreact to changes in hardware utilisation. More recent and original efforts attempt to balance software workloads on heterogeneous multicore architectures, such as the Tegra K1, which includes a quad-core CPU and a CUDA-capable GPU. However, it is not known whether it is possible to utilise these processor elements in parallel to save energy. Research into these types of systems are unfortunately often evaluated with the Performance Per Watt (PPW) metric, which is an unaccurate method because it ignores constant power usage from idle components. We show that this metric can end up increase energy usage on the Tegra K1, and give a false impression of how such systems consume energy. In reality, we show that it is much harder to save energy by balancing workloads between the heterogeneous cores of the Tegra K1, where we demonstrate only a 5% energy saving by offloading 10% DCT workload from the GPU to the CPU. Significantly more energy can be saved (up to 50%) using the appropriate processor for different workloads.
Afilliation | Communication Systems |
Project(s) | Unified PCIe IO: Unified PCI Express for Distributed Component Virtualization, Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | 8th annual ACM conference on Multimedia Systems (MMSys) |
Pagination | 124-135 |
Date Published | 06/2017 |
Publisher | ACM |
ISBN Number | 978-1-4503-5002-0 |
URL | http://dl.acm.org/citation.cfm?doid=3083187.3083195 |
DOI | 10.1145/3083187.3083195 |
Medical Multimedia Information Systems (MMIS)
In the 2017 ACMProceedings of the 2017 ACM on Multimedia Conference - MM '17. Mountain View, California, USANew York, New York, USA: ACM Press, 2017.Status: Published
Medical Multimedia Information Systems (MMIS)
Afilliation | Communication Systems, Machine Learning |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | the 2017 ACMProceedings of the 2017 ACM on Multimedia Conference - MM '17 |
Publisher | ACM Press |
Place Published | Mountain View, California, USANew York, New York, USA |
ISBN Number | 9781450349062 |
URL | http://dl.acm.org/citation.cfm?doid=3123266http://dl.acm.org/citation.cf... |
DOI | 10.1145/312326610.1145/3123266.3130142 |
Medical Multimedia Information Systems (MMIS)
In ACM Multimedia. Mountain View: ACM, 2017.Status: Published
Medical Multimedia Information Systems (MMIS)
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM Multimedia |
Date Published | 10/2017 |
Publisher | ACM |
Place Published | Mountain View |
Multimedia for medicine: the medico Task at mediaEval 2017
In MediaEval Benchmark 2017. Dublin, Ireland: CEUR-WS.org, 2017.Status: Published
Multimedia for medicine: the medico Task at mediaEval 2017
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | MediaEval Benchmark 2017 |
Date Published | 09/2017 |
Publisher | CEUR-WS.org |
Place Published | Dublin, Ireland |
Multimodal Analysis of Image Search Intent - Intent Recognition in Image Search from User Behavior and Visual Content
In ACM International Conference on Multimedia Retrieval. Bucharest: ACM, 2017.Status: Published
Multimodal Analysis of Image Search Intent - Intent Recognition in Image Search from User Behavior and Visual Content
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Conference on Multimedia Retrieval |
Date Published | 07/2017 |
Publisher | ACM |
Place Published | Bucharest |
ISBN Number | 978-1-4503-4701-3 |
DOI | 10.1145/3078971.3078995 |
Nerthus: A Bowel Preparation Quality Video Dataset
In ACM Multimedia Systems. Taipei: ACM, 2017.Status: Published
Nerthus: A Bowel Preparation Quality Video Dataset
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM Multimedia Systems |
Publisher | ACM |
Place Published | Taipei |
The JORD System - Linking sky and social multimedia data to Natural and Technological Disasters
In ACM International Conference on Multimedia Retrieval. Bucharest: ACM, 2017.Status: Published
The JORD System - Linking sky and social multimedia data to Natural and Technological Disasters
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ACM International Conference on Multimedia Retrieval |
Date Published | 06/2017 |
Publisher | ACM |
Place Published | Bucharest |
ISBN Number | 978-1-4503-4701-3 |
DOI | 10.1145/3078971.3079013 |
Journal Article
Efficient disease detection in gastrointestinal videos – global features versus neural networks
Multimedia Tools and Applications 76, no. 21 (2017): 22493-22525.Status: Published
Efficient disease detection in gastrointestinal videos – global features versus neural networks
Analysis of medical videos from the human gastrointestinal (GI) tract for detection and localization of abnormalities like lesions and diseases requires both high precision and recall. Additionally, it is important to support efficient, real-time processing for live feedback during (i) standard colonoscopies and (ii) scalability for massive population-based screening, which we conjecture can be done using a wireless video capsule endoscope (camera-pill). Existing related work in this field does neither provide the necessary combination of accuracy and performance for detecting multiple classes of abnormalities simultaneously nor for particular disease localization tasks. In this paper, a complete end-to-end multimedia system is presented where the aim is to tackle automatic analysis of GI tract videos. The system includes an entire pipeline ranging from data collection, processing and analysis, to visualization. The system combines deep learning neural networks, information retrieval, and analysis of global and local image features in order to implement multi-class classification, detection and localization. Furthermore, it is built in a modular way, so that it can be easily extended to deal with other types of abnormalities. Simultaneously, the system is developed for efficient processing in order to provide real-time feedback to the doctors and for scalability reasons when potentially applied for massive population-based algorithmic screenings in the future. Initial experiments show that our system has multiclass detection accuracy and polyp localization precision at least as good as state-of-the-art systems, and provides additional novelty in terms of real-time performance, low resource consumption and ability to extend with support for new classes of diseases.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | Multimedia Tools and Applications |
Volume | 76 |
Issue | 21 |
Pagination | 22493–22525 |
Date Published | 11/2017 |
Publisher | ACM/Springer |
ISSN | 1380-7501 |
Keywords | Algorithmic screening, Automatic disease detection, Deep learning neural networks, Global and local image features, Information retrieval, Medical, performance evaluation |
URL | https://link.springer.com/article/10.1007%2Fs11042-017-4989-y |
DOI | 10.1007/s11042-017-4989-y |
From Annotation to Computer-Aided Diagnosis: Detailed Evaluation of a Medical Multimedia System
ACM Transactions on Multimedia Computing, Communications, and Applications 13, no. 3 (2017).Status: Published
From Annotation to Computer-Aided Diagnosis: Detailed Evaluation of a Medical Multimedia System
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | ACM Transactions on Multimedia Computing, Communications, and Applications |
Volume | 13 |
Issue | 3 |
Publisher | ACM |
Poster
A High Precision Power Model for the Tegra K1 CPU, GPU and RAM
In GPU Technology Conference 2016. Nvidia, 2016.Status: Published
A High Precision Power Model for the Tegra K1 CPU, GPU and RAM
Power modelling is an important topic in many areas of computing, for example to save energy in texture streaming for gaming[1] or to select efficient H.264 video encoding parameters[2]. However, researchers' view of how hardware consume power is limited. They typically resort to ratebased models to describe the energy consumption of hardware, where power usage is correlated directly with hardware access rates (for example instructions or cache misses per second)[3,4,5,6]. This approach ignores many mechanisms that impact the power usage of a system, such as rail voltages, core and clock gating, frequency scaling and variable cost of instruction execution. Because of this, they can mispredict up to 70 % on the Tegra K1. We show that by taking all these factors into account with sufficient hardware knowledge, it is possible to bridge the gap between power usage and software execution to build power models which are over 98 % accurate over all CPU, GPU and memory frequency combinations.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Poster |
Year of Publication | 2016 |
Secondary Title | GPU Technology Conference 2016 |
Date Published | 04/2016 |
Publisher | Nvidia |
URL | http://on-demand.gputechconf.com/gtc/2016/posters/images/1920x1607/GTC_2... |
Proceedings, refereed
A High-Precision, Hybrid GPU, CPU and RAM Power Model for Generic Multimedia Workloads
In 7th annual ACM conference on Multimedia Systems (MMSys). ACM, 2016.Status: Published
A High-Precision, Hybrid GPU, CPU and RAM Power Model for Generic Multimedia Workloads
Energy efficiency of multimedia processing is a hot topic in modern, mobile computing where the lifetime of battery- powered devices is low. Authors often use power models as tools to evaluate the energy-efficiency of multimedia work- loads and processing schemes. A challenge with these mod- els is that they are built without sufficiently deep hardware knowledge and as a result they have the potential to mis- predict substantially depending on hardware configuration. Typical rate-based power models can for example mispredict up to 70 % on the Tegra K1 SoC. Inspired by multimedia workloads, we introduce a modelling methodology which can be used to build a generic, high-precision power model for the Tegra K1’s GPU and memory. By considering hardware utilisation, rail voltages, leakage currents and clocks, the model achieves an average accuracy above 99 % over all op- erating frequencies, and has been rigorously tested on several multimedia workloads. Our method exposes detailed insight into hardware and how it consumes energy. This knowledge is not only useful for researchers to understand how power models should be built, but also helps to understand what developers can do to minimise power usage. For example, experiments show that for a DCT benchmark, 3 % power can be saved by utilising non-coherent caches and smaller datatypes.
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | 7th annual ACM conference on Multimedia Systems (MMSys) |
Pagination | 14:1-14:12 |
Date Published | 05/2016 |
Publisher | ACM |
ISBN Number | 978-1-4503-4297-1 |
DOI | 10.1145/2910017.2910591 |
ChaLearn Joint Contest on Multimedia Challenges Beyond Visual Analysis: An Overview
In International Conference on Pattern Recognition. Cancun, Mexico: IEEE, 2016.Status: Published
ChaLearn Joint Contest on Multimedia Challenges Beyond Visual Analysis: An Overview
This paper provides an overview of the Joint Contest on Multimedia Challenges Beyond Visual Analysis. We organized an academic competition that focused on four problems that require effective processing of multimodal information in order to be solved. Two tracks were devoted to gesture spotting and recognition from RGB-D video, two fundamental problems for human computer interaction. Another track was devoted to a second round of the first impressions challenge of which the goal was to develop methods to recognize personality traits from short video clips. For this second round we adopted a novel collaborative-competitive (i.e., coopetition) setting. The fourth track was dedicated to the problem of video recommendation for improving user experience. The challenge was open for about 45 days, and received outstanding participation: almost 200 participants registered to the contest, and 20 teams sent predictions in the final stage. The main goals of the challenge were fulfilled: the state of the art was advanced considerably in the four tracks, with novel solutions to the proposed problems (mostly relying on deep learning). However, further research is still required. The data of the four tracks will be available to allow researchers to keep making progress in the four tracks.
Afilliation | Communication Systems, Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Conference on Pattern Recognition |
Date Published | 12/2016 |
Publisher | IEEE |
Place Published | Cancun, Mexico |
Computer Aided Disease Detection System for Gastrointestinal Examinations
In Multimedia Systems Conference 2016. New York: ACM, 2016.Status: Published
Computer Aided Disease Detection System for Gastrointestinal Examinations
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia Systems Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Crowdsourcing as Self Fulfilling Prophecy: Influence of Discarding Workers in Subjective Assessment Tasks
In International Workshop on Content-based Multimedia Indexing. IEEE / ACM, 2016.Status: Published
Crowdsourcing as Self Fulfilling Prophecy: Influence of Discarding Workers in Subjective Assessment Tasks
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Workshop on Content-based Multimedia Indexing |
Date Published | 06/2016 |
Publisher | IEEE / ACM |
Device Lending in PCI Express Networks
In Proceedings of the 26th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). ACM, 2016.Status: Published
Device Lending in PCI Express Networks
The challenge of scaling IO performance of multimedia systems to demands of their users has attracted much research. A lot of effort has gone into development of distributed systems that add little latency and computing overhead. For machines in PCI Express (PCIe) clusters, we propose Device Lending as a novel solution which works at a system level. Device Lending achieves low latency and extremely low computing overhead without requiring any application-specific distribution mechanisms. For applications, the remote IO resource appears local. In fact, even the drivers of the operating system remain unaware that hardware resources are located in remote machines. By enabling machines in a PCIe cluster to lend a wide variety of hardware, cluster machines can get temporary access to a pool of IO resources. Network cards, FPGAs, SSDs, and even GPUs can easily be shared among computers. Our proposed solution, Device Lending, works transparently without requiring any modifications to drivers, operating systems or software applications.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Proceedings of the 26th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) |
Pagination | 10:1-10:6 |
Date Published | 05/2016 |
Publisher | ACM |
ISBN Number | 978-1-4503-4356-5 |
DOI | 10.1145/2910642.2910650 |
Efficient Processing of Videos in a Multi Auditory Environment Using Device Lending of GPUs
In The 7th International Conference on Multimedia Systems (MMSys). ACM, 2016.Status: Published
Efficient Processing of Videos in a Multi Auditory Environment Using Device Lending of GPUs
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | The 7th International Conference on Multimedia Systems (MMSys) |
Pagination | 36:1-36:4 |
Date Published | 05/2016 |
Publisher | ACM |
ISBN Number | 978-1-4503-4297-1 |
DOI | 10.1145/2910017.2910636 |
EIR - Efficient Computer Aided Diagnosis Framework for Gastrointestinal Endoscopies
In International Workshop on Content-based Multimedia Indexing. IEEE / ACM, 2016.Status: Published
EIR - Efficient Computer Aided Diagnosis Framework for Gastrointestinal Endoscopies
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Workshop on Content-based Multimedia Indexing |
Date Published | 06/2016 |
Publisher | IEEE / ACM |
Explorative Hyperbolic-Tree-Based Clustering Tool for Unsupervised Knowledge Discovery
In International Workshop on Content-based Multimedia Indexing. IEEE / ACM, 2016.Status: Published
Explorative Hyperbolic-Tree-Based Clustering Tool for Unsupervised Knowledge Discovery
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | International Workshop on Content-based Multimedia Indexing |
Date Published | 06/2016 |
Publisher | IEEE / ACM |
GPU-accelerated Real-time Gastrointestinal Diseases Detection
In CBMS 2016 : The 29th International Symposium on Computer-Based Medical Systems. IEEE, 2016.Status: Published
GPU-accelerated Real-time Gastrointestinal Diseases Detection
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | CBMS 2016 : The 29th International Symposium on Computer-Based Medical Systems |
Date Published | 07/2016 |
Publisher | IEEE |
Heimdallr: a dataset for sport analysis
In ACM Multimedia System. ACM, 2016.Status: Published
Heimdallr: a dataset for sport analysis
In this paper, we present Heimdallr, a dataset that aims to serve two different purposes. The first purpose is action recognition and pose estimation, which requires a dataset of annotated sequences of athlete skeletons. We employed a crowdsourcing platform where people around the world were asked to annotate frames and obtained more than 3000 fully annotated frames for 42 different sequences with a variety of poses and actions. The second purpose is an improved understanding of crowdworkers, and for this purpose, we collected over 10000 written feedbacks from 592 crowdworkers. This is valuable information for crowdsourcing researchers who explore algorithms for worker quality assessment. In addition to the complete dataset, we also provide the code for the application that has been used to collect the data as an open source software.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ACM Multimedia System |
Date Published | 05/2016 |
Publisher | ACM |
DOI |
High-Precision Power Modelling of the Tegra K1 Variable SMP Processor Architecture
In 10th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC). IEEE, 2016.Status: Published
High-Precision Power Modelling of the Tegra K1 Variable SMP Processor Architecture
Energy efficiency is an important issue for many embedded systems, where limited battery lifetime and power- hungry hardware constrain the usefulness of such devices. Modern Systems-on-Chip (SoCs) such as the Tegra K1 employ advanced power management capabilities such as two CPU clus- ters, clock-gating, power-gating and dynamic frequency tuning to meet application demands. At design or runtime phases, it is challenging for system architects and software developers to understand the effects that these mechanisms have in terms of power and performance in all parts of the system. This is especially because it is impossible to measure directly the power usage of cores, caches, memory and other hardware components. Rate-based power models are often proposed as a solution for this, but unfortunately these can mispredict substantially on the Tegra K1 up to 30 %. In this paper, we propose a power modelling method for the Tegra K1 CPU which overcomes the limitations of the most common types of models found in literature, but still only requires power measurement of the board. Through extensive empirical validation we demonstrate an accuracy which is close to 100 %. Through preliminary experiments we show that our methodology is able to capture instruction power of individual system processes and applications and produce detailed power breakdowns of all components in the system.
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | 10th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) |
Pagination | 193-200 |
Date Published | 09/2016 |
Publisher | IEEE |
DOI | 10.1109/MCSoC.2016.28 |
ImageCLEF 2017 LifeLog task
In Challenges in Machine Learning: Gaming and Education (CiML - NIPS 2016 workshop). Barcelona, Spain: NIPS, 2016.Status: Published
ImageCLEF 2017 LifeLog task
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Challenges in Machine Learning: Gaming and Education (CiML - NIPS 2016 workshop) |
Date Published | 11/2016 |
Publisher | NIPS |
Place Published | Barcelona, Spain |
Immersed gaming in Minecraft
In The 7th ACM International Conference on Multimedia System (MMSys 2016). ACM, 2016.Status: Published
Immersed gaming in Minecraft
This demonstration will showcase mixed reality technologies that we developed for a series of public art performances in Vienna in October 2015 in a collaboration of performance artists and researchers. The focus of the demonstration is on natural interaction techniques that can be used intuitively to control an avatar in a virtual 3D world. We combine virtual reality devices with optical location tracking, hand gesture recognition and smart devices. Conference attendees will be able to walk around in a Minecraft world by physically moving in the real world and to perform actions on virtual world items using hand gestures. They can also test our initial system for shared avatar control, in which a user in the real world cooperates with a user in the virtual world. Finally, attendees will have the opportunity to give us feedback about their experience with our system.
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | The 7th ACM International Conference on Multimedia System (MMSys 2016) |
Publisher | ACM |
DOI | 10.1145/2910017.2910632 |
LIRE - Open Source Visual Information Retrieval
In Multimedia System Conference 2016. New York: ACM, 2016.Status: Published
LIRE - Open Source Visual Information Retrieval
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia System Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Multimedia and Medicine: Teammates for Better Disease Detection and Survival
In ACM Multimedia. Amsterdam, The Netherlands, The Netherlands: ACM, 2016.Status: Published
Multimedia and Medicine: Teammates for Better Disease Detection and Survival
Afilliation | Communication Systems, Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ACM Multimedia |
Date Published | 10/2016 |
Publisher | ACM |
Place Published | Amsterdam, The Netherlands, The Netherlands |
DOI | 10.1145/2964284.2976760 |
OpenVQ - A Video Quality Assessment Toolkit
In ACM Multimedia. Amsterdam, The Netherlands, The Netherlands: ACM, 2016.Status: Published
OpenVQ - A Video Quality Assessment Toolkit
Afilliation | Communication Systems, Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ACM Multimedia |
Date Published | 10/2016 |
Publisher | ACM |
Place Published | Amsterdam, The Netherlands, The Netherlands |
DOI | 10.1145/2964284.2973800 |
Right inflight? A dataset for exploring the automatic prediction of movies suitable for a watching situation
In Multimedia Systems Conference 2016. New York: ACM, 2016.Status: Published
Right inflight? A dataset for exploring the automatic prediction of movies suitable for a watching situation
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Multimedia Systems Conference 2016 |
Date Published | 05/2016 |
Publisher | ACM |
Place Published | New York |
Robustness of 3D Point Positions to Camera Baselines in Markerless AR Systems
In The 7th ACM International Conference on Multimedia System (MMSys 2016). ACM, 2016.Status: Published
Robustness of 3D Point Positions to Camera Baselines in Markerless AR Systems
In the Augmented Reality (AR) applications, high quality relates to an accurate augmentation of virtual objects in the real scene. This can be accomplished only if the position of the observer is accurately known. This boils down to solving image-based location problem by an accurate camera pose (relative position and orientation) estimation, when a stereo or multiple camera setup is used. Consider a relevant appli- cation scenario as in a movie production set, where the di- rector is able to preview a scene as an integrated view of the real scene augmented with animated 3D models. The main camera shoots the scene, where as secondary stereo camera pair is used for image registration and localization. The di- rector can view the integrated preview from any viewpoint perfectly, as long as the camera pose estimation is accurate.
Moreover, in the case of a markerless AR system, the chal- lenge for camera pose estimation, is strongly influenced by the precision of detected feature correspondences between the images. Unfortunately, several of the state-of-art fea- ture extractors (detectors and descriptors) cannot guarantee a consistent accuracy of camera pose estimation, especially at varied camera baselines (viewpoints). As a consequence, the precise augmentation of objects, as desired in an AR application, is compromised. Hence, it becomes necessary to understand the magnitude of this error in relation to the camera baseline depending on the chosen feature extractors.
We, therefore, assess the quality of the position and the orientation of 3D reconstruction by evaluating 26 feature extractor combinations over 50 different camera baselines. To be directly relevant for AR applications, we evaluate by measuring the reconstruction error in 3D space, instead of re-projection error in 2D space. After the experiment, we have found the SIFT and KAZE feature extractors to be highly accurate and more robust to large camera baselines than others. Importantly, as a result of our study, we provide a recommendation for system builders to help them make a better choice of the feature extractor and/or the camera density required for their application.
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | The 7th ACM International Conference on Multimedia System (MMSys 2016) |
Publisher | ACM |
DOI | 10.1145/2910017.2910611 |
Simula @ MediaEval 2016 Context of Experience Task
In MediaEval 2016 Workshop. Hilversum, Netherlands: CEUR Workshop Proceedings, 2016.Status: Published
Simula @ MediaEval 2016 Context of Experience Task
This paper presents our approach for the Context of Multimedia Experience Task of the MediaEval 2016 Benchmark. We present different analyses of the given data using different subsets of data sources and combinations of it. Our approach gives a baseline evaluation indicating that metadata approaches work well but that also visual features can provide useful information for the given problem to solve.
Afilliation | Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | MediaEval 2016 Workshop |
Date Published | 10/2016 |
Publisher | CEUR Workshop Proceedings |
Place Published | Hilversum, Netherlands |
The MediaEval 2016 Context of Experience Task: Recommending Videos Suiting a Watching Situation
In MediaEval 2016 Workshop. Hilversum, Netherlands: CEUR Workshop Proceedings, 2016.Status: Published
The MediaEval 2016 Context of Experience Task: Recommending Videos Suiting a Watching Situation
In this paper we present an overview of the Context of Experience Task: recommending videos suiting a watching situation which is part of the MediaEval 2016 Benchmark. The aim of the task is to explore multimedia content that is watched under a certain situation. The scope of the this years task lies on movies watched during a flight. We hypothesize that users will have different preferences for movies that are watched during a flight compared to when a movie is watched at home or the cinema. This is most probably influenced by the context and the devices used to watch. In the case of being on a flight, the context is clearly different to normal situation (noise, compact, bad air) and also the devices differ (small screens, bad audio quality). The main goal of the task is to estimate if a person would like to watch a certain movie on the airplane or not. As dataset we provide a large collection of movies, collected from an airline, including pre-extracted visual, text and audio features.
Afilliation | Communication Systems, Communication Systems |
Project(s) | Efficient EONS: Execution of Large Workloads on Elastic Heterogeneous Resources |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | MediaEval 2016 Workshop |
Date Published | 10/2016 |
Publisher | CEUR Workshop Proceedings |
Place Published | Hilversum, Netherlands |
Edited books
Proceedings of the 8th International Workshop on Mobile Video (MoVid'16)
ACM Digital Library, 2016.Status: Published
Proceedings of the 8th International Workshop on Mobile Video (MoVid'16)
Afilliation | Communication Systems |
Project(s) | No Simula project |
Publication Type | Edited books |
Year of Publication | 2016 |
Publisher | ACM Digital Library |
ISBN Number | 978-1-4503-4356-5 |
Journal Article
Tiling in Interactive Panoramic Video: Approaches and Evaluation
IEEE Transactions on Multimedia 18, no. 9 (2016): 1819-1831.Status: Published
Tiling in Interactive Panoramic Video: Approaches and Evaluation
Interactive panoramic systems are currently on therise. However, one of the major challenges in such a system isthe overhead involved in transferring a full-quality panoramato the client when only a part of the panorama is used toextract a virtual view. Thus, such a system should maximize theuser experience while simultaneously minimizing the bandwidthrequired. In this paper, we apply tiling to deliver different qualitylevels for different parts of the panorama. Tiling has traditionallybeen applied to the delivery of very high-resolution content toclients. Here, we apply similar ideas in a real-time interactivepanoramic video system. A major challenge lies in the movementof such a virtual view, for which clients’ regions of interestchange dynamically and independently from each other. Weshow that our algorithms, which progressively increase in qualitytowards the point of the view, manage to (i) reduce the bandwidthrequirement and (ii) provide a similar QoE compared to a fullpanorama system.
Afilliation | Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | IEEE Transactions on Multimedia |
Volume | 18 |
Issue | 9 |
Pagination | 1819-1831 |
Publisher | IEEE |
Proceedings, refereed
A Logical Memory Model for Scaling Parallel Multimedia Workloads
In Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). New York, NY, USA: ACM, 2015.Status: Published
A Logical Memory Model for Scaling Parallel Multimedia Workloads
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) |
Pagination | 49-54 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-3352-8 |
Keywords | cache, coherency, cpu, false sharing, multicore, programming, scalability |
URL | http://doi.acm.org/10.1145/2736084.2736096 |
DOI | 10.1145/2736084.2736096 |
Energy Efficient Continuous Multimedia Processing Using the Tegra K1 Mobile SoC
In Proceedings of the 7th ACM International Workshop on Mobile Video (MoVid). ACM, 2015.Status: Published
Energy Efficient Continuous Multimedia Processing Using the Tegra K1 Mobile SoC
Energy consumption is an important issue for mobile devices, as the technological development in battery technology has not kept pace with the power requirements of mobile hardware. In this paper, we use a video rotation filter to study the efects of CPU and GPU frequency scaling in terms of performance and energy. Our platform is the Tegra K1 mobile processor with a quad-core CPU and a CUDA capable GPU. We find that most energy can be saved by minimising CPU frequency while meeting the filter’s framerate requirement. Interestingly, the frequency scaling affects GPUs differently, where the best frequency is always moderately higher than the minimum which meets the framerate requirement. Using these heuristics, it is possible to save up to 10 % energy compared to the standard Linux frequency scaling algorithms, which use processor utilisation to adjust processor frequency.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 7th ACM International Workshop on Mobile Video (MoVid) |
Pagination | 15-16 |
Date Published | 03/2015 |
Publisher | ACM |
ISBN Number | 978-1-4503-3353-5 |
URL | http://dl.acm.org/citation.cfm?id=2727044 |
DOI | 10.1145/2727040.2727044 |
Energy Efficient Video Encoding Using the Tegra K1 Mobile Processor [Demo Paper]
In Proceedings of the 6th ACM Multimedia Systems Conference (MMSys). ACM, 2015.Status: Published
Energy Efficient Video Encoding Using the Tegra K1 Mobile Processor [Demo Paper]
Energy consumption is an important concern for mobile devices, where the evolution in battery storage capacity has not followed the power usage requirements of modern hardware. However, innovative and flexible hardware platforms give developers better means of optimising the energy consumption of their software. For example, the Tegra K1 System-on-Chip (SoC) offers two CPU clusters, GPU offloading, frequency scaling and other mechanisms to control the power and performance of applications. In this demonstration, the scenario is live video encoding, and participants can experiment with power usage and performance using the Tegra K1’s hardware capabilities. A popular power-saving approach is a “race to sleep” strategy where the highest CPU frequency is used while the CPU has work to do, and then the CPU is put to sleep. Our own experiments indicate that an energy reduction of 28 % can be achieved by running the video encoder on the lowest CPU frequency at which the platform achieves an encoding frame rate equal to the minimum frame rate of 25 Frames Per Second (FPS).
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 6th ACM Multimedia Systems Conference (MMSys) |
Date Published | 03/2015 |
Publisher | ACM |
ISBN Number | 978-1-4503-3351-1 |
URL | http://dl.acm.org/citation.cfm?id=2713186 |
DOI | 10.1145/2713168.2713186 |
Expert Driven Semi-Supervised Elucidation Tool for Medical Endoscopic Videos
In Proceedings of the 6th ACM Multimedia Systems Conference. ACM, 2015.Status: Published
Expert Driven Semi-Supervised Elucidation Tool for Medical Endoscopic Videos
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 6th ACM Multimedia Systems Conference |
Date Published | 03/2015 |
Publisher | ACM |
Exploitation of Producer Intent in Relation to Bandwidth and QoE for Online Video Streaming Services
In NOSSDAV at Multimedia Systems Conference . Portland, USA: ACM, 2015.Status: Published
Exploitation of Producer Intent in Relation to Bandwidth and QoE for Online Video Streaming Services
This paper is the product of recent advances in research on users’ intent during multimedia content retrieval. Our goal is to save bandwidth while streaming video clips from a browsable on-demand service, while maintaining or even improving the users’ quality of experience (QoE). Understanding user intent allows us to predict whether streaming a particular video in a low quality constitutes a reduced QoE for a user. However, many VoD streaming services today are used by users for a wide variety of reasons, meaning that user intent cannot be inferred from their use of the service alone. However, our investigation demonstrates that user intent does in most cases coincide with producer intent. We can also demonstrate that the latter can be inferred from the content itself as well as associated metadata. By transitivity, we can choose a default video quality that satisfies the users QoE in the majority of cases.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | NOSSDAV at Multimedia Systems Conference |
Date Published | 03/2015 |
Publisher | ACM |
Place Published | Portland, USA |
Introduction to a Task on Context of Experience: Recommending Videos Suiting a Watching Situation
In MediaEval Benchmarking Initiative for Multimedia Evaluation. Wurzen, Germany: CEUR Workshop Proceedings, 2015.Status: Published
Introduction to a Task on Context of Experience: Recommending Videos Suiting a Watching Situation
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | MediaEval Benchmarking Initiative for Multimedia Evaluation |
Date Published | 09/2015 |
Publisher | CEUR Workshop Proceedings |
Place Published | Wurzen, Germany |
Latency and Fairness Trade-Off for Thin Streams Using Redundant Data Bundling in TCP
In IEEE 40th Conference on Local Computer Networks (LCN). Clearwater Beach, FL, USA: IEEE, 2015.Status: Published
Latency and Fairness Trade-Off for Thin Streams Using Redundant Data Bundling in TCP
Time-dependent applications using TCP often send thin-stream traffic, characterised by small packets and high inter-transmission-times. Retransmissions after packet loss can result in very high delays for such flows as they often cannot trigger fast retransmit. Redundant Data Bundling is a mechanism that preempts the experience of loss for a flow by piggybacking unacknowledged segments with new data as long as the total packet size is lower than the flow maximum segment size. Although successful at reducing retransmission latency, this mechanism had design issues leaving it open for abuse, effectively making it unsuitable for general Internet deployment. In this paper, we have redesigned the RDB mechanism to make it safe for deployment. We improve the trigger for when to apply it and evaluate its fairness towards competing traffic. Extensive experimental results confirm that our proposed modifications allows for inter-flow fairness while maintaining the significant latency reductions from the original RDB mechanism.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | IEEE 40th Conference on Local Computer Networks (LCN) |
Pagination | 287-294 |
Date Published | 10/2015 |
Publisher | IEEE |
Place Published | Clearwater Beach, FL, USA |
Keywords | communication, latency, networks, thin streams |
DOI | 10.1109/LCN.2015.7366322 |
Online Re-calibration for Robust 3D Measurement Using Single Camera-PantoInspect Train Monitoring System
In Proceedings of the International Conference on Computer Vision Systems (ICVS) - Lecture Notes in Computer Science Volume 9163. Lecture Notes in Computer Science, Springer.Verlag, 2015.Status: Published
Online Re-calibration for Robust 3D Measurement Using Single Camera-PantoInspect Train Monitoring System
Vision-based inspection systems measures defects accurately with the help of a checkerboard calibration (CBC) method. However, the 3D measurements of such systems are prone to errors, caused by physical misalignment of the object-of-interest and noisy image data. The PantoInspect Train Monitoring System (PTMS), is one such system that inspects defects on pantographs mounted on top of the electric trains. In PTMS, the measurement errors can compromise railway safety. Although this problem can be solved by re-calibrating the cameras, the process involves manual intervention leading to large servicing times. Therefore, in this paper, we propose Feature Based Calibration (FBC) in place of CBC, to cater an obvious need for online re-calibration that enhances the usability of the system. FBC involves feature extraction, pose estimation, back-projection of defect points and estimation of 3D measurements. We explore four state-of-the-art pose estimation algorithms in FBC using very few feature points.
This paper evaluates and discusses the performance of FBC and its robustness against practical problems, in comparison to CBC. As a result, we identify the best FBC algorithm type and operational scheme for PTMS. In conclusion, we show that, by adopting FBC in PTMS and other related 3D systems, better performance and robustness can be achieved compared to CBC.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the International Conference on Computer Vision Systems (ICVS) - Lecture Notes in Computer Science Volume 9163 |
Publisher | Lecture Notes in Computer Science, Springer.Verlag |
DOI | 10.1007/978-3-319-20904-3_45 |
Playing Around the Eye Tracker - A Serious Game Based Dataset
In Second International Workshop on Gamification for Information Retrieval (GamifIR’15) . Vienna, Austria: CEUR Workshop Proceedings, 2015.Status: Published
Playing Around the Eye Tracker - A Serious Game Based Dataset
In this paper we present a dataset of visual saliency segments
associated to social images, collected through a mobile game
in conjunction with a crowdsourcing campaign. Our data
are collected in the game itself, where players guess what
is depicted in a gradually uncovered image. The game mechanics
allow us to collect information about which image
segments are required by the user to correctly recover the
image content. The provided dataset can be applied to both,
computer vision and image retrieval algorithms, and it aims
to contribute to the understanding of human visual perception
and attention. Moreover, the end objective is to test the
game as a potential substitute to professional eye tracking
systems.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Second International Workshop on Gamification for Information Retrieval (GamifIR’15) |
Date Published | 03/2015 |
Publisher | CEUR Workshop Proceedings |
Place Published | Vienna, Austria |
Scaling Virtual Camera Services to a Large Number of Users
In Proceedings of the 6th annual ACM conference on Multimedia Systems (MMSYS). New York, NY, USA: ACM, 2015.Status: Published
Scaling Virtual Camera Services to a Large Number of Users
By processing video footage from a camera array, one can easily make wide-field-of-view panorama videos. From the single panorama video, one can further generate multiple virtual cameras supporting personalized views to a large number of users based on only the few physical cameras in the array. However, giving personalized services to large numbers of users potentially introduces both bandwidth and processing bottlenecks, depending on where the virtual camera is processed.
In this demonstration, we present a system that address the large cost of transmitting entire panorama video to the end-user where the user creates the virtual views on the client device. Our approach is to divide the panorama into tiles, each encoded in multiple qualities. Then, the panorama video tiles are retrieved by the client in a quality (and thus bit rate) depending on where the virtual camera is pointing, i.e., the video quality of the tile changes dynamically according to the user interaction. Our initial experiments indicate that there is a large potential of saving bandwidth on the cost of trading quality of in areas of the panorama frame not used for the extraction of the virtual view.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 6th annual ACM conference on Multimedia Systems (MMSYS) |
Pagination | 93-96 |
Date Published | 03/2015 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-3351-1 |
TADA: An Active Measurement Tool for Automatic Detection of AQM
In Valuetools 2015, Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools, 2015.Status: Published
TADA: An Active Measurement Tool for Automatic Detection of AQM
Afilliation | Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Valuetools 2015, Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools |
Tiling of Panorama Video for Interactive Virtual Cameras: Overheads and Potential Bandwidth Requirement Reduction
In 21st International Packet Video Workshop (PV 2015). IEEE, 2015.Status: Published
Tiling of Panorama Video for Interactive Virtual Cameras: Overheads and Potential Bandwidth Requirement Reduction
Delivering high resolution, high bitrate panorama
video to a large number of users introduces huge scaling
challenges. To reduce the resource requirement, researchers have
earlier proposed tiling in order to deliver different qualities in
different spatial parts of the video. In our work, providing an
interactive moving virtual camera to each user, tiling may be used
to reduce the quality depending on the position of the virtual
view. This raises new challenges compared to existing tiling
approaches as the need for high quality tiles dynamically change.
In this paper, we describe a tiling approach of panorama video
for interactive virtual cameras where we provide initial results
showing the introduced overheads and the potential reduction in
bandwidth requirement.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | 21st International Packet Video Workshop (PV 2015) |
Date Published | 06/2015 |
Publisher | IEEE |
Why Design Matters - Crowdsourcing of Complex Tasks
In ACM CrowdMM. Brisbane, Australia: ACM, 2015.Status: Published
Why Design Matters - Crowdsourcing of Complex Tasks
In this paper, we show how the power of the crowd can be used
to do complex tasks using human pose estimation as a use case.
Crowdsourcing tasks are usually not very complex and easy to
solve by workers that are not experienced in a certain topic. For
more complex tasks, it is general recommended to use experienced
workers or experts. However, we show that tasks can also be more
complex for non-expert workers and that they produce data that is
close to what experts would report. Therefore, a detailed description
of the crowdsourcing campaign, the methods applied and the
results will be given, and we discuss how the obtained data can be
used.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | ACM CrowdMM |
Date Published | 10/2015 |
Publisher | ACM |
Place Published | Brisbane, Australia |
Keywords | Annotation, Crowdsourcing, Multimedia |
DOI | 10.1145/2810188.2810190 |
Why Race-to-Finish is Energy-Inefficient for Continuous Multimedia Workloads
In Proceedings of the 9th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC). IEEE, 2015.Status: Published
Why Race-to-Finish is Energy-Inefficient for Continuous Multimedia Workloads
It is often believed that a "race-to-finish" approach, where processing is finished quickly, is the best way to conserve energy on modern mobile architectures. However, from earlier work we know that for continuous multimedia workloads, the best way to conserve energy is to minimise processor frequency such that application deadlines are met. In this paper, we investigate the reasons behind this. We develop an original method to model dynamic and static power on individual power rails of the Tegra K1 by only measuring the total power usage of the board. Our model has an average error of only 8 %. We find that the way an application scales performance with frequency is very important for energy efficiency. We demonstrate a 37 % energy saving by minimising processor and memory frequency of a video processing filter such that a framerate of 20 FPS is met.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 9th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) |
Pagination | 57-64 |
Date Published | 09/2015 |
Publisher | IEEE |
URL | http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7328187 |
DOI | 10.1109/MCSoC.2015.20 |
Journal Article
An Experimental Evaluation of Debayering Algorithms on GPUs for Recording Panoramic Video in Real-time
International Journal of Multimedia Data Engineering and Management (IJMDEM) 6, no. 3 (2015): 1-16.Status: Published
An Experimental Evaluation of Debayering Algorithms on GPUs for Recording Panoramic Video in Real-time
Modern video cameras often only capture a single color per pixel in a single pass operation. This process is called ltering, where pixels are ltered through a color lter array, and the Bayer filter is perhaps the most common filter used today. This means that we must restore the missing color channels in the image or the video frame in a post-processing step, i.e., a process referred to as debayering. In a live video scenario, this operation must be performed eciently in order to output each video frame in real-time, while also yielding acceptable visual quality. Here, we evaluate debayering algorithms implemented on a GPU for real-time panoramic video recordings using multiple 2K-resolution cameras.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | International Journal of Multimedia Data Engineering and Management (IJMDEM) |
Volume | 6 |
Issue | 3 |
Pagination | 1-16 |
Date Published | 07/2015 |
Publisher | IGI Global |
ISSN | 1947-8534 |
DOI | 10.4018/ijmdem.2015070101 |
Audiovisual robustness: Exploring perceptual tolerance to asynchrony and quality distortion
Multimedia Tools and Applications 74, no. 2 (2015): 345-365.Status: Published
Audiovisual robustness: Exploring perceptual tolerance to asynchrony and quality distortion
Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Multimedia Tools and Applications |
Volume | 74 |
Issue | 2 |
Pagination | 345-365 |
Publisher | Springer |
Keywords | Workshop |
Cache-Centric Video Recommendation: An Approach to Improve the Efficiency of YouTube Caches
ACM Trans. Multimedia Comput. Commun. Appl. 11, no. 4 (2015): 1-20.Status: Published
Cache-Centric Video Recommendation: An Approach to Improve the Efficiency of YouTube Caches
In this article, we take advantage of the user behavior of requesting videos from the top of the related list provided by YouTube to improve the performance of YouTube caches. We recommend that local caches reorder the related lists associated with YouTube videos, presenting the cached content above noncached content. We argue that the likelihood that viewers select content from the top of the related list is higher than selection from the bottom, and pushing contents already in the cache to the top of the related list would increase the likelihood of choosing cached content. To verify that the position on the list really is the selection criterion more dominant than the content itself, we conduct a user study with 40 YouTube-using volunteers who were presented with random related lists in their everyday YouTube use. After confirming our assumption, we analyze the benefits of our approach by an investigation that is based on two traces collected from a university campus. Our analysis shows that the proposed reordering approach for related lists would lead to a 2 to 5 times increase in cache hit rate compared to an approach without reordering the related list. This increase in hit rate would lead to reduction in server load and backend bandwidth usage, which in turn reduces the latency in streaming the video requested by the viewer and has the potential to improve the overall performance of YouTube's content distribution system. An analysis of YouTube's recommendation system reveals that related lists are created from a small pool of videos, which increases the potential for caching content from related lists and reordering based on the content in the cache.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | ACM Trans. Multimedia Comput. Commun. Appl. |
Volume | 11 |
Issue | 4 |
Pagination | 1--20 |
Date Published | 06/2015 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISSN | 1551-6857 |
Keywords | Caching, recommendation, YouTube |
URL | http://doi.acm.org/10.1145/2716310 |
DOI | 10.1145/2716310 |
The Cameraman Operating My Virtual Camera Is Artificial: Can The Machine Be As Good As A Human?
ACM Transactions on Multimedia Computing, Communications and Applications 11, no. 4 (2015): 56:1-56:20.Status: Published
The Cameraman Operating My Virtual Camera Is Artificial: Can The Machine Be As Good As A Human?
In this paper, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | ACM Transactions on Multimedia Computing, Communications and Applications |
Volume | 11 |
Issue | 4 |
Pagination | 56:1--56:20 |
Date Published | 06/2015 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISSN | 1551-6857 |
URL | http://doi.acm.org/10.1145/2744411 |
DOI | 10.1145/2744411 |
Using a Commodity Hardware Video Encoder for Interactive Applications
International Journal of Multimedia Data Engineering and Management (IJMDEM) 6, no. 3 (2015): 17-31.Status: Published
Using a Commodity Hardware Video Encoder for Interactive Applications
Over the last years, video streaming has become one of the most dominant Internet services. Due to the increased availability of high-speed Internet access, multimedia services are becoming more interactive. Examples of such applications are both cloud gaming and systems where users can interact with high-resolution content. During the last few years, programmable hardware video encoders have been built into commodity hardware such as CPUs and GPUs. We evaluate one of these encoders in a scenario where we have individual streams delivered to the end users. Our results show that the visual video quality and the frame size of the hardware-based encoder are comparable to a software-based approach. To evaluate a complete system, we have implemented our proposed streaming pipeline into Quake III. We found that running the game on a remote server and streaming the video output to a client web browser located in a typical home environment is possible and enjoyable. The interaction latency is measured to be less than 90 ms, which is below what is reported for OnLive in a similar environment
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | International Journal of Multimedia Data Engineering and Management (IJMDEM) |
Volume | 6 |
Issue | 3 |
Pagination | 17-31 |
Date Published | 07/2015 |
Publisher | IGI Global |
ISSN | 1947-8534 |
DOI | 10.4018/ijmdem.2015070102 |
Poster
Energy and Performance Optimization of a Simple Video Encoder on the Jetson-TK1
In GPU Technology Conference 2015. Nvidia, 2015.Status: Published
Energy and Performance Optimization of a Simple Video Encoder on the Jetson-TK1
This poster analyses the energy consumption of a simple video encoder running on NVIDIA's Tegra K1 processor. The total energy consumption of the video encoder is investigated under the influence of different hardware configurations, such as which processors (CPU clusters or GPU) are used, DVFS algorithms, and whether performance optimisations like NEON are implemented. We find that NEON instructions and multithreading generally have positive effects on energy consumption, saving between 25 to 40 % energy compared to a nonoptimised, naive implementation. GPU offloading is found to be marginally better than CPU execution by an amount of 1.7 %.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Poster |
Year of Publication | 2015 |
Secondary Title | GPU Technology Conference 2015 |
Date Published | 03/2015 |
Publisher | Nvidia |
URL | http://on-demand.gputechconf.com/gtc/2015/posters/GTC_2015_Embedded_Syst... |
Edited books
Proceedings of the 7th ACM International Workshop on Mobile Video (MOVID'15)
ACM: ACM, 2015.Status: Published
Proceedings of the 7th ACM International Workshop on Mobile Video (MOVID'15)
We welcome you to the seventh ACM Workshop on Mobile Video (MoVid), held in conjunction with ACM Multimedia Systems (MMSys) 2015 Conference on March 18-20, 2015 in Portland, Oregon, USA. The workshop is sponsored by ACM SIGMM and in cooperation with SIGOPS and SIGCOMM.
The mission of the MoVid workshop series is to deepen the understanding of research and deployment challenges in building the Next Generation Mobile Video technologies. The MoVid workshop series aims to address multiple aspects of delivering visually-rich applications to mobile users. This year's workshop continues this tradition of presentation of novel research results and experimental reports on cutting edge issues of mobile video applications. MoVid 2015 gives researchers and practitioners a unique opportunity to share their perspectives with others who are interested in various aspects of mobile multimedia. In this respect, MoVid 2015 features a half-day workshop with a keynote, two regular paper sessions and a joint panel with NOSSDAV.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Edited books |
Year of Publication | 2015 |
Publisher | ACM |
Place Published | ACM |
ISBN Number | 978-1-4503-3353-5 |
URL | http:://dl.acm.org/citation.cfm?id=2727040&coll=DL&dl=GUIDE&CFID=4916045... |
Journal Article
A correction to Anderssonʼs fusion tree construction
Theoretical Computer Science 520 (2014): 130-132.Status: Published
A correction to Anderssonʼs fusion tree construction
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | Theoretical Computer Science |
Volume | 520 |
Pagination | 130–132 |
Publisher | Elsevier |
ISSN | 03043975 |
URL | http://linkinghub.elsevier.com/retrieve/pii/S0304397513007202 |
DOI | 10.1016/j.tcs.2013.09.028 |
Adaptive Media Streaming to Mobile Devices: Challenges, Enhancements, and Recommendations
Advances in Multimedia 2014 (2014): 1-21.Status: Published
Adaptive Media Streaming to Mobile Devices: Challenges, Enhancements, and Recommendations
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | Advances in Multimedia |
Volume | 2014 |
Pagination | 1–21 |
Publisher | Hindawi |
ISSN | 1687-5680 |
URL | http://www.hindawi.com/journals/am/2014/805852/ |
DOI | 10.1155/2014/805852 |
Bagadus: an Integrated Real-Time System for Soccer Analytics
ACM Transactions on Multimedia Computing, Communications, and Applications 10 (2014): 14:1-14:21.Status: Published
Bagadus: an Integrated Real-Time System for Soccer Analytics
The importance of winning has increased the role of performance analysis in the sports industry, and this underscores how statistics and technology keep changing the way sports are played. Thus, this is a growing area of interest, both from a computer system view in managing the technical challenges and from a sport performance view in aiding the development of athletes. In this respect, Bagadus is a real-time prototype of a sports analytics application using soccer as a case study. Bagadus integrates a sensor system, a soccer analytics annotations system, and a video processing system using a video camera array. A prototype is currently installed at Alfheim Stadium in Norway, and in this article, we describe how the system can be used in real-time to playback events. The system supports both stitched panorama video and camera switching modes and creates video summaries based on queries to the sensor system. Moreover, we evaluate the system from a systems point of view, benchmarking different approaches, algorithms, and trade-offs, and show how the system runs in real time.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | ACM Transactions on Multimedia Computing, Communications, and Applications |
Volume | 10 |
Number | 1s |
Pagination | 14:1-14:21 |
Date Published | January |
DOI | 10.1145/2541011 |
Bagadus: An integrated real-time system for soccer analytics
ACM Transactions on Multimedia Computing, Communications, and Applications 10 (2014): 1-21.Status: Published
Bagadus: An integrated real-time system for soccer analytics
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | ACM Transactions on Multimedia Computing, Communications, and Applications |
Volume | 10 |
Pagination | 1–21 |
Publisher | ACM |
ISSN | 15516857 |
URL | http://dl.acm.org/citation.cfm?doid=2576908.2541011 |
DOI | 10.1145/2541011 |
Processing Panorama Video in Real-time
International Journal of Semantic Computing 08 (2014): 209-227.Status: Published
Processing Panorama Video in Real-time
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | International Journal of Semantic Computing |
Volume | 08 |
Pagination | 209–227 |
Publisher | World Scientific |
ISSN | 1793-351X |
URL | http://www.worldscientific.com/doi/abs/10.1142/S1793351X14400054 |
DOI | 10.1142/S1793351X14400054 |
Processing Panorama Video in Real-Time
International Journal of Semantic Computing 8 (2014): 209-227.Status: Published
Processing Panorama Video in Real-Time
There are many scenarios where high resolution, wide field of view video is useful. Such panorama video may be generated using camera arrays where the feeds from multiple cameras pointing at different parts of the captured area are stitched together. However, processing the different steps of a panorama video pipeline in real-time is challenging due to the high data rates and the stringent timeliness requirements. In our research, we use panorama video in a sport analysis system called Bagadus. This system is deployed at Alfheim stadium in Tromsø, and due to live usage, the video events must be generated in real-time. In this paper, we describe our real-time panorama system built using a low-cost CCD HD video camera array. We describe how we have implemented different components and evaluated alternatives. The performance results from experiments ran on commodity hardware with and without co-processors like graphics processing units (GPUs) show that the entire pipeline is able to run in real-time.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | International Journal of Semantic Computing |
Volume | 8 |
Number | 2 |
Pagination | 209-227 |
Date Published | September |
DOI | 10.1142/S1793351X14400054 |
Proceedings, refereed
An Evaluation of Debayering Algorithms on GPU for Real-Time Panoramic Video Recording
In IEEE International Symposium on Multimedia (ISM 2014). IEEE, 2014.Status: Published
An Evaluation of Debayering Algorithms on GPU for Real-Time Panoramic Video Recording
Modern video cameras normally only capture a single color per pixel, commonly arranged in a Bayer pattern. This means that we must restore the missing color channels in the image or the video frame in post-processing, a process referred to as debayering. In a live video scenario, this operation must be performed efficiently in order to output each frame in real-time, while also yielding acceptable visual quality. Here, we evaluate debayering algorithms implemented on a GPU for real-time panoramic video recordings using multiple 2K-resolution cameras.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | IEEE International Symposium on Multimedia (ISM 2014) |
Publisher | IEEE |
Analysis of SR ARQ Delays Using Data-Bundling Over Markov Channels
In The Nineteenth IEEE Symposium on Computers and Commuications (IEEE ISCC). IEEE, 2014.Status: Published
Analysis of SR ARQ Delays Using Data-Bundling Over Markov Channels
Data-bundling is a useful technique that decreases the delivery delay of packet streams when they are transmitted over noisy channels and are subject to retransmission-based error control. In this paper, we investigate the packet delay statistics for a fully reliable selective repeat automatic repeat request (SR ARQ) where a data-bundling mechanism is employed. In more detail, we discuss a model for data-bundling to analyze the SR ARQ mechanism over wireless channels based on Markov chains. We evaluate various channel error distributions and analyze the buffer occupancy to check if the data-bundling mechanism provides efficient results. We further analyze the queueing, delivery and overall delay statistics at link layer. We found that using data-bundling can improve the delay performance of the SR ARQ mechanism, especially when bursty channels with heavily correlated errors are considered. Thus, this technique can bring useful improvements for real-time services, multimedia, and other delay-sensitive applications over wireless networks.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | The Nineteenth IEEE Symposium on Computers and Commuications (IEEE ISCC) |
Date Published | June |
Publisher | IEEE |
Keywords | Conference |
Automatic event extraction and video summaries from soccer games
In Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys '14. New York, New York, USA: ACM Press, 2014.Status: Published
Automatic event extraction and video summaries from soccer games
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys '14 |
Pagination | 176–179 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450327053 |
URL | http://dl.acm.org/citation.cfm?doid=2557642.2579374 |
DOI | 10.1145/2557642.2579374 |
Automatic Event Extraction and Video Summaries From Soccer Games
In Proceedings of the 5th annual ACM conference on Multimedia Systems (MMSYS). ACM, 2014.Status: Published
Automatic Event Extraction and Video Summaries From Soccer Games
Bagadus is a prototype of a soccer analysis application which integrates a sensor system, a video camera array and soccer analytics annotations. The current prototype is installed at Alfheim Stadium in Norway, and provides a large set of new functions compared to existing solutions. One important feature is to automatically extract video events and sum- maries from the games, i.e., an operation that traditionally consumes a huge amount of time. In this demo, we demon- strate how our integration of subsystems enable several types of summaries to be generated automatically, and we show that the video summaries are displayed with a response time around one second.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the 5th annual ACM conference on Multimedia Systems (MMSYS) |
Date Published | March |
Publisher | ACM |
DOI | 10.1145/2557642.2579374 |
Automatic Exposure for Panoramic Systems in Uncontrolled Lighting Conditions: a Football Stadium Case Study
In The Engineering Reality of Virtual Reality. Proceedings of SPIE/IS&T Electronic Imaging. SPIE, 2014.Status: Published
Automatic Exposure for Panoramic Systems in Uncontrolled Lighting Conditions: a Football Stadium Case Study
One of the most common ways of capturing wide field-of-view scenes like arena sports is by capturing panoramic videos. Several experiments were conducted in order to devise an approach for driving a panorama video capture system in a completely automatic fashion. The target scene is an outdoor football stadium, where the light is uncontrolled and widely varying owing to the weather. This paper provides in detail various attempted approaches and the outcomes. Several papers propose global color correction for adjusting the colors, yet practically such a correction introduces new artifacts into the video streams. It is observed that visually pleasing results are obtaining by appropriate synchronization of exposure settings.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | The Engineering Reality of Virtual Reality |
Publisher | SPIE |
Keywords | Conference |
Automatic Real-Time Zooming and Panning on Salient Objects from a Panoramic Video
In Proceedings of the ACM International Conference on Multimedia - MM '14. New York, New York, USA: ACM Press, 2014.Status: Published
Automatic Real-Time Zooming and Panning on Salient Objects from a Panoramic Video
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the ACM International Conference on Multimedia - MM '14 |
Pagination | 725–726 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450330633 |
URL | http://dl.acm.org/citation.cfm?doid=2647868.2654882 |
DOI | 10.1145/2647868.2654882 |
Automatic Real-Time Zooming and Panning on Salient Objects From a Panoramic Video
In ACM International Conference on Multimedia. ACM, 2014.Status: Published
Automatic Real-Time Zooming and Panning on Salient Objects From a Panoramic Video
The proposed demo shows how our system automatically zooms and pans into tracked objects in panorama videos. At the conference site, we will set up a two-camera version of the system, generating live panorama videos, where the system zooms and pans tracking people using colored hats. Additionally, using a stored soccer game video from a five 2K camera setup at Alfheim stadium in Tromso from the European league game between Tromsø IL and Tottenham Hotspurs, the system automatically follows the ball.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | ACM International Conference on Multimedia |
Date Published | November |
Publisher | ACM |
Keywords | Conference |
Be your own cameraman: real-time support for zooming and panning into stored and live panoramic video
In Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys '14. New York, New York, USA: ACM Press, 2014.Status: Published
Be your own cameraman: real-time support for zooming and panning into stored and live panoramic video
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys '14 |
Pagination | 168–171 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450327053 |
URL | http://dl.acm.org/citation.cfm?doid=2557642.2579370 |
DOI | 10.1145/2557642.2579370 |
Be Your Own Cameraman: Real-Time Support for Zooming and Panning Into Stored and Live Panoramic Video
In Proceedings of the 5th annual ACM conference on Multimedia Systems (MMSYS). ACM, 2014.Status: Published
Be Your Own Cameraman: Real-Time Support for Zooming and Panning Into Stored and Live Panoramic Video
High-resolution panoramic video with a wide field-of-view is popular in many contexts. However, in many examples, like surveillance and sports, it is often desirable to zoom and pan into the generated video. A challenge in this respect is real-time support, but in this demo, we present an end-to-end real-time panorama system with interactive zoom and panning. Our system installed at Alfheim stadium, a Norwegian premier league soccer team, generates a cylindrical panorama from five 2K cameras live where the perspective is corrected in real-time when presented to the client. This gives a better and more natural zoom compared to existing systems using perspective panoramas and zoom operations using plain crop. Our experimental results indicate that virtual views can be generated far below the frame-rate threshold, i.e., on a GPU, the processing requirement per frame is about 10\~milliseconds. The proposed demo lets participants interactively zoom and pan into stored panorama videos generated at Alfheim stadium and from a live 2-camera array on-site.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the 5th annual ACM conference on Multimedia Systems (MMSYS) |
Date Published | March |
Publisher | ACM |
DOI | 10.1145/2557642.2579370 |
Event Understanding in Endoscopic Surgery Videos
In HuEvent '14: Proceedings of the 1st ACM International Workshop on Human Centered Event Understanding from Multimedia. New York, NY, USA: ACM, 2014.Status: Published
Event Understanding in Endoscopic Surgery Videos
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | HuEvent '14: Proceedings of the 1st ACM International Workshop on Human Centered Event Understanding from Multimedia |
Pagination | 17–22 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-3120-3 |
DOI | 10.1145/2660505.2660509 |
Interactive Zoom and Panning from Live Panoramic Video
In Proceedings of Network and Operating System Support on Digital Audio and Video Workshop. New York, NY, USA: ACM, 2014.Status: Published
Interactive Zoom and Panning from Live Panoramic Video
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of Network and Operating System Support on Digital Audio and Video Workshop |
Pagination | 19:19––19:24 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-2706-0 |
Keywords | panning, panorama video, real-time, zoom |
URL | http://doi.acm.org/10.1145/2578260.2578264 |
DOI | 10.1145/2578260.2578264 |
Interactive Zoom and Panning From Live Panoramic Video
In Proceeding of the 24th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). New York, NY, USA: ACM, 2014.Status: Published
Interactive Zoom and Panning From Live Panoramic Video
Panorama video is becoming increasingly popular, and we present an end-to-end real-time system to interactively zoom and pan into high-resolution panoramic videos. Compared to existing systems using perspective panoramas with cropping, our approach creates a cylindrical panorama. Here, the perspective is corrected in real-time, and the result is a better and more natural zoom. Our experimental results also indicate that such zoomed virtual views can be generated far below the frame-rate threshold. Taking into account recent trends in device development, our approach should be able to scale to a large number of concurrent users in the near future.
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceeding of the 24th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) |
Publisher | ACM |
Place Published | New York, NY, USA |
Keywords | Workshop |
Real-Time HDR Panorama Video
In Proceedings of the ACM International Conference on Multimedia - MM '14. New York, New York, USA: ACM Press, 2014.Status: Published
Real-Time HDR Panorama Video
Afilliation | Communication Systems, Communication Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the ACM International Conference on Multimedia - MM '14 |
Pagination | 1205–1208 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450330633 |
URL | http://dl.acm.org/citation.cfm?doid=2647868.2655049 |
DOI | 10.1145/2647868.2655049 |