Department of Holistic Systems
Today, everyone is talking about big-data analytics, video streaming, machine learning, web search, mobile apps, etc., and we all take it for granted that these "services" work perfectly. However, underneath such applications there are large systems, and individual components of these systems are often “only” a part of a big ecosystem of integrated building blocks needed to enable a functional service. In this respect, the HOST department addresses challenges potentially covering all components of the entire system, from data creation to visualisation of the results.
Using a combination of basic and applied research, we are targeting numerous applications, many in the field of sports and medicine, where we have a holistic view of the system and perform basic research, do experimental prototyping and run experiments in the intended environments.
Currently, machine learning is an important component of our research where we for example, in real- time, aim to analyse athlete performance and detect diseases in medical videos. In our holistic systems view, not only the accuracy of the machine learning analysis is of importance, but also the complete pipe-line integration and the system performance (e.g., resource consumption and scalability). We architect complete systems and optimise for particular application requirements, both functional and non-functional, in order to provide a best possible quality of the service and a lowest possible resource consumption. Finally, we have a goal to put the results into use for the society where we contribute to open source projects and have spun of new industries
People at Department of Holistic Systems
Who we are?
Simula Metropolitan employees are researchers, postdoctoral fellows, PhD students, engineers and administrative people. We are from all over the world, ranging from newly educated to experienced researchers, all working on making research in digital engineering at the highest international level possible.
Projects at Department of Holistic Systems
Interview training of child-welfare and law-enforcement professionals interviewing maltreated children supported via artificial avatars
The interdisciplinary FRIPRO project aims to improve interviews with maltreated children through a training program using realistic and interactive child avatars.
The department of Holistic Systems (HOST) at SimulaMet will be working with the Faculty of Social Sciences at OsloMet. The project will begin on the 1st of April 2021 and end on the 31st of March in 2024. It is funded by The Research Council of Norway with 12 million NOK and will include three Ph.D. positions.
Maltreatment and abuse of children is a significant societal problem that has serious and damaging effects on children’s behavior, psychological development, and adjustment. Detection and prevention of violence and sexual abuse against children is, therefore, a high priority for Child Protective Services (CPS) and law-enforcement professionals. The conversations and investigative interviews that are conducted with these children must be of high quality. However, both Norwegian and international research shows that despite investments in methodology, the current interview and conversation skills still need to be improved.
By using an empirically informed training system in highly realistic child avatars, this project aims to develop and maintain the advanced skills needed for interviewing maltreated children. They will use data from past investigative interviews with maltreated children and create a real-looking avatar that is capable of expressing emotion and spontaneous responses.
The planned avatar will be a combination of technologies from multiple areas in computer science including AI, computer vision, and natural language processing. The aim is for the child avatars to be a part of an interview-training program that will be implemented in cooperation with the CPS and the police. The training system will be evaluated by the project scientists to judge effectiveness in relation to real-world needs.
The project also involves collaborations with researchers from Griffith University in Australia and the University of Cambridge in the United Kingdom.
FFC: Female Football Centre
The Department of Holistic Systems at SimulaMet is collaborating with UiT - The Arctic University of Norway and Forzasys AS on the Female Football Centre (FFC), funded by the Tromsø Research Foundation.
The main goal of the centre is to gain new and fundamental insights into what affects the performance and overall health of female elite football players. A general objective is to devise novel methodologies for epidemiological research that might impact research fields in both sports and medicine. In particular, we aim to develop a non-invasive, privacy-preserving technology that enables us to continuously quantify and monitor athlete behavior where we derive analytic insights from different perspectives (e.g., biomechanics, sports-specific science, medicine, coaches, and athletes).
In the current gold standards for epidemiology, observational prospective cohort studies include that cohort subjects are followed in detail over a longer period. This is an error-prone and tedious task that has for a long time been carried out using pen and paper, and later doing a manual, tedious analysis. Making this entire process easier is the main responsibility of the researchers from the Department of Holistic Systems at SimulaMet.
In cooperation with UiT and Forzasys AS, SimulaMet has earlier developed and used an automatic performance monitoring system for athletes used by both national and elite series soccer teams. The goal is to quantify and develop accurate analysis technologies that enable a personalized assessment and performance development of elite athletes.
The automatic performance monitoring system collects athletes’ subjective parameters, like training load, wellness, injury, and illness, using a small questionnaire-app running on their mobile phones, and the data is transferred to a cloud-based backend system. Then, from a trainer-portal, the data can be automatically visualized for both individual players and team overviews.
In FFC, the objective is to extend the system further to include female-specific parameters and introduce more automatic analysis using, for example, machine learning. We will host and develop the system for all the teams participating in the project, and we will initiate automatic analyses that might be able to predict future overuse injuries or to help maybe to find the best development process of a player or a team.
Svein Arne Pettersen (head of research at the School of sports sciences, UiT) is the centre leader, and he has collaborated with the researchers at SimulaMet for a long time.
Read more about the new Female Football Centre (FFC)
FFC is funded by Tromsø Research Foundation.
Publications at Department of Holistic Systems
Journal Article
Complexity and Variability Analyses of Motor Activity Distinguish Mood States in Bipolar Disorder
PLOS ONE (2022).Status: Accepted
Complexity and Variability Analyses of Motor Activity Distinguish Mood States in Bipolar Disorder
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | PLOS ONE |
| Publisher | PLOS ONE |
Estimating tukey depth using incremental quantile estimators
Pattern Recognition 122 (2022): 108339.Status: Published
Estimating tukey depth using incremental quantile estimators
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Pattern Recognition |
| Volume | 122 |
| Pagination | 108339 |
| Publisher | {Elsevier |
Artificial intelligence in dry eye disease
The Ocular Surface 23 (2022): 74-86.Status: Published
Artificial intelligence in dry eye disease
Dry eye disease (DED) has a prevalence of between 5 and 50%, depending on the diagnostic criteria used and population under study. However, it remains one of the most underdiagnosed and undertreated conditions in ophthalmology. Many tests used in the diagnosis of DED rely on an experienced observer for image interpretation, which may be considered subjective and result in variation in diagnosis. Since artificial intelligence (AI) systems are capable of advanced problem solving, use of such techniques could lead to more objective diagnosis. Although the term ‘AI’ is commonly used, recent success in its applications to medicine is mainly due to advancements in the sub-field of machine learning, which has been used to automatically classify images and predict medical outcomes. Powerful machine learning techniques have been harnessed to understand nuances in patient data and medical images, aiming for consistent diagnosis and stratification of disease severity. This is the first literature review on the use of AI in DED. We provide a brief introduction to AI, report its current use in DED research and its potential for application in the clinic. Our review found that AI has been employed in a wide range of DED clinical tests and research applications, primarily for interpretation of interferometry, slit-lamp and meibography images. While initial results are promising, much work is still needed on model development, clinical testing and standardisation.
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | The Ocular Surface |
| Volume | 23 |
| Pagination | 74 - 86 |
| Date Published | Jan-01-2022 |
| Publisher | Elsevier |
| ISSN | 15420124 |
| Keywords | artificial intelligence, Dry eye disease, Machine learning |
| URL | https://linkinghub.elsevier.com/retrieve/pii/S1542012421001324 |
| DOI | 10.1016/j.jtos.2021.11.004 |
Proceedings, refereed
Unsupervised Image Segmentation via Self-Supervised Learning Image Classification
In MediaEval 2021. Working Notes Proceedings of the MediaEval 2021 Workshop ed. CEUR Workshop Proceedings, 2022.Status: Accepted
Unsupervised Image Segmentation via Self-Supervised Learning Image Classification
This paper presents the submission of team Medical-XAI for the Medico: Transparency in Medical Image Segmentation task held at MediaEval 2021. We propose an unsupervised method that utilizes tools from the field of explainable artificial intelligence to create segmentation masks. We extract heat maps, which are useful in order to explain how the `black box' model predicts the category of a certain image, and the segmentation masks are directly derived from the heat maps. Our results show that the created masks can capture the relevant findings to a certain extent using only a small amount of image-level labeled data for the classification model and no segmentation masks at all for the training. This is promising for addressing different challenges within the intersection of artificial intelligence for medicine such as availability of data, cost of labeling and interpretable and explainable results.
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | MediaEval 2021 |
| Edition | Working Notes Proceedings of the MediaEval 2021 Workshop |
| Publisher | CEUR Workshop Proceedings |
| Keywords | clustering, Explainable artificial intelligence, Global Features, Grad-CAM, Image segmentation, Medical imaging, Polyp Detection, Self-supervised learning |
| URL | https://2021.multimediaeval.com/paper12.pdf |
Multimedia Streaming Analytics: Quo Vadis?
In ACM Mile High Video. ACM, 2022.Status: Accepted
Multimedia Streaming Analytics: Quo Vadis?
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | ACM Mile High Video |
| Publisher | ACM |
Investigative Interviews using a Multimodal Virtual Avatar
In American Psychology-Law Society Conference 2022. Denver USA,: American Psychology-Law Society, 2022.Status: Accepted
Investigative Interviews using a Multimodal Virtual Avatar
To meet best-practice standards, we are developing an interactive virtual avatar aiming as a training tool to raise interviewing skills of child-welfare and law-enforcement professionals. Therefore, we present the “Ilma” avatar that recognizes interviewers’ behavior during open-ended, closed and leading questions, and which can automatically respond to the conversation. We conducted a user study in which master students (N=3) and child protective workers (N=8) interviewed “Ilma” and rated their perception of the interaction. The results show that the participants valued the interaction and found the avatar useful. Thus, it has great potential to be an effective training tool.
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | American Psychology-Law Society Conference 2022 |
| Publisher | American Psychology-Law Society |
| Place Published | Denver USA, |
Multimodal Virtual Avatars for Investigative Interviews with Children
In ICMR '21: International Conference on Multimedia RetrievalProceedings of the 2021 Workshop on Intelligent Cross-Data Analysis and Retrieval. Taipei TaiwanNew York, NY, USA: ACM, 2021.Status: Published
Multimodal Virtual Avatars for Investigative Interviews with Children
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type |
Temperature Forecasting using Tower Networks
In ICMR '21: International Conference on Multimedia RetrievalProceedings of the 2021 Workshop on Intelligent Cross-Data Analysis and Retrieval. Taipei TaiwanNew York, NY, USA: ACM, 2021.Status: Published
Temperature Forecasting using Tower Networks
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type |
Modeling and Understanding the Quality of Experience of Online Mobile Gaming Services
In 2021 13th International Conference on Quality of Multimedia Experience (QoMEX)2021 13th International Conference on Quality of Multimedia Experience (QoMEX). Montreal, QC, Canada: IEEE, 2021.Status: Published
Modeling and Understanding the Quality of Experience of Online Mobile Gaming Services
| Afilliation | Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type |
Book Chapter
Artificial Intelligence in Gastroenterology
In Artificial Intelligence in Medicine, 1-20. Cham: Springer International Publishing, 2021.Status: Published
Artificial Intelligence in Gastroenterology
The holy grail in endoscopy examinations has for a long time been assisted diagnosis using Artificial Intelligence (AI). Recent developments in computer hardware are now enabling technology to equip clinicians with promising tools for computer-assisted diagnosis (CAD) systems. However, creating viable models or architectures, training them, and assessing their ability to diagnose at a human level, are complicated tasks. This is currently an active area of research, and many promising methods have been proposed. In this chapter, we give an overview of the topic. This includes a description of current medical challenges followed by a description of the most commonly used methods in the field. We also present example results from research targeting some of these challenges, and a discussion on open issues and ongoing work is provided. Hopefully, this will inspire and enable readers to future develop CAD systems for gastroenterology.
| Afilliation | Communication Systems, Machine Learning |
| Project(s) | Department of Holistic Systems |
| Publication Type | Book Chapter |
| Year of Publication | 2021 |
| Book Title | Artificial Intelligence in Medicine |
| Pagination | 1 - 20 |
| Date Published | 09/2021 |
| Publisher | Springer International Publishing |
| Place Published | Cham |
| ISBN Number | 978-3-030-58080-3 |
| Keywords | Anomaly detection, artificial intelligence, Gastrointestinal endoscopy, Hand-crafted features, Neural Networks, Performance, Semantic segmentation |
| URL | https://link.springer.com/referenceworkentry/10.1007%2F978-3-030-58080-3... |
| DOI | 10.1007/978-3-030-58080-3_163-2 |
