Department of Machine Intelligence
Machine learning has gained enormous international momentum, and it has been defined as a separate research topic at Simula since 2018. In particular, as a part of the new Simula Metropolitan Center for Digital Engineering, the new Machine Intelligence Department (MIND) was established in 2018.
MIND aims at advancing frontiers of machine learning and data science by developing novel methodologies and algorithmic solutions for the analysis of complex systems and high-dimensional data in science and industry. Specifically, the research activities represented within MIND span the following areas: statistical learning and regularization theory; data mining with a focus on matrix/tensor factorizations and their applications in various fields including neuroscience, systems biology, social networks and communication networks.
People at Department of Machine Intelligence
Who we are?
Simula Metropolitan employees are researchers, postdoctoral fellows, PhD students, engineers and administrative people. We are from all over the world, ranging from newly educated to experienced researchers, all working on making research in digital engineering at the highest international level possible.
Projects at Department of Machine Intelligence
TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion
Data mining holds the promise to improve our understanding of dynamics of complex systems such as the human brain and human metabolome (i.e., the complete set of small biochemical compounds in the human body) by discovering the underlying patterns, i.e., subsystems, in data collected from these systems. However, discovering those patterns and understanding their evolution in time is a challenging task. The complexity of the systems requires collection of both time-evolving and static data from multiple sources using different technologies recording the behavior of the system from complementary viewpoints, and there is a lack of data mining methods that can find the hidden patterns in such complex data.
The goal of this multidisciplinary project is to develop novel data mining techniques to jointly analyze static and dynamic data sets to discover underlying patterns, understand temporal dynamics of those patterns, and capture early signs of future outcomes. We will introduce a scalable and constrained data fusion framework that can jointly factorize heterogeneous data in the form of matrices and multi-way arrays, by incorporating temporal as well as domain-specific constraints.
These methods will be motivated by a real, challenging system: the human metabolome, and used to jointly analyze static genetic information and longitudinal metabolomics data to discover interpretable patterns, i.e., subsystems corresponding to metabolic networks (networks of metabolites acting together), with the ultimate goal of understanding their role in the transition from healthy to diseased states. The project will play a significant role in terms of developing the data mining tools needed to extract meaningful information from the surge of data, referred to as "personal data clouds" being collected in predictive medicine studies, where participants give blood samples regularly to track their health status and will be alerted of early signs of diseases.
Funding Source
Research Council of Norway, IKTPLUSS (2020-2023)
Novo Nordisk Foundation, Exploratory Interdisciplinary Synergy Grant (2020-2022)
Partners
COPSAC (Danish Pediatric Asthma Center)
University of Copenhagen
University of Amsterdam
Publications at Department of Machine Intelligence
Journal Article
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
IEEE Journal of Selected Topics in Signal Processing (2021).Status: Published
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
| Afilliation | Machine Learning |
| Project(s) | TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion, Department of Machine Intelligence |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | IEEE Journal of Selected Topics in Signal Processing |
| Publisher | IEEE |
Talks, contributed
Exploring Dynamic Metabolomics Data with Multi-Way Data Analysis: A Simulation Study
In SIAM Conference on Applications of Dynamical Systems, 2021.Status: Published
Exploring Dynamic Metabolomics Data with Multi-Way Data Analysis: A Simulation Study
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | SIAM Conference on Applications of Dynamical Systems |
| URL | https://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=72403 |
Tracing Dynamic Networks through Constrained Parafac2 Decomposition
In Virtual Conference. SIAM Conference on Applied Linear Algebra (LA21), 2021.Status: Published
Tracing Dynamic Networks through Constrained Parafac2 Decomposition
Time-evolving data analysis is crucial for understanding complex systems such as the brain. Methods that assume static networks have successfully recovered spatial networks of connectivity from neuroimaging data. Still, discovering both underlying networks and their evolution is a challenging task.
To capture temporal evolution of connectivity networks, we arrange dynamic data as a tensor and use a tensor factorization method called PARAFAC2. PARAFAC2 deciphers the hidden structure of dynamic networks and yields unique and interpretable components. Preliminary results using PARAFAC2 in neuroimaging data analysis are promising. However, the constant cross-product constraint on the time-evolving mode hinders the use of additional constraints or regularization (e.g. spatial smoothness) on this mode. Currently, the only way to regularize the time-evolving mode of a PARAFAC2 model is with a flexible coupling approach, which finds the solution through regularized least-squares subproblems. Instead, we use an alternating direction method of multipliers (ADMM) based approach to widen the possible regularization penalties to any proximable function.
Our numerical experiments demonstrate that the proposed ADMM-based algorithmic approach for PARAFAC2 can accurately recover the underlying evolving components, is flexible in terms of imposing constraints and also computationally efficient.
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | Virtual Conference |
| Publisher | SIAM Conference on Applied Linear Algebra (LA21) |
Talks, invited
A Flexible Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorizations based on the Alternating Direction Method of Multipliers
In Europt21, 18th Workshop on Advances in Continuous Optimization, 2021.Status: Published
A Flexible Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorizations based on the Alternating Direction Method of Multipliers
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Europt21, 18th Workshop on Advances in Continuous Optimization |
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
In SIAM Conference on Applied Linear Algebra (LA21), 2021.Status: Published
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | SIAM Conference on Applied Linear Algebra (LA21) |
| Type of Talk | Minisymposium |
From Data Mining using Tensor Factorizations to Multimodal Data Mining using Coupled Matrix/Tensor Factorizations
In Nordic Probabilistic AI School (virtual), 2021.Status: Published
From Data Mining using Tensor Factorizations to Multimodal Data Mining using Coupled Matrix/Tensor Factorizations
| Afilliation | Machine Learning |
| Project(s) | TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion, Department of Machine Intelligence |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Nordic Probabilistic AI School (virtual) |
| URL | https://probabilistic.ai/ |
Journal Article
Multi-Linear Population Analysis (MLPA) of LFP Data Using Tensor Decompositions
Frontiers in Applied Mathematics and Statistics 6 (2020).Status: Published
Multi-Linear Population Analysis (MLPA) of LFP Data Using Tensor Decompositions
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Frontiers in Applied Mathematics and Statistics |
| Volume | 6 |
| Number | 41 |
| Date Published | Aug-09-2020 |
| Publisher | Frontiers |
| Keywords | CANDECOMP/PARAFAC, independent component analysis (ICA), local field potential (LFP), Neuroscience, population rate model, principal component analysis (PCA), tensor decompositions |
| URL | https://www.frontiersin.org/article/10.3389/fams.2020.00041/fullhttps://... |
| DOI | 10.3389/fams.2020.00041 |
Talks, invited
Multi Modal Data Mining using Coupled Matrix/Tensor Factorizations
In Tufts University - TRIPODS Seminar (virtual), 2020.Status: Published
Multi Modal Data Mining using Coupled Matrix/Tensor Factorizations
| Afilliation | Machine Learning |
| Project(s) | TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, invited |
| Year of Publication | 2020 |
| Location of Talk | Tufts University - TRIPODS Seminar (virtual) |
| URL | https://tripods.tufts.edu/about/events/multi-modal-data-mining-using-cou... |
DeCipher
Cancer is a significant cause of morbidity and mortality worldwide. In Norway alone, there are more than 33,000 new cancer patients each year, and 11,000 cancer-associated deaths in 2017. A large proportion of these incidents are preventable. For example, a mass-screening program against cervical cancer established in the Nordic countries has demonstrated a reduction in morbidity and mortality almost by 80 %. Despite this success, it remains a significant challenge to improve the screening program, such as minimize over screening and undertreatment and hence reduce expenditure in a broad public health perspective.
Current knowledge about the disease, together with a wealth of available data and modern technologies, can offer far better-personalized prevention, by deriving an individual-based time till the next screening. Existing automatic decision support systems for cervical cancer prevention are, however, extremely conservative as they are mostly limited to identifying patients who are overdue for their next routine screening, without providing any personalized recommendations for follow-ups.
By intelligent use of existing registries and health data, DeCipher aims to develop a data-driven framework to provide a personalized time-varying risk assessment for cancer initiation and identify subgroups of individuals and factors leading to similar disease progression. By unveiling structure hidden in the data, we will develop novel theoretically grounded machine learning methods for the analysis of large-scale registry and health data.
DeCipher consists of an excellent multidisciplinary research team from diverse fields such as machine learning, data mining, screening programs, and epidemiology. Available to screening programs, clinicians, and individuals in the population, the DeCipher results will allow for an improvement of an individual’s preventive cancer healthcare while reducing the cost of screening programs.
SimulaMet’s Role
SimulaMet will play a central role in the development of machine learning algorithms for longitudinal screening data analysis. Moreover, as the coordinator, SimulaMet is responsible for overall project management and dissemination activities.
Funding source
Research Council of Norway, IKTPLUSS
All partners
Cancer Registry Norway
Karolinska University Hospital, Sweden
Lawrence Livermore National Lab, USA
Coordinator
SimulaMet
UPSKILL
An incomplete mapping of the skills of a given individual, combined with insufficient insight into a company's actual need for competence, give rise to quite a few challenges. For instance, it may lead to hiring the wrong candidates, lack of insight into the best path for personal development and challenges when deciding relevant content for courses, learning material and for continued education.
UPSKILL will introduce a global platform for professional networking. The platform will connect individuals, companies and learning providers, and offer automatic methods for identification, mapping, and development of skills and abilities.
The project will result in new methods for representing the skills of an individual, mapping a company's need for competence, as well as new methods for matching available skills and abilities with the actual need for
competence. The methods will be self-learning, applicable for commercial use and independent of industry.
As a result, the UPSKILL platform will lead to simplified and less expensive hiring and restructuring processes, reduced risk of hiring wrong candidates, free competence guidance for individuals, and content recommendation for learning providers. The platform will be launched in Europe and Southeast Asia after project completion.
Simula’s Role
Simula plays a central role in designing and developing data-driven algorithms for the automatic and unbiased hiring process, identification of individual’s competence profile from available data sources, and development of matching algorithms for potential employees and employers. The developed methods will form a solid foundation for the UPSKILL platform.
Funding source
Research Council of Norway, BIA
All partners
Simula Metropolitan Center for Digital Engineering
Oslo Metropolitan University
University of South-Eastern Norway
Coordinator
Conexus AS
Publications at Department of Machine Intelligence
Journal Article
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
IEEE Journal of Selected Topics in Signal Processing (2021).Status: Published
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
| Afilliation | Machine Learning |
| Project(s) | TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion, Department of Machine Intelligence |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | IEEE Journal of Selected Topics in Signal Processing |
| Publisher | IEEE |
Poster
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
European Signal Processing Conference (EUSIPCO) 2020, 2021.Status: Published
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence |
| Publication Type | Poster |
| Year of Publication | 2021 |
| Place Published | European Signal Processing Conference (EUSIPCO) 2020 |
Proceedings, refereed
Data-driven Personalized Cervical Cancer Risk Prediction: A Graph-Perspective
In IEEE Statistical Signal Processing Workshop 2021, 2021.Status: Accepted
Data-driven Personalized Cervical Cancer Risk Prediction: A Graph-Perspective
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE Statistical Signal Processing Workshop 2021 |
Talks, contributed
Exploring Dynamic Metabolomics Data with Multi-Way Data Analysis: A Simulation Study
In SIAM Conference on Applications of Dynamical Systems, 2021.Status: Published
Exploring Dynamic Metabolomics Data with Multi-Way Data Analysis: A Simulation Study
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | SIAM Conference on Applications of Dynamical Systems |
| URL | https://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=72403 |
Tracing Dynamic Networks through Constrained Parafac2 Decomposition
In Virtual Conference. SIAM Conference on Applied Linear Algebra (LA21), 2021.Status: Published
Tracing Dynamic Networks through Constrained Parafac2 Decomposition
Time-evolving data analysis is crucial for understanding complex systems such as the brain. Methods that assume static networks have successfully recovered spatial networks of connectivity from neuroimaging data. Still, discovering both underlying networks and their evolution is a challenging task.
To capture temporal evolution of connectivity networks, we arrange dynamic data as a tensor and use a tensor factorization method called PARAFAC2. PARAFAC2 deciphers the hidden structure of dynamic networks and yields unique and interpretable components. Preliminary results using PARAFAC2 in neuroimaging data analysis are promising. However, the constant cross-product constraint on the time-evolving mode hinders the use of additional constraints or regularization (e.g. spatial smoothness) on this mode. Currently, the only way to regularize the time-evolving mode of a PARAFAC2 model is with a flexible coupling approach, which finds the solution through regularized least-squares subproblems. Instead, we use an alternating direction method of multipliers (ADMM) based approach to widen the possible regularization penalties to any proximable function.
Our numerical experiments demonstrate that the proposed ADMM-based algorithmic approach for PARAFAC2 can accurately recover the underlying evolving components, is flexible in terms of imposing constraints and also computationally efficient.
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | Virtual Conference |
| Publisher | SIAM Conference on Applied Linear Algebra (LA21) |
Talks, invited
A Flexible Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorizations based on the Alternating Direction Method of Multipliers
In Europt21, 18th Workshop on Advances in Continuous Optimization, 2021.Status: Published
A Flexible Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorizations based on the Alternating Direction Method of Multipliers
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Europt21, 18th Workshop on Advances in Continuous Optimization |
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
In SIAM Conference on Applied Linear Algebra (LA21), 2021.Status: Published
An Optimization Framework for Regularized Linearly Coupled Matrix-Tensor Factorization
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | SIAM Conference on Applied Linear Algebra (LA21) |
| Type of Talk | Minisymposium |
From Data Mining using Tensor Factorizations to Multimodal Data Mining using Coupled Matrix/Tensor Factorizations
In Nordic Probabilistic AI School (virtual), 2021.Status: Published
From Data Mining using Tensor Factorizations to Multimodal Data Mining using Coupled Matrix/Tensor Factorizations
| Afilliation | Machine Learning |
| Project(s) | TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion, Department of Machine Intelligence |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Nordic Probabilistic AI School (virtual) |
| URL | https://probabilistic.ai/ |
Book Chapter
Multilinear Models, Iterative Methods
In Comprehensive Chemometrics (Second Edition), 267-304. Chemical and Biochemical Data Analysis. Elsevier, 2020.Status: Published
Multilinear Models, Iterative Methods
In this section, multilinear models for multi-way arrays requiring iterative fitting algorithms are outlined. Among them: the PARAFAC (PARAllel FACtor analysis) model and one of its variants (the PARAFAC2 model); Tucker models in which one or more modes are reduced (viz., the N-way Tucker-N and Tucker-m models); hybrid models having intermediate properties between PARAFAC and Tucker ones; and coupled matrix and tensor decompositions (CMTF) which simultaneously decomposes multiple tensors. Five examples are included as to illustrate some practical aspects concerning the use of these models on analytical data.
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence |
| Publication Type | Book Chapter |
| Year of Publication | 2020 |
| Book Title | Comprehensive Chemometrics (Second Edition) |
| Secondary Title | Chemical and Biochemical Data Analysis |
| Pagination | 267-304 |
| Publisher | Elsevier |
| ISBN Number | 978-0-12-409547-2 |
| Keywords | -way Tucker models, CANDECOMP, Curve resolution, Exploratory analysis, Least squares, Linked mode PARAFAC, Multi-way analysis, Multi-way array, Multilinear model, PARAFAC, PARAFAC2, PARALIND, Restricted Tucker models, Tensor decomposition, Tensor-matrix factorization |
| URL | http://www.sciencedirect.com/science/article/pii/B9780124095472146098 |
| DOI | 10.1016/B978-0-12-409547-2.14609-8 |
Journal Article
Multi-Linear Population Analysis (MLPA) of LFP Data Using Tensor Decompositions
Frontiers in Applied Mathematics and Statistics 6 (2020).Status: Published
Multi-Linear Population Analysis (MLPA) of LFP Data Using Tensor Decompositions
| Afilliation | Machine Learning |
| Project(s) | Department of Machine Intelligence, TrACEr: Time-Aware ConstrainEd Multimodal Data Fusion |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Frontiers in Applied Mathematics and Statistics |
| Volume | 6 |
| Number | 41 |
| Date Published | Aug-09-2020 |
| Publisher | Frontiers |
| Keywords | CANDECOMP/PARAFAC, independent component analysis (ICA), local field potential (LFP), Neuroscience, population rate model, principal component analysis (PCA), tensor decompositions |
| URL | https://www.frontiersin.org/article/10.3389/fams.2020.00041/fullhttps://... |
| DOI | 10.3389/fams.2020.00041 |