Publications
Proceedings, refereed
A Comparative Study of Interactive Environments for Investigative Interview of A Virtual Child Avatar
In IEEE international symposium on multimedia (ISM). IEEE, 2022.Status: Published
A Comparative Study of Interactive Environments for Investigative Interview of A Virtual Child Avatar
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | IEEE international symposium on multimedia (ISM) |
Pagination | 194-201 |
Publisher | IEEE |
DOI | 10.1109/ISM55400.2022.00043 |
Comparison of Crowdsourced and Remote Subjective User Studies: A Case Study of Investigative Child Interviews
In The 14th International Conference on Quality of Multimedia Experience. IEEE, 2022.Status: Published
Comparison of Crowdsourced and Remote Subjective User Studies: A Case Study of Investigative Child Interviews
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | The 14th International Conference on Quality of Multimedia Experience |
Publisher | IEEE |
URL | https://ieeexplore.ieee.org/document/9900900 |
DOI | 10.1109/QoMEX55416.2022.9900900 |
Experiences and Lessons Learned from a Crowdsourced-Remote Hybrid User Survey Framework
In 2022 IEEE International Symposium on Multimedia (ISM). Italy: IEEE, 2022.Status: Published
Experiences and Lessons Learned from a Crowdsourced-Remote Hybrid User Survey Framework
Subjective user studies are important to ensure the fidelity and usability of systems that generate multimedia content. Testing how end-users and domain experts perceive multimedia assets might provide crucial information. In this paper, we present our experiences with the open source hybrid crowdsourced-remote user survey framework called Huldra, which is intended for conducting web-based subjective user studies and aims to integrate the individual benefits associated with traditional, crowdsourced, and remote methods. We disseminate our experiences and insights from two actively deployed use cases and discuss challenges and opportunities associated with using Huldra as a framework for conducting user studies.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 2022 IEEE International Symposium on Multimedia (ISM) |
Pagination | 161-162 |
Publisher | IEEE |
Place Published | Italy |
URL | https://ieeexplore.ieee.org/document/10019678 |
DOI | 10.1109/ISM55400.2022.00035 |
Huldra: A Framework for Collecting Crowdsourced Feedback on Multimedia Assets
In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). ACM, 2022.Status: Published
Huldra: A Framework for Collecting Crowdsourced Feedback on Multimedia Assets
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22) |
Pagination | 203-209 |
Publisher | ACM |
DOI | 10.1145/3524273.3532887 |
Human vs. GPT-3: The challenges of extracting emotions from child responses
In The 14th International Conference on Quality of Multimedia Experience. IEEE, 2022.Status: Published
Human vs. GPT-3: The challenges of extracting emotions from child responses
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | The 14th International Conference on Quality of Multimedia Experience |
Publisher | IEEE |
ISBN Number | 978-1-6654-8794-8 |
ISSN Number | 2472-7814 |
Accession Number | 22114185 |
URL | https://ieeexplore.ieee.org/document/9900885 |
DOI | 10.1109/QoMEX55416.2022.9900885 |
Investigative Interviews using a Multimodal Virtual Avatar
In American Psychology-Law Society Conference 2022. Denver USA,: American Psychology-Law Society, 2022.Status: Accepted
Investigative Interviews using a Multimodal Virtual Avatar
To meet best-practice standards, we are developing an interactive virtual avatar aiming as a training tool to raise interviewing skills of child-welfare and law-enforcement professionals. Therefore, we present the “Ilma” avatar that recognizes interviewers’ behavior during open-ended, closed and leading questions, and which can automatically respond to the conversation. We conducted a user study in which master students (N=3) and child protective workers (N=8) interviewed “Ilma” and rated their perception of the interaction. The results show that the participants valued the interaction and found the avatar useful. Thus, it has great potential to be an effective training tool.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | American Psychology-Law Society Conference 2022 |
Publisher | American Psychology-Law Society |
Place Published | Denver USA, |
Is More Realistic Better? A Comparison of Game Engine and GAN-based Avatars for Investigative Interviews of Children
In ICDAR '22: Proceedings of the 3rd ACM Workshop on Intelligent Cross-Data Analysis and Retrieval. New York, NY, USA: ACM, 2022.Status: Published
Is More Realistic Better? A Comparison of Game Engine and GAN-based Avatars for Investigative Interviews of Children
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | ICDAR '22: Proceedings of the 3rd ACM Workshop on Intelligent Cross-Data Analysis and Retrieval |
Pagination | 41-49 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 9781450392419 |
URL | https://dl.acm.org/doi/proceedings/10.1145/3512731 |
DOI | 10.1145/351273110.1145/3512731.3534209 |
Soccer Game Summarization using Audio Commentary, Metadata, and Captions
In NarSUM '22: Proceedings of the 1st Workshop on User-centric Narrative Summarization of Long Videos. New York, NY, USA: ACM, 2022.Status: Published
Soccer Game Summarization using Audio Commentary, Metadata, and Captions
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | NarSUM '22: Proceedings of the 1st Workshop on User-centric Narrative Summarization of Long Videos |
Pagination | 13-22 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 9781450394932 |
URL | https://doi.org/10.1145/3552463.3557019 |
DOI | 10.1145/3552463.3557019 |
Towards an AI-driven talking avatar in virtual reality for investigative interviews of children
In GameSys '22: Proceedings of the 2nd Workshop on Games Systems. New York, NY, USA: ACM, 2022.Status: Published
Towards an AI-driven talking avatar in virtual reality for investigative interviews of children
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | GameSys '22: Proceedings of the 2nd Workshop on Games Systems |
Pagination | 9-15 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 9781450393812 |
URL | https://dl.acm.org/doi/10.1145/3534085.3534340 |
DOI | 10.1145/353408510.1145/3534085.3534340 |
Virtual Reality Talking Avatar for Investigative Interviews of Maltreat Children
In 19th International Conference on Content-based Multimedia Indexing. New York, NY, USA: Association for Computing Machinery (ACM), 2022.Status: Published
Virtual Reality Talking Avatar for Investigative Interviews of Maltreat Children
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 19th International Conference on Content-based Multimedia Indexing |
Pagination | 201-204 |
Publisher | Association for Computing Machinery (ACM) |
Place Published | New York, NY, USA |
ISBN Number | 9781450397209 |
URL | https://doi.org/10.1145/3549555.3549572 |
DOI | 10.1145/3549555.3549572 |
When Every Millisecond Counts: The Impact of Delay in VR Gaming
In The 14th International Conference on Quality of Multimedia Experience. IEEE, 2022.Status: Published
When Every Millisecond Counts: The Impact of Delay in VR Gaming
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | The 14th International Conference on Quality of Multimedia Experience |
Publisher | IEEE |
URL | https://ieeexplore.ieee.org/document/9900908 |
DOI | 10.1109/QoMEX55416.2022.9900908 |
Journal Article
Synthesizing a Talking Child Avatar to Train Interviewers Working with Maltreated Children
Big Data and Cognitive Computing 6, no. 2 (2022): 62.Status: Published
Synthesizing a Talking Child Avatar to Train Interviewers Working with Maltreated Children
Afilliation | Communication Systems, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Big Data and Cognitive Computing |
Volume | 6 |
Issue | 2 |
Pagination | 62 |
Date Published | Jan-06-2022 |
Publisher | MDPI |
URL | https://www.mdpi.com/2504-2289/6/2/62https://www.mdpi.com/2504-2289/6/2/... |
DOI | 10.3390/bdcc6020062 |
Book
The Influence of Delay on Cloud Gaming Quality of Experience
Cham: Springer, 2022.Status: Published
The Influence of Delay on Cloud Gaming Quality of Experience
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Book |
Year of Publication | 2022 |
Edition | 1 |
Number of Pages | 156 |
Date Published | 06/2022 |
Publisher | Springer |
Place Published | Cham |
ISBN Number | 978-3-030-99868-4 |
URL | https://link.springer.com/book/10.1007/978-3-030-99869-1 |
DOI | 10.1109/QoMEX55416.2022.9900908 |
Proceedings, refereed
Modeling and Understanding the Quality of Experience of Online Mobile Gaming Services
In 2021 13th International Conference on Quality of Multimedia Experience (QoMEX). Montreal, QC, Canada: IEEE, 2021.Status: Published
Modeling and Understanding the Quality of Experience of Online Mobile Gaming Services
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 2021 13th International Conference on Quality of Multimedia Experience (QoMEX) |
Pagination | 157-162 |
Publisher | IEEE |
Place Published | Montreal, QC, Canada |
URL | https://ieeexplore.ieee.org/document/9465431/http://xplorestaging.ieee.o... |
DOI | 10.1109/QoMEX51781.2021.9465431 |
PhD Thesis
Understanding and Mitigating the Influence of Delay on Cloud Gaming Quality of Experience
In Technische Universität Berlin, 2021.Status: Published
Understanding and Mitigating the Influence of Delay on Cloud Gaming Quality of Experience
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | PhD Thesis |
Year of Publication | 2021 |
Degree awarding institution | Technische Universität Berlin |
Proceedings, refereed
A Latency Compensation Technique Based on Game Characteristics to Mitigate the Influence of Delay on Cloud Gaming Quality of Experience
In ACM Multimedia Systems Conference 2020 (MMSys 2020). New York, NY, USA: ACM, 2020.Status: Published
A Latency Compensation Technique Based on Game Characteristics to Mitigate the Influence of Delay on Cloud Gaming Quality of Experience
Cloud Gaming (CG) is an immersive multimedia service that promises many benefits. In CG, the games are rendered in a cloud server, and the resulted scenes are streamed as a video sequence to the client. Using CG users are not forced to update their gaming hardware frequently, and available games can be played on any operating system or suitable device. However, cloud gaming requires a reliable and low-latency network, which makes it a very challenging service. Transmission latency strongly affects the playability of a cloud game and consequently reduces the users' Quality of Experience (QoE). In this paper, we propose a latency compensation technique using game adaptation that mitigates the influence of delay on QoE. This technique uses five game characteristics for the adaptation. These characteristics, in addition to an Aim-assistance technique, were implemented in four games for evaluation. A subjective study using 194 participants was conducted using a crowdsourcing approach. The results showed that the majority of the proposed adaptation techniques lead to significant improvements in the cloud gaming QoE.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ACM Multimedia Systems Conference 2020 (MMSys 2020) |
Pagination | 15-25 |
Publisher | ACM |
Place Published | New York, NY, USA |
URL | https://dl.acm.org/doi/abs/10.1145/3339825.3391855 |
DOI | 10.1145/3339825.3391855 |
Assessing Interactive Gaming Quality of Experience Using a Crowdsourcing Approach
In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2020.Status: Published
Assessing Interactive Gaming Quality of Experience Using a Crowdsourcing Approach
Traditionally, the Quality of Experience (QoE) is assessed in a controlled laboratory environment where participants give their opinion about the perceived quality of a stimulus on a standardized rating scale. Recently, the usage of crowdsourcing micro-task platforms for assessing the media quality is increasing. The crowdsourcing platforms provide access to a pool of geographically distributed, and demographically diverse group of workers who participate in the experiment in their own working environment and using their own hardware. The main challenge in crowdsourcing QoE tests is to control the effect of interfering influencing factors such as a user's environment and device on the subjective ratings. While in the past, the crowdsourcing approach was frequently used for speech and video quality assessment, research on a quality assessment for gaming services is rare. In this paper, we present a method to measure gaming QoE under typically considered system influence factors including delay, packet loss, and framerates as well as different game designs. The factors are artificially manipulated due to controlled changes in the implementation of games. The results of a total of five studies using a developed evaluation method based on a combination of the ITU-T Rec. P.809 on subjective evaluation methods for gaming quality and the ITU-T Rec. P.808 on subjective evaluation of speech quality with a crowdsourcing approach will be discussed. To evaluate the reliability and validity of results collected using this method, we finally compare subjective ratings regarding the effect of network delay on gaming QoE gathered from interactive crowdsourcing tests with those from equivalent laboratory experiments.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) |
Publisher | IEEE |
URL | https://ieeexplore.ieee.org/abstract/document/9123122 |
DOI | 10.1109/QoMEX48832.2020.9123122 |
Delay Sensitivity Classification of Cloud Gaming Content
In International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE'20). New York, NY, USA: Association for Computing Machinery, 2020.Status: Published
Delay Sensitivity Classification of Cloud Gaming Content
Cloud Gaming is an emerging service that catches growing interest in the research community as well as industry. Cloud Gaming require a highly reliable and low latency network to achieve a satisfying Quality of Experience (QoE) for its users. Using a cloud gaming service with high latency would harm the interaction of the user with the game, leading to a decrease in playing performance and, thus players frustrations. However, the negative effect of delay on gaming QoE depends strongly on the game content. At a certain level of delay, a slow-paced card game is typically not as delay sensitive as a shooting game. For optimal resource allocation and quality estimation, it is highly important for cloud providers, game developers, and network planners to consider the impact of the game content. This paper contributes to a better understanding of the delay impact on QoE for cloud gaming applications by identifying game characteristics influencing the delay perception of the users. In addition, an expert evaluation methodology to quantify these characteristics as well as a delay sensitivity classification based on a decision tree are presented. The results indicated an excellent level of agreement, which demonstrates the reliability of the proposed method. Additionally, the decision tree reached an accuracy of 90% on determining the delay sensitivity classes which were derived from a large dataset of subjective input quality ratings during a series of experiments.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE'20) |
Publisher | Association for Computing Machinery |
Place Published | New York, NY, USA |
ISBN Number | 9781450379472 |
URL | https://dl.acm.org/doi/abs/10.1145/3386293.3397116 |
DOI | 10.1145/3386293.3397116 |
Quality Estimation Models for Gaming Video Streaming Services Using Perceptual Video Quality Dimensions
In ACM Multimedia Systems Conference 2020 (MMSys 2020). New York, NY, USA: Association for Computing Machinery (ACM), 2020.Status: Published
Quality Estimation Models for Gaming Video Streaming Services Using Perceptual Video Quality Dimensions
The gaming industry is one of the largest digital markets for decades and is steady developing as evident by new emerging gaming services such as gaming video streaming, online gaming, and cloud gaming. While the market is rapidly growing, the quality of these services depends strongly on network characteristics as well as resource management. With the advancement of encoding technologies such as hardware accelerated engines, fast encoding is possible for delay sensitive applications such as cloud gaming. Therefore, already existing video quality models do not offer a good performance for cloud gaming applications. Thus, in this paper, we provide a gaming video quality dataset that considers hardware accelerated engines for video compression using the H.264 standard. In addition, we investigate the performance of signal-based and parametric video quality models on the new gaming video dataset. Finally, we build two novel parametric-based models, a planning and a monitoring model, for gaming quality estimation. Both models are based on perceptual video quality dimensions and can be used to optimize the resource allocation of gaming video streaming services.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | ACM Multimedia Systems Conference 2020 (MMSys 2020) |
Publisher | Association for Computing Machinery (ACM) |
Place Published | New York, NY, USA |
URL | https://dl.acm.org/doi/abs/10.1145/3339825.3391872 |
DOI | 10.1145/3339825.3391872 |
Towards the Impact of Gamers Strategy and User Inputs on the Delay Sensitivity of Cloud Games
In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2020.Status: Published
Towards the Impact of Gamers Strategy and User Inputs on the Delay Sensitivity of Cloud Games
Cloud Gaming is an emerging service that is considered by many as the future of the gaming industry. This service requires a highly reliable network with low latency and high bandwidth. If these requirements are not satisfied, cloud gaming services cannot create a good Quality of Experience (QoE) for its users. However, gaming QoE can vary significantly among different game scenarios and users. For an optimal resource allocation and quality estimation, it is highly important for cloud providers, game developers, and network planners to consider the influence of the game content and gamers. This paper presents the result of a subjective study that investigated the impact of different player strategies and user inputs on their perceived delay. The results indicated that the user input characteristics vary among the games but stays the same between different users and different strategies. In addition to the users' inputs, the input quality and the overall gaming experience of the users were also investigated, and results did not show any main effect of user strategy on the delay sensitivity of the games.
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) |
Date Published | 23 June 2020 |
Publisher | IEEE |
ISBN Number | 978-1-7281-5965-2 |
URL | https://ieeexplore.ieee.org/abstract/document/9123138 |
DOI | 10.1109/QoMEX48832.2020.9123138 |
Journal Article
QUALINET white paper on definitions of immersive media experience (IMEx)
arXiv preprint arXiv:2007.07032 (2020).Status: Published
QUALINET white paper on definitions of immersive media experience (IMEx)
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | arXiv preprint arXiv:2007.07032 |
Publisher | QUALINET |
Proceedings, refereed
Influence of primacy, recency and peak effects on the game experience questionnaire
In MMVE '19: Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems. New York, New York, USA: ACM Press, 2019.Status: Published
Influence of primacy, recency and peak effects on the game experience questionnaire
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | MMVE '19: Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems |
Pagination | 22-27 |
Publisher | ACM Press |
Place Published | New York, New York, USA |
ISBN Number | 9781450362993 |
URL | http://dl.acm.org/citation.cfm?doid=3304113http://dl.acm.org/citation.cf... |
DOI | 10.1145/330411310.1145/3304113.3326113 |
Towards the Impact of Gamers' Adaptation to Delay Variation on Gaming Quality of Experience
In 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX). Berlin, Germany: IEEE, 2019.Status: Published
Towards the Impact of Gamers' Adaptation to Delay Variation on Gaming Quality of Experience
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX) |
Pagination | 1-6 |
Publisher | IEEE |
Place Published | Berlin, Germany |
URL | https://ieeexplore.ieee.org/document/8743239/http://xplorestaging.ieee.o... |
DOI | 10.1109/QoMEX.2019.8743239 |
Proceedings, refereed
A Novel Objective Quality Assessment Method for Perceptually-Coded Cloud Gaming Video
In 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, 2018.Status: Published
A Novel Objective Quality Assessment Method for Perceptually-Coded Cloud Gaming Video
Cloud Gaming (CG) as a viable alternative to console gaming is gaining more acceptance and growing its market share in the gaming industry. In CG, the game events are processed in the cloud and the resulting scenes are streamed as a video sequence to players. In this new paradigm, one of the most important factors that has a significant impact on user quality of experience is video quality. To address the inherent high bandwidth requirement of CG, game videos should be compressed. This compression may have a negative impact on the user's quality of experience (QoE) and the assessment of this impact on user satisfaction is a challenging task. Over the years, many research works have investigated the objective and subjective quality of video, but none are directly suitable for the assessment of perceptual video quality in the context of CG. Other methods, such as eye-tracking weighted peak signal-to-noise ratio (EWPSNR) that may work in this context, require an eye-tracking device that is not always available. In this paper, we propose a new weighted PSNR objective quality method that does not require any eye-tracker or information from the game designer (such as the importance of objects in the game) to measure game video quality. Our evaluation based on 3 actual games show that our proposed method has 51% and 11% better correlation with the Mean Opinion Score (MOS) compared to PSNR and SSIM measures, respectively.
Afilliation | Communication Systems |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) |
Publisher | IEEE |
ISBN Number | 978-1-5386-1857-8 |
URL | https://ieeexplore.ieee.org/abstract/document/8396977 |
DOI | 10.1109/MIPR.2018.00021 |
Towards Applying Game Adaptation to Decrease the Impact of Delay on Quality of Experience
In 2018 IEEE International Symposium on Multimedia (ISM). Taichung, Taiwan: IEEE, 2018.Status: Published
Towards Applying Game Adaptation to Decrease the Impact of Delay on Quality of Experience
Afilliation | Communication Systems |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE International Symposium on Multimedia (ISM) |
Publisher | IEEE |
Place Published | Taichung, Taiwan |
URL | https://ieeexplore.ieee.org/document/8603269/http://xplorestaging.ieee.o... |
DOI | 10.1109/ISM.2018.00028 |