Publications
Book
Deep Learning-based Pose Estimation for Dystonia Score Prediction
Online: Eliva Press, 2023.Status: Published
Deep Learning-based Pose Estimation for Dystonia Score Prediction
Dystonia is a movement disorder that causes unusual movements and involuntary muscle contractions affecting some parts of the whole body. Selecting drugs and doses is a highly personalized process for dystonia, requiring frequent visits to the clinic, pointing toward the need for more systematic and objective methods of collecting patient data. A deep learning-based pose estimation algorithm can be a good candidate for aiding independent clinical assessment of dystonia as it has outperformed the classical approach to human pose estimation. The deep learning-based model can help patients and physicians assess the first symptoms of neurological diseases and build low-cost solutions not only for dystonia score prediction but also to monitor the progress of the disease. Pose estimation algorithms with convolution networks have already been shown to extract relevant information about the motor signals of Parkinson’s disease from video assessments, and the calculated score correlates well with the clinical score. OpenPose algorithm was used for human pose estimation in videos of dystonia patients being clinically assessed to annotate body key points in the videos. This project explored the basic pipeline steps required to process the clinical videos, including spatiotemporal key points normalization. CNN successfully predicted neck dystonia scores to around the scores obtained from standard clinical assessment, leaving space for further validations and research with more data and methods.
Afilliation | Software Engineering, Machine Learning |
Project(s) | No Simula project |
Publication Type | Book |
Year of Publication | 2023 |
Publisher | Eliva Press |
Place Published | Online |
ISBN Number | 978-999931009-3 |
Keywords | Deep Learning, Dystonia Score Prediction, Human Pose Estimation |
DOI | 10.5281/zenodo.8302944 |
Master's thesis
AI-based Soccer Game Summarization: From Video Highlights to Dynamic Text Summaries
In Tribhuvan University, 2022.Status: Published
AI-based Soccer Game Summarization: From Video Highlights to Dynamic Text Summaries
Soccer dominates the global sports market, and viewers’ interest in watching videos of soccer matches is ramping up. Globally, there is a huge and constantly increasing amount of soccer game content being generated, including video footage, audio commentary, text metadata, goal and player statistics, scores, and rankings. As a large percentage of audiences prefer to follow only the major highlights of a game, the creation of multimodal (video/audio/text) summaries is of great interest to broadcasters and fans alike. In this regard, it’s crucial to provide game summaries and highlights of the major game moments. However, creating summaries and annotating events most often necessitates the use of expensive equipment and a significant amount of time-consuming manual labor. Recent advancements in Artificial Intelligence (AI) technology have demonstrated great promise in this context. The purpose of this thesis is to use AI to support an automated pipeline for summarizing soccer matches. With Natural Language Processing (NLP) tools and heuristics, the emphasis is on creating comprehensive game summaries in textual form with variable length constraints, based on raw game multimedia (e.g., video and audio streams) and, where appropriate, easily accessible game meta-data. A longformer model has been fine-tuned to output a game summary for a given textual input of game captions. This work also explores the use of game audio in prioritizing game events from a summarization perspective. In particular, the Root Mean Square (RMS) audio intensity score has been extracted and used to extract the event priority to be included in the summary.
Afilliation | Software Engineering, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Master's thesis |
Year of Publication | 2022 |
Degree awarding institution | Tribhuvan University |
Proceedings, refereed
Assisting Soccer Game Summarization via Audio Intensity Analysis of Game Highlights
In Proceedings of 12th IOE Graduate Conference. Vol. 12. Institute of Engineering, Tribhuvan University, Nepal, 2022.Status: Published
Assisting Soccer Game Summarization via Audio Intensity Analysis of Game Highlights
In association football, the development of multimodal summaries is of great importance to both broadcasters and spectators since a large number of viewers choose to follow just the soccer game highlights. The fundamental drive for the development of summarization systems is the requirement to manage huge amounts of data in different formats. By highlighting the most pertinent facts and limiting or omitting unnecessary aspects, summarization helps avoid "information overload." The properties of the audio signals during a particular event can be used to calculate excitement around that event and filter events based on their importance. A root-mean-square (RMS) analysis of audio events was carried out to analyse the excitement across the events in the SoccerNet dataset. It was clearly seen that important events with excitement have a high and distinguishable RMS audio intensity. It was also observed that the generated noise of the crowd was significantly different across various events and if it happened for the home or away team. The intensity was higher for events related to the home team. Likewise, as the wavelet has the benefit of integrating a wave with a specific period, Morlet wavelet analysis was performed for various event types, and the power of the signal across various wavelet scales was analyzed. A distinct signature across various wavelet scales was observed for different events.
Afilliation | Software Engineering, Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | Proceedings of 12th IOE Graduate Conference |
Volume | 12 |
Pagination | 25 – 32 |
Date Published | October |
Publisher | Institute of Engineering, Tribhuvan University, Nepal |
Keywords | association football, audio signal, soccer game highlights, summarization |
URL | http://conference.ioe.edu.np/publications/ioegc12/IOEGC-12-004-12009.pdf |
DOI | 10.13140/RG.2.2.34457.70240/1 |
Soccer Game Summarization using Audio Commentary, Metadata, and Captions
In NarSUM '22: Proceedings of the 1st Workshop on User-centric Narrative Summarization of Long Videos. New York, NY, USA: ACM, 2022.Status: Published
Soccer Game Summarization using Audio Commentary, Metadata, and Captions
Afilliation | Machine Learning |
Project(s) | Department of Holistic Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | NarSUM '22: Proceedings of the 1st Workshop on User-centric Narrative Summarization of Long Videos |
Pagination | 13-22 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 9781450394932 |
URL | https://doi.org/10.1145/3552463.3557019 |
DOI | 10.1145/3552463.3557019 |
Poster
Motion sensing and ageing prediction of the Kupondole-Thapathali Bagmati bridge using Machine Learning and Time-series analysis
Kathmandu, Nepal: Student Research Symposium on Earthquake, 2020.Status: Published
Motion sensing and ageing prediction of the Kupondole-Thapathali Bagmati bridge using Machine Learning and Time-series analysis
This project, for the first time in its nature, introduces a new research paradigm of remote motion sensing for health monitoring of civil construction in the public safety domain in Nepal. Preliminary data from a piloting study from BRB encourages us to move forward with an ageing analysis of such civil structures. Students from DoECE at IOE, Pulchowk Campus will collaborate to test network configurations and hardware types. One such configuration will be a Master/Slave configuration over Raspberry Pi Server with Bluetooth 5.0 and multiple Arduino BLE Kits. The data collected will create an opportunity to study vehicle mobility and its impact on the bridge as well as over other multiple domains which we are very open for collaborations.
Afilliation | Communication Systems, Software Engineering |
Project(s) | No Simula project |
Publication Type | Poster |
Year of Publication | 2020 |
Publisher | Student Research Symposium on Earthquake |
Place Published | Kathmandu, Nepal |
URL | https://rgdoi.net/10.13140/RG.2.2.31727.18089 |
DOI | 10.13140/RG.2.2.31727.18089 |
Book
Near Real-Time Mobile Profiling and Modeling of Fine-Scale Environmental Proxies Along Major Road Lines of Nepal
In EAI/Springer Innovations in Communication and Computing. International Conference on Mobile Computing and Sustainable Informatics ed. Vol. 86222434940. Cham: Springer International Publishing, 2020.Status: Published
Near Real-Time Mobile Profiling and Modeling of Fine-Scale Environmental Proxies Along Major Road Lines of Nepal
Project(s) | No Simula project |
Publication Type | Book |
Year of Publication | 2020 |
Secondary Title | EAI/Springer Innovations in Communication and Computing |
Volume | 86222434940 |
Edition | International Conference on Mobile Computing and Sustainable Informatics |
Number of Pages | 605 - 617 |
Publisher | Springer International Publishing |
Place Published | Cham |
ISBN | 2522-8595 |
ISBN Number | 978-3-030-49794-1 |
URL | http://link.springer.com/10.1007/978-3-030-49795-8 |
DOI | 10.1007/978-3-030-49795-810.1007/978-3-030-49795-8_58 |
Talks, contributed
Near-real Time Profiling of Fine Scale Environmental Proxies Using Mobile Sensors along Kathmandu Road Lines
In Kathmandu, Nepal. 10th International Conference on Quality, Reliability, Infocom Technology and Business Operations: ICQRIT, 2019.Status: Published
Near-real Time Profiling of Fine Scale Environmental Proxies Using Mobile Sensors along Kathmandu Road Lines
Afilliation | Communication Systems, Software Engineering |
Project(s) | No Simula project |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | Kathmandu, Nepal |
Publisher | ICQRIT |
Place Published | 10th International Conference on Quality, Reliability, Infocom Technology and Business Operations |
Type of Talk | Presentation in Conference |
URL | http://rgdoi.net/10.13140/RG.2.2.17795.35360 |
DOI | 10.13140/RG.2.2.17795.35360 |
Proceedings, refereed
Real Time-Based Smart Traffic Light System With Its Simulation Using 8051 Microcontroller
In KEC Conference. Vol. 1, 2018.Status: Published
Real Time-Based Smart Traffic Light System With Its Simulation Using 8051 Microcontroller
The street lighting system is based upon the electronic controller that utilizes the traffic density survey data. Anandroid mobile app was developed for this purpose. Data was collected and analyzed at different busy junctions of Kathmandu Valley. The app maintained a database record of each vehicles type that enter in the system and simultaneously records the time they enter the junction. The data gives an insight into the number of vehicles entering the junction and the time required for them to cross it. This is helpful to calculate the stoppage time which was programmed into the system for optimized and efficient traffic management.
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | KEC Conference |
Volume | 1 |
Date Published | 2018 |
Keywords | Optimization, Stoppage time, Traffic Counter |
URL | https://kec.edu.np/real-time-based-smart-traffic-light-system-with-its-s... |
Sentence Ranking and Answer Pinpointing in Online Discussion Forums Utilising User-generated Metrics and Highlights
In Proceedings of the 9th National Student's Conference on Information Technology. Vol. 9. Kathmandu, Nepal, 2018.Status: Published
Sentence Ranking and Answer Pinpointing in Online Discussion Forums Utilising User-generated Metrics and Highlights
One of the major challenges in searching on the internet has been that the search engines and online forums have not been able to extract and pinpoint the exact answer to people's query despite information being available on the internet. Extraction of to-the-point answers from articles, posts and blogs tend to improve the search accuracy. Sentence Ranking helps to rank answers according to score that represents the positive remark for the relevance of sentence. User-generated metrics can be used to improve sentence ranking. Also, the text selected and saved as highlights by users can be used to extract the most important parts of the content. Answer pinpointing in simple forums can be achieved by allowing users to highlight parts of text, store it in a database and analyse such highlights using sentence ranking engine followed by answer extraction to find best chunk of texts. It can prove to be a milestone in providing exact and relevant answers as per the searchers' intent and can also facilitate the improvement of question answering in discussion forums.
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 9th National Student's Conference on Information Technology |
Volume | 9 |
Date Published | 2018 |
Place Published | Kathmandu, Nepal |
URL | https://www.researchgate.net/publication/330041750_Sentence_Ranking_and_... |