Harshinee Sriram

Harshinee Sriram

she/her

PhD Candidate in AI for Alzheimer’s disease diagnosis and drug discovery

Sitemap

A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.

Pages

Posts

Future Blog Post

less than 1 minute read

Published:

This post will show up by default. To disable scheduling of future posts, edit config.yml and set future: false.

Blog Post number 4

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Headings are cool

You can have many headings

Aren’t headings cool?

Blog Post number 3

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Headings are cool

You can have many headings

Aren’t headings cool?

Blog Post number 2

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Headings are cool

You can have many headings

Aren’t headings cool?

Blog Post number 1

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Headings are cool

You can have many headings

Aren’t headings cool?

courses-books

Pandora’s Box: Horn OK Please: 10

Written by authors from around the world, Pandora’s Box is an aspiration to release hope in the world today. This Anthology marks the 10th book in the book series titled ‘Horn OK Please’. Pandora’s box is an artifact in Greek mythology, taken from the myth of Pandora’s creation in Hesiod’s Works and Days. The box was actually a large jar given to Pandora. It contained all the evils of the world. Pandora was actually sent as a curse to Zeus’ men and was given the jar upon her marriage. It was never meant to be opened and yet she did. Just the way the forbidden fruit was never meant to be eaten. Upon opening the box, she unleashed eight demons unto the world. The first ones being the seven deadly sins and the last one which she managed to capture back was of course - hope. And hope forms the basis of our very existence in the world we live in today. The only thing we need is hope. We cling on to it. It makes life awesome. In a world devoid of hope, humanity cannot survive. For opening Pandora’s box refers to getting into a situation over which one has very little control over. The very essence of this book is hope for as you will flip through the pages, you will release hope unto this world. We hope you will spread the cheer.

The Horrors of Happiness

Life, as you subconsciously know, is built up of an infinite number of moments. Moments which change us, moments which improve us, moments which define us. Therefore, with this principle in mind, this book was created. There is no specific category for this book, although most would classify it as horror. It is, and not, about Love. It is, and not, about Fear. It is, and not, about Dreams. And maybe, it is that very uncertainty that appeals to us. The What if in life. What would have happened if I had said yes to him? What would have happened if I had accepted the offer that night? What would have happened if nobody intervened to save me? This book is a collection of short stories and poems that celebrate all that the human race is about: uncertainty, hope, hatred, and creating an infinitely large collection of infinitesimal moments…

UBCx: Introduction to Quantum Mechanics

This course offers a basic introduction to quantum mechanics, covering both theoretical concepts and practical applications. It aims to equip students with a foundational understanding of quantum phenomena and prepare them for further studies.

portfolio

publications

Classification of Alzheimer’s using Deep-learning Methods on Webcam-based Gaze Data

Published in Proc. ACM Hum.-Comput. Interact, 7, ETRA, Article 157, 2023

There has been increasing interest in non-invasive predictors of Alzheimer’s disease (AD) as an initial screen for this condition. Previously, successful attempts leveraged eye-tracking and language data generated during picture narration and reading tasks. These results were obtained with high-end, expensive eye-trackers. Instead, we explore classification using eye-tracking data collected with a webcam, where our classifiers are built using a deep-learning approach. Our results show that the webcam gaze classifier is not as good as the classifier based on high-end eye-tracking data. However, the webcam-based classifier still beats the majority-class baseline classifier in terms of AU-ROC, indicating that predictive signals can be extracted from webcam gaze tracking. Hence, although our results indicate that there is still a long way to go before webcam gaze tracking can reach practical relevance, they still provide an encouraging proof of concept that this technology should be further explored as an affordable alternative to high-end eye-trackers for the detection of AD.

Recommended citation: Anuj Harisinghani, Harshinee Sriram, Cristina Conati, Giuseppe Carenini, Thalia Field, Hyeju Jang, and Gabriel Murray. (2023). "Classification of Alzheimer’s using Deep-learning Methods on Webcam-based Gaze Data." Proc. ACM Hum.-Comput. Interact, 7, ETRA, Article 157.

Evaluating the overall sensitivity of saliency-based explanation methods

Published in IJCAI-XAI workshop, 2023

We address the need to generate faithful explanations of “black box” Deep Learning models. Several tests have been proposed to determine aspects of faithfulness of explanation methods, but they lack cross-domain applicability and a rigorous methodology. Hence, we select an existing test that is model agnostic and is well-suited for comparing one aspect of faithfulness (i.e., sensitivity) of multiple explanation methods, and extend it by specifying formal thresholds and building criteria to determine the over-all sensitivity of the explanation method. We present examples of how multiple explanation methods for Convolutional Neural Networks can be compared using this extended methodology. Finally, we discuss the relationship between sensitivity and faithfulness and consider how the test can be adapted to assess different explanation methods in other domains.

Recommended citation: Harshinee Sriram, Cristina Conati. (2023). "Evaluating the overall sensitivity of saliency-based explanation methods." IJCAI-XAI 2023 workshop.

Classification of Alzheimer’s Disease with Deep Learning on Eye-tracking Data

Published in ICMI '23: Proceedings of the 25th International Conference on Multimodal Interaction, 2023

Existing research has shown the potential of classifying Alzheimer’s Disease (AD) from eye-tracking (ET) data with classifiers that rely on task-specific engineered features. In this paper, we investigate whether we can improve on existing results by using a Deep Learning classifier trained end-to-end on raw ET data. This classifier (VTNet) uses a GRU and a CNN in parallel to leverage both visual (V) and temporal (T) representations of ET data and was previously used to detect user confusion while processing visual displays. A main challenge in applying VTNet to our target AD classification task is that the available ET data sequences are much longer than those used in the previous confusion detection task, pushing the limits of what is manageable by LSTM-based models. We discuss how we address this challenge and show that VTNet outperforms the state-of-the-art approaches in AD classification, providing encouraging evidence on the generality of this model to make predictions from ET data.

Recommended citation: Harshinee Sriram, Cristina Conati, and Thalia Field. (2023). "Classification of Alzheimer's Disease with Deep Learning on Eye-tracking Data." ICMI '23: Proceedings of the 25th International Conference on Multimodal Interaction.

Initial results on personalizing explanations of AI hints in an ITS

Published in UMAP '24: Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization, 2024

Previous research on an Intelligent Tutoring System (referred to as ACSP), showed the need to personalize explanations of its AI-driven hints for users with low Need for Cognition (N4C) and low Conscientiousness (Cons.). Specifically, this work found that explanations should be provided to these users with the objective of increasing user interaction with them. In this paper, we present and evaluate design alterations to the original ACSP explanation interface aimed at achieving this objective. Our results provide initial evidence that the implemented personalization, in the form of the design alterations, had a positive impact on users with low N4C and Cons., by increasing attention to explanations and contributing to learning gains.

Recommended citation: Vedant Bahel, Harshinee Sriram, and Cristina Conati. (2024). "Initial results on personalizing explanations of AI hints in an ITS." UMAP '24: Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization.

Detecting when Users Disagree with Generated Captions

Published in ICMI '24 Companion: Companion Proceedings of the 26th International Conference on Multimodal Interaction, 2024

The pervasive integration of artificial intelligence (AI) into daily life has led to a growing interest in AI agents that can learn continuously. Interactive Machine Learning (IML) has emerged as a promising approach to meet this need, essentially involving human experts in the model training process, often through iterative user feedback. However, repeated feedback requests can lead to frustration and reduced trust in the system. Hence, there is increasing interest in refining how these systems interact with users to ensure efficiency without compromising user experience. Our research investigates the potential of eye tracking data as an implicit feedback mechanism to detect user disagreement with AI-generated captions in image captioning systems. We conducted a study with 30 participants using a simulated captioning interface and gathered their eye movement data as they assessed caption accuracy. The goal of the study was to determine whether eye tracking data can predict user agreement or disagreement effectively, thereby strengthening IML frameworks. Our findings reveal that, while eye tracking shows promise as a valuable feedback source, ensuring consistent and reliable model performance across diverse users remains a challenge.

Recommended citation: Omair Shahzad Bhatti, Harshinee Sriram, Abdulrahman Mohamed Selim, Cristina Conati, Michael Barz, and Daniel Sonntag. (2024). "Detecting when Users Disagree with Generated Captions." ICMI '24 Companion: Companion Proceedings of the 26th International Conference on Multimodal Interaction.

Personalizing Explanations of AI-Driven Hints to Users’ Characteristics: An Empirical Evaluation

Published in International Conference on Artificial Intelligence in Education (AIED 2025), 2025

The paper extends an existing Intelligent Tutoring System (ITS) that supports students’ learning via AI-driven personalized hints and can generate explanations to justify why/how the hints were generated. In this work, we investigate personalizing these hint explanations to students with low levels of two traits, Need for Cognition and Conscientiousness in order to enhance their engagement with the explanations, based on prior findings that these students generally do not ask for the explanations although they would benefit from them. We evaluate the effectiveness of the personalized hint explanations with a formal user study. Our results show that the personalization increases our target users’ interaction with the hint explanations, their understanding of the hints, and their learning. Hence, this work contributes to exiting initial evidence on the value of Personalized Explainable AI (PXAI) in education.

Recommended citation: Vedant Bahel, Harshinee Sriram, and Cristina Conati. (2025). "Personalizing Explanations of AI-Driven Hints to Users’ Characteristics: An Empirical Evaluation." Artificial Intelligence in Education (AIED 2025), Lecture Notes in Computer Science (LNAI, volume 15877):411–423.

Multimodal Classification of Alzheimer’s Disease by Combining Facial and Eye-Tracking Data

Published in Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259, 2025

In recent years, there has been growing interest in developing a non-invasive tool for detecting Alzheimer’s Disease (AD). Previous studies have shown that a single modality such as speech or eye-tracking (ET) data can be effective for classifying AD patients from healthy individuals. However, understanding the role of other modalities, and especially the integration of facial analysis with ET for enhancing dementia classification, remains under-explored. In this paper, we investigate whether we can leverage facial patterns in AD patients by building on EMOTION-FAN—a deep learning model initially developed for recognizing seven distinct human emotions, now fine-tuned for our facial analysis tasks. We also explore the efficacy of leveraging multimodal information by combining the results from the facial and ET data through a late fusion technique. Specifically, our approach uses a neural classifier to learn from raw ET data (VTNet) alongside the fine-tuned EMOTION-FAN model that learns from the facial data. Experimental results show that facial data gives superior results than ET data. Notably, we obtain higher scores when both modalities are combined, providing strong evidence that integrating multimodal data benefits performance on this task.

Recommended citation: Shih-Han Chou, Miini Teng, Harshinee Sriram, Chuyuan Li, Giuseppe Carenini, Cristina Conati, Thalia S. Field, Hyeju Jang, and Gabriel Murray. (2025). "Multimodal Classification of Alzheimer’s Disease by Combining Facial and Eye-Tracking Data." Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:219-232.

SWARM-FR: Benchmarking Virtual Cell Metrics

Submitted to NeurIPS, 2026

So excited to share more about my newest first-author work when it’s available! For now, here’s an abstract visualization of the work’s abstract. Is that abstract enough?

software

AI Learning Assistant

This prototype explores how Large Language Models (LLMs) can enhance education by offering a personalized and adaptive learning experience. The LLM complements an instructor’s role by providing tailored feedback, identifying knowledge gaps, and recommending targeted resources to students. This approach resonates with the core principles of personalized education, transforming the learning experience into a journey of self-discovery and growth. Here’s the link to the full article.

Determining Course Flexibility

The purpose of this prototype is to gain an understanding of what level of flexibility is offered to students in courses, due to the lack of aggregate knowledge and dataset of syllabi. This prototype analyzes syllabi using machine learning to determine the flexibility of courses, and displays the results on a dashboard. Here’s the link to the full article.

Digital Strategy Assistant

This prototype explores how Large Language Models (LLMs) can enhance digital learning by providing an accessible and interactive way for the general public, educators, and administrators to engage with the Digital Strategy Assistant (DSA). Acting as a conversational guide, the chatbot allows users to ask questions and receive tailored responses aligned with DSA principles and recommendations. This approach fosters a broader understanding of technology-enhanced learning, making digital literacy concepts and strategies more approachable and relevant across educational and public contexts. Here’s the link to the full article.

Grant Program Analytics

The Grant Program Analytics prototype aims to increase the discoverability of funds that are used to enrich student learning. The prototype leverages AWS technology to move data storage to the cloud and simplify data cleaning processes, and innovates on the structure and presentation of data with advanced filtering and dynamic views. Here’s the link to the full article.

Quantum Key Distribution (QKD) Kitchen

QKD Kitchen is an interactive web-based visualizer designed to demystify Quantum Key Distribution. In quantum cryptography, there are three primary families of protocols that serve as the foundation for modern implementations. If you are familiar with French cooking, think of these core QKD algorithms as Mother Sauces: foundational recipes that can be altered, built upon, and refined to create a plethora of useful alternatives (like turning an Espagnole into a Bordelaise). This platform provides an intuitive, step-by-step visual breakthrough of how security is established over quantum channels using these fundamental concepts. I enjoy low-stakes vibe coding.

Quantum 2048

Quantum 2048 is a special version of the classic game 2048, with a quantum twist. The rules of the game entangle, shift and tunnel as you use the principles of quantum mechanics and try to get the mythical 2048 tile (and even beyond).

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Heading 1

Heading 2

Heading 3

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.

Heading 1

Heading 2

Heading 3