publications

2024

1. AIIM

   

   Evaluating the clinical utility of artificial intelligence assistance and its explanation on the glioma grading task

   Weina Jin, Mostafa Fatehi, Ru Guo, and Ghassan Hamarneh

   Artificial Intelligence in Medicine, 2024

   Abs DOI Bib HTML PDF Video Blog Code Slides

   Clinical evaluation evidence and model explainability are key gatekeepers to ensure the safe, accountable, and effective use of artificial intelligence (AI) in clinical settings. We conducted a clinical user-centered evaluation with 35 neurosurgeons to assess the utility of AI assistance and its explanation on the glioma grading task. Each participant read 25 brain MRI scans of patients with gliomas, and gave their judgment on the glioma grading without and with the assistance of AI prediction and explanation. The AI model was trained on the BraTS dataset with 88.0% accuracy. The AI explanation was generated using the explainable AI algorithm of SmoothGrad, which was selected from 16 algorithms based on the criterion of being truthful to the AI decision process. Results showed that compared to the average accuracy of 82.5±8.7% when physicians performed the task alone, physicians’ task performance increased to 87.7±7.3% with statistical significance (p-value = 0.002) when assisted by AI prediction, and remained at almost the same level of 88.5±7.0% (p-value = 0.35) with the additional assistance of AI explanation. Based on quantitative and qualitative results, the observed improvement in physicians’ task performance assisted by AI prediction was mainly because physicians’ decision patterns converged to be similar to AI, as physicians only switched their decisions when disagreeing with AI. The insignificant change in physicians’ performance with the additional assistance of AI explanation was because the AI explanations did not provide explicit reasons, contexts, or descriptions of clinical features to help doctors discern potentially incorrect AI predictions. The evaluation showed the clinical utility of AI to assist physicians on the glioma grading task, and identified the limitations and clinical usage gaps of existing explainable AI techniques for future improvement.

   @article{JIN2024102751,
     title = {Evaluating the clinical utility of artificial intelligence assistance and its explanation on the glioma grading task},
     journal = {Artificial Intelligence in Medicine},
     volume = {148},
     pages = {102751},
     year = {2024},
     issn = {0933-3657},
     doi = {https://doi.org/10.1016/j.artmed.2023.102751},
     url = {https://www.sciencedirect.com/science/article/pii/S0933365723002658},
     author = {Jin, Weina and Fatehi, Mostafa and Guo, Ru and Hamarneh, Ghassan},
     keywords = {Artificial intelligence, Neuro-imaging, Neurosurgery, Explainable artificial intelligence, Clinical study, Human-centered artificial intelligence},
     project = {brain_ai_clinical_study},
   }

2023

1. MedIA

   

   Guidelines and evaluation of clinical explainable AI in medical image analysis

   Weina Jin, Xiaoxiao Li, Mostafa Fatehi, and Ghassan Hamarneh

   Medical Image Analysis, 2023

   Abs DOI arXiv Bib HTML PDF Blog Code

   Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1) Understandability and G2 Clinical relevance. For the chosen explanation form, its specific XAI technique should be optimized for G3 Truthfulness, G4 Informative plausibility, and G5 Computational efficiency. Following the guidelines, we conducted a systematic evaluation on a novel problem of multi-modal medical image explanation with two clinical tasks, and proposed new evaluation metrics accordingly. Sixteen commonly-used heatmap XAI techniques were evaluated and found to be insufficient for clinical use due to their failure in G3 and G4. Our evaluation demonstrated the use of Clinical XAI Guidelines to support the design and evaluation of clinically viable XAI.

   @article{JIN2023102684,
     title = {Guidelines and evaluation of clinical explainable AI in medical image analysis},
     journal = {Medical Image Analysis},
     volume = {84},
     pages = {102684},
     year = {2023},
     issn = {1361-8415},
     doi = {https://doi.org/10.1016/j.media.2022.102684},
     url = {https://www.sciencedirect.com/science/article/pii/S1361841522003127},
     author = {Jin, Weina and Li, Xiaoxiao and Fatehi, Mostafa and Hamarneh, Ghassan},
     keywords = {Interpretable machine learning, Medical image analysis, Multi-modal medical image, Explainable AI evaluation},
     project = {xai_eval},
   }

2. MethodsX

   

   Generating post-hoc explanation from deep neural networks for multi-modal medical image analysis tasks

   Weina Jin, Xiaoxiao Li, Mostafa Fatehi, and Ghassan Hamarneh

   MethodsX, 2023

   Abs DOI Bib HTML PDF Code

   Explaining model decisions from medical image inputs is necessary for deploying deep neural network (DNN) based models as clinical decision assistants. The acquisition of multi-modal medical images is pervasive in practice for supporting the clinical decision-making process. Multi-modal images capture different aspects of the same underlying regions of interest. Explaining DNN decisions on multi-modal medical images is thus a clinically important problem. Our methods adopt commonly-used post-hoc artificial intelligence feature attribution methods to explain DNN decisions on multi-modal medical images, including two categories of gradient- and perturbation-based methods. • Gradient-based explanation methods – such as Guided BackProp, DeepLift – utilize the gradient signal to estimate the feature importance for model prediction. • Perturbation-based methods – such as occlusion, LIME, kernel SHAP – utilize the input-output sampling pairs to estimate the feature importance. • We describe the implementation details on how to make the methods work for multi-modal image input, and make the implementation code available.

   @article{JIN2023102009,
     title = {Generating post-hoc explanation from deep neural networks for multi-modal medical image analysis tasks},
     journal = {MethodsX},
     volume = {10},
     pages = {102009},
     year = {2023},
     issn = {2215-0161},
     doi = {https://doi.org/10.1016/j.mex.2023.102009},
     url = {https://www.sciencedirect.com/science/article/pii/S2215016123000146},
     author = {Jin, Weina and Li, Xiaoxiao and Fatehi, Mostafa and Hamarneh, Ghassan},
     keywords = {Interpretable machine learning, Explainable artificial intelligence, Medical image analysis, Multi-modal medical image, Post-hoc explanation},
     project = {xai_eval},
   }

2022

1. AAAI

   

   Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?

   Weina Jin, Xiaoxiao Li, and Ghassan Hamarneh

   Proceedings of the AAAI Conference on Artificial Intelligence, Jun 2022

   Abs DOI arXiv Bib HTML PDF Blog Code Poster Slides

   Being able to explain the prediction to clinical end-users is a necessity to leverage the power of artificial intelligence (AI) models for clinical decision support. For medical images, a feature attribution map, or heatmap, is the most common form of explanation that highlights important features for AI models’ prediction. However, it is unknown how well heatmaps perform on explaining decisions on multi-modal medical images, where each image modality or channel visualizes distinct clinical information of the same underlying biomedical phenomenon. Understanding such modality-dependent features is essential for clinical users’ interpretation of AI decisions. To tackle this clinically important but technically ignored problem, we propose the modality-specific feature importance (MSFI) metric. It encodes clinical image and explanation interpretation patterns of modality prioritization and modality-specific feature localization. We conduct a clinical requirement-grounded, systematic evaluation using computational methods and a clinician user study. Results show that the examined 16 heatmap algorithms failed to fulfill clinical requirements to correctly indicate AI model decision process or decision quality. The evaluation and MSFI metric can guide the design and selection of explainable AI algorithms to meet clinical requirements on multi-modal explanation.

   @article{Jin_Li_Hamarneh_2022,
     title = {Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?},
     volume = {36},
     url = {https://ojs.aaai.org/index.php/AAAI/article/view/21452},
     doi = {10.1609/aaai.v36i11.21452},
     number = {11},
     journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
     author = {Jin, Weina and Li, Xiaoxiao and Hamarneh, Ghassan},
     year = {2022},
     month = jun,
     pages = {11945-11953},
     project = {xai_eval_aaai},
   }

2. 

   What Explanations Do Doctors Require From Artificial Intelligence?

   Weina Jin and Ghassan Hamarneh

   Manuscript in preparation, 2022

   Bib Blog

   @article{jin2022doctorUserStudy,
     author = {Jin, Weina and Hamarneh, Ghassan},
     journal = {Manuscript in preparation},
     title = {What Explanations Do Doctors Require From Artificial Intelligence?},
     year = {2022},
     project = {jin2022doctorUserStudy},
   }

2021

1. ICML-w

   

   One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images

   Weina Jin, Xiaoxiao Li, and Ghassan Hamarneh

   ICML 2021 Workshop on Interpretable Machine Learning in Healthcare, 2021

   Abs arXiv Bib PDF Blog Code Poster Slides

   Being able to explain the prediction to clinical end-users is a necessity to leverage the power of AI models for clinical decision support. For medical images, saliency maps are the most common form of explanation. The maps highlight important features for AI model’s prediction. Although many saliency map methods have been proposed, it is unknown how well they perform on explaining decisions on multi-modal medical images, where each modality/channel carries distinct clinical meanings of the same underlying biomedical phenomenon. Understanding such modality-dependent features is essential for clinical users’ interpretation of AI decisions. To tackle this clinically important but technically ignored problem, we propose the MSFI (Modality-Specific Feature Importance) metric to examine whether saliency maps can highlight modality-specific important features. MSFI encodes the clinical requirements on modality prioritization and modality-specific feature localization. Our evaluations on 16 commonly used saliency map methods, including a clinician user study, show that although most saliency map methods captured modality importance information in general, most of them failed to highlight modality-specific important features consistently and precisely. The evaluation results guide the choices of saliency map methods and provide insights to propose new ones targeting clinical applications.

   @article{jin2021one_map_not_fit_all,
     author = {Jin, Weina and Li, Xiaoxiao and Hamarneh, Ghassan},
     title = {One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal
                    Medical Images},
     journal = {ICML 2021 Workshop on Interpretable Machine Learning in Healthcare},
     year = {2021},
     url = {https://arxiv.org/abs/2107.05047},
     eprinttype = {arXiv},
     timestamp = {Tue, 20 Jul 2021 15:08:33 +0200},
     biburl = {https://dblp.org/rec/journals/corr/abs-2107-05047.bib},
     bibsource = {dblp computer science bibliography, https://dblp.org},
     project = {one_map},
   }

2. arXiv

   

   EUCA: the End-User-Centered Explainable AI Framework

   Weina Jin, Jianyu Fan, Diane Gromala, Philippe Pasquier, and Ghassan Hamarneh

   2021

   Abs arXiv Bib PDF Blog Code Poster Website

   The ability to explain decisions to end-users is a necessity to deploy AI as critical decision support. Yet making AI explainable to non-technical end-users is a relatively ignored and challenging problem. To bridge the gap, we first identify twelve end-user- friendly explanatory forms that do not require technical knowledge to comprehend, including feature-, example-, and rule-based explanations. We then instantiate the explanatory forms as prototyping cards in four AI-assisted critical decision-making tasks, and conduct a user study to co-design low-fidelity prototypes with 32 layperson participants. The results confirm the relevance of using explanatory forms as building blocks of explanations, and identify their proprieties — pros, cons, applicable explanation goals, and design implications. The explanatory forms, their proprieties, and prototyping supports (including a suggested prototyping process, design templates and exemplars, and associated algorithms to actualize explanatory forms) constitute the End-User-Centered explainable AI framework EUCA, and is available at http://weinajin.github.io/end-user-xai. It serves as a practical prototyping toolkit for HCI/AI practitioners and researchers to understand user requirements and build end-user-centered explainable AI.

   @article{jin2021euca,
     title = {{EUCA}: the End-User-Centered Explainable {AI} Framework},
     author = {Jin, Weina and Fan, Jianyu and Gromala, Diane and Pasquier, Philippe and Hamarneh, Ghassan},
     url = {https://arxiv.org/abs/2102.02437},
     year = {2021},
     archiveprefix = {arXiv},
     primaryclass = {cs.HC},
     project = {euca},
   }

2020

1. JNE

   

   Artificial Intelligence in Glioma Imaging: Challenges and Advances

   Weina Jin, Mostafa Fatehi, Kumar Abhishek, Mayur Mallya, Brian Toyota, and Ghassan Hamarneh

   Journal of Neural Engineering, Apr 2020

   Abs DOI arXiv Bib HTML PDF Slides

   Primary brain tumors including gliomas continue to pose significant management challenges to clinicians. While the presentation, the pathology, and the clinical course of these lesions are variable, the initial investigations are usually similar. Patients who are suspected to have a brain tumor will be assessed with computed tomography (CT) and magnetic resonance imaging (MRI). The imaging findings are used by neurosurgeons to determine the feasibility of surgical resection and plan such an undertaking. Imaging studies are also an indispensable tool in tracking tumor progression or its response to treatment. As these imaging studies are non-invasive, relatively cheap and accessible to patients, there have been many efforts over the past two decades to increase the amount of clinically-relevant information that can be extracted from brain imaging. Most recently, artificial intelligence (AI) techniques have been employed to segment and characterize brain tumors, as well as to detect progression or treatment-response. However, the clinical utility of such endeavours remains limited due to challenges in data collection and annotation, model training, and the reliability of AI-generated information. We provide a review of recent advances in addressing the above challenges. First, to overcome the challenge of data paucity, different image imputation and synthesis techniques along with annotation collection efforts are summarized. Next, various training strategies are presented to meet multiple desiderata, such as model performance, generalization ability, data privacy protection, and learning with sparse annotations. Finally, standardized performance evaluation and model interpretability methods have been reviewed. We believe that these technical approaches will facilitate the development of a fully-functional AI tool in the clinical care of patients with gliomas.

   @article{Jin_2020_review_brain_ai,
     author = {Jin, Weina and Fatehi, Mostafa and Abhishek, Kumar and Mallya, Mayur and Toyota, Brian and Hamarneh, Ghassan},
     doi = {10.1088/1741-2552/ab8131},
     journal = {Journal of Neural Engineering},
     month = apr,
     number = {2},
     pages = {21002},
     publisher = {{\{}IOP{\}} Publishing},
     title = {{Artificial Intelligence in Glioma Imaging: Challenges and Advances}},
     volume = {17},
     year = {2020},
   }

2019

1. IEEE VIS poster

   

   Bridging AI Developers and End Users: an End-User-Centred Explainable AI Taxonomy and Visual Vocabularies

   Weina Jin, Sheelagh Carpendale, Ghassan Hamarneh, and Diane Gromala

   In IEEE VIS 2019 Conference Poster Abstract, 2019

   Abs Bib HTML PDF Supp Video Poster Slides

   Researchers in the re-emerging field of explainable/interpretable artificial intelligence (XAI) have not paid enough attention to the end users of AI, who may be lay persons or domain experts such as doctors, drivers, and judges. We took an end-user-centric lens and conducted a literature review of 59 technique papers on XAI algorithms and/or visualizations. We grouped the existing explanatory forms in the literature into the end-user-friendly XAI taxonomy. It consists of three forms that explain AI’s decisions: feature attribute , instance, and decision rules/trees. We also analyzed the visual representations for each explanatory form, and summarized them as the XAI visual vocabularies. Our work is a synergy of XAI algorithm, visualization, and user-centred design. It provides a practical toolkit for AI developers to define the explanation problem from a user-centred perspective, and expand the visualization space of explanations to develop more end-user-friendly XAI systems.

   @inproceedings{Jin2019vis,
     author = {Jin, Weina and Carpendale, Sheelagh and Hamarneh, Ghassan and Gromala, Diane},
     keywords = {Explainable AI,Visualization},
     booktitle = {IEEE VIS 2019 Conference Poster Abstract},
     title = {{Bridging AI Developers and End Users: an End-User-Centred Explainable AI Taxonomy and Visual Vocabularies}},
     year = {2019},
     project = {euca},
   }

2018

1. IEEE GEM

   Automatic Prediction of Cybersickness for Virtual Reality Games

   Weina Jin, Jianyu Fan, Diane Gromala, and Philippe Pasquier

   In 2018 IEEE Games, Entertainment, Media Conference (GEM), 2018

   Abs DOI Bib PDF

   Cybersickness, which is also called Virtual Reality (VR) sickness, poses a significant challenge to the VR user experience. Previous work demonstrated the viability of predicting cybersickness for VR 360°videos. Is it possible to automatically predict the level of cybersickness for interactive VR games? In this paper, we present a machine learning approach to automatically predict the level of cybersickness for VR games. First, we proposed a novel ranking-rating (RR) score to measure the ground-truth annotations for cybersickness. We then verified the RR scores by comparing them with the Simulator Sickness Questionnaire (SSQ) scores. Next, we extracted features from heterogeneous data sources including the VR visual input, the head movement, and the individual characteristics. Finally, we built three machine learning models and evaluated their performances: the Convolutional Neural Network (CNN) trained from scratch, the Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) trained from scratch, and the Support Vector Regression (SVR). The results indicated that the best performance of predicting cybersickness was obtained by the LSTM-RNN, providing a viable solution for automatically cybersickness prediction for interactive VR games.

   @inproceedings{jin2018cybersickness,
     author = {Jin, Weina and Fan, Jianyu and Gromala, Diane and Pasquier, Philippe},
     booktitle = {2018 IEEE Games, Entertainment, Media Conference (GEM)},
     title = {Automatic Prediction of Cybersickness for Virtual Reality Games},
     year = {2018},
     volume = {},
     number = {},
     pages = {1-9},
     doi = {10.1109/GEM.2018.8516469},
   }

2017

1. CSCW poster

   

   A Collaborative Visualization Tool to Support Doctors’ Shared Decision-Making on Antibiotic Prescription

   Weina Jin, Diane Gromala, Carman Neustaedter, and Xin Tong

   In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Portland, Oregon, USA, 2017

   Abs DOI Bib PDF Poster

   The inappropriate prescription of antibiotics may cause severe medical outcomes such as antibiotic resistance. To prevent such situations and facilitate appropriate antibiotic prescribing, we designed and developed an asynchronous collaborative visual analytics tool. It visualizes the antibiotics’ coverage spectrum that allows users choose the most appropriate antibiotics. The asynchronous collaboration around visualization mimics the actual collaboration scenarios in clinical settings, and provides supportive information during physician’s decision-making process. Our work contributes to the CSCW community by providing a design prototype to support asynchronous collaboration among healthcare professionals, which is crucial but lacks in many of the present clinical decision support systems.

   @inproceedings{jin2017cscw,
     author = {Jin, Weina and Gromala, Diane and Neustaedter, Carman and Tong, Xin},
     title = {A Collaborative Visualization Tool to Support Doctors' Shared Decision-Making on Antibiotic Prescription},
     year = {2017},
     isbn = {9781450346887},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi-org.proxy.lib.sfu.ca/10.1145/3022198.3026311},
     doi = {10.1145/3022198.3026311},
     booktitle = {Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing},
     pages = {211–214},
     numpages = {4},
     keywords = {clinical decision support, asynchronous collaboration, information visualization},
     location = {Portland, Oregon, USA},
     series = {CSCW '17 Companion},
   }

2016

1. CHI poster

   

   AS IF: A Game as an Empathy Tool for Experiencing the Activity Limitations of Chronic Pain Patients

   Weina Jin, Servet Ulas, and Xin Tong

   In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, California, USA, 2016

   Abs DOI Bib PDF Slides

   Pain is both a universal and unique experience for its sufferers. Nonetheless, pain is also invisible and incommunicable that it becomes difficult for the public to understand or even believe the suffering, especially for the persistent form of pain: Chronic Pain. Therefore, we designed and developed the game – AS IF – to foster non-patients’ empathy for Chronic Pain sufferers. In this game, players engage with the connecting dots tasks through whole body interaction. After they generate the connection with their virtual body, they will experience a certain degree of activity limitation that mimics one of the sufferings of Chronic Pain. In this paper, we introduce the game design that facilitates the enhancement of empathy for Chronic Pain experience, and illustrate how this game acts as a form of communication media that may help to enhance understanding.

   @inproceedings{jin2016asif,
     author = {Jin, Weina and Ulas, Servet and Tong, Xin},
     title = {AS IF: A Game as an Empathy Tool for Experiencing the Activity Limitations of Chronic Pain Patients},
     year = {2016},
     isbn = {9781450340823},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi-org.proxy.lib.sfu.ca/10.1145/2851581.2890369},
     doi = {10.1145/2851581.2890369},
     booktitle = {Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems},
     pages = {172–175},
     numpages = {4},
     keywords = {chronic pain experience, gaming for a purpose, mobility, stimulate empathy, body interaction},
     location = {San Jose, California, USA},
     series = {CHI EA '16},
   }

2015

1. IEEE GEM poster

   

   Serious game for serious disease: Diminishing stigma of depression via game experience

   Weina Jin, Diane Gromala, and Xin Tong

   In 2015 IEEE Games Entertainment Media Conference (GEM), 2015

   Abs DOI Bib PDF Code Poster Slides

   Stigma is a common and serious problem for patients who suffer from depression and other mental illnesses. We designed a serious game to address this problem. The game enables player to experience and strive to overcome the disempowering aspects of depression during the journey to recovery. Through the game’s interaction, player may gain a better understanding of the relationship between patient and the disease, which in turn may help to change the players’ moral model with the disease model, and thereby diminish the stigma of depression.

   @inproceedings{jin2015likemind,
     author = {Jin, Weina and Gromala, Diane and Tong, Xin},
     booktitle = {2015 IEEE Games Entertainment Media Conference (GEM)},
     title = {Serious game for serious disease: Diminishing stigma of depression via game experience},
     year = {2015},
     volume = {},
     number = {},
     pages = {1-2},
     doi = {10.1109/GEM.2015.7377256},
     project = {likemind},
   }