Predicting diagnostic performance

Multiple choice tests are generally poor predictors of skilled performance in real world decision making, such as medical diagnosis. A central aspect of the SlideTutor project is to develop the necessary formalisms for measuring and predicting diagnostic performance. SlideTutor captures a wealth of data about intermediate steps and skills, from which our Hidden Markov Models learn a model of individual skill acquisition over time.  We are currently working on improving our low level models. Future work will focus on aggregating these models to predict student performance at the case level.

Measuring and enhancing metacognition

An often neglected aspect of expert medical performance is the ability to accurately predict one’s own performance. Ongoing projects in our group are focused on determining what aspects of tutoring lead to over-confidence and under-confidence in self assessment of performance.

Detecting and correcting cognitive biases

We are currently exploring whether SlideTutor can detect heuristic errors (cognitive biases) such as representativeness and pseudo-diagnosticity. Current projects include development of production rules to capture biases and development of measurement instruments for validating SlideTutor’s inferences. Future work will focus on how to use the system to reduce these errors.

Evaluating SlideTutor as a patient safety intervention

In collaboration with Dana Grzybicki, MD,PhD at University of Colorado, we are testing whether SlideTutor reduces diagnostic errors among practicing pathologists, when compared to standard continuing medical education. This work represents the first summative evaluation of an intelligent tutoring system in any medical domain.

Enhancement, deployment and integration of SlideTutor

We are now extending the SlideTutor system to other areas of pathology, enhancing the authoring system, and integrating the system into real workflows.

The Cancer Training Web – a distributed Intelligent Tutoring System

In collaboration with George Xu, MD, PhD at University of Pennsylvania, we are working on the development of technology for a distributed tutoring system. This project moves us closer to the vision of a Semantic Web for Medical Education.