What is in common between the interaction between a tutor and a student, the interaction between a therapist and a patient, and the interaction between a customer and a service staff member? All these types of social interactions are provided as professional service that involves built-in roles (parties that participate in the interaction) and goals (performance metrics that ought to be improved):
What is in common between the interaction between a teacher and a student, the interaction between a therapist and a patient, and the interaction between a customer and a service staff member? All these types of social interactions are provided as professional service that
- Roles: therapist, patient, consultant, client, instructor, student, etc.
- Goals: correct diagnosis of illness, client satisfaction, provider performance, service efficacy, etc.
Traditionally, the evaluation and improvement of these goals relied on surveys. For example, at the end of a psychotherapy session, patients and therapists fill out a form reflecting on how well the session went. At the end of a semester, students fill out surveys about their learning experience and the instructor’s performance. However, the multi-modal social signals exchanged during the interaction that can now be captured by sensing technologies such as eye trackers, facial recognition, and physiology wearables, may offer rich and just-in-time evidence for the quality of social interaction experienced by individuals in all roles.
In this project, we aim to extract real-time digital biomarkers of social experience in professional interactions from multi-modal sensor data and explore the possibility of feeding these insights back to the service provider to improve task performance.
Status of this project: actively seeking students and collaborators.
Preliminary data: (1) medical-grade wearable sensor data from two individuals engaging in mock conversations; (2) de-identified, transcribed dialogue text between psychiatrists and patients in Zoom counseling sessions.