A joint project of the Graduate School, Peabody College, and the Jean & Alexander Heard Library

Title page for ETD etd-05142019-201139

Type of Document Dissertation
Author Fan, Jing
Author's Email Address jingfan.jf@gmail.com
URN etd-05142019-201139
Title Models of Adaptation in Intelligent Human-Machine Interaction and Their Applications to Elder Care and Autism Spectrum Disorder Intervention
Degree PhD
Department Electrical Engineering
Advisory Committee
Advisor Name Title
Nilanjan Sarkar Committee Chair
Douglas Fisher Committee Member
Gabor Karsai Committee Member
Lorraine Mion Committee Member
Mitchell Wilkes Committee Member
  • affective computing
  • machine intelligence
  • human-machine interaction
Date of Defense 2019-05-13
Availability unrestricted
The role of human-machine interaction (HMI) has been increasingly important in our everyday lives. This dissertation focused on creating formal methods, algorithms, and architectures for adaptive HMI with specific applications to elder care and autism spectrum disorder (ASD) intervention. Human-machine systems have been explored to engage older adults in activity-oriented therapies and provide treatments for individuals with ASD. While these systems are promising, they are limited in their ability to i) understand the implicit mental states of a user; ii) provide many-to-one interaction with multiple users; and iii) generalize design methods for a variety of interaction scenarios and learn the relationship between users’ responses and system behaviors. This research seeks to address these limitations by developing models of people for mental states estimation, developing novel socially assistive robotic (SAR) systems, and designing generalized models of interaction including a model of machine.

First, a data-driven mental state models of individuals with ASD based on their electroencephalogram (EEG) responses was built. Feature engineering and machine learning methods were used to recognize four affective states and mental workload. Results demonstrated the possibility of group-level affect and workload recognition with high accuracy during realistic driving tasks. Second, a novel multi-user engagement-based SAR architecture, ROCARE, for elder care was designed. ROCARE is a user-centric model tied to the core area of engagement and featured multi-user human-robot interaction (HRI) and individualized activity management. Three closed-loop SAR systems, which provided various physical, cognitive, and social stimuli using multimodal interaction, were developed based on ROCARE for one-to-one and triadic interaction with older adults. Laboratory and field studies on these systems indicated positive acceptance and engagement of older adults, the potential for ROCARE-based systems to involve more than one older adult, to facilitate interpersonal communication, and to quantitatively measure older adults’ social interaction and activity engagement. Finally, a new mathematical model for multi-user HRI that formally defined and integrated a model of user, a model of robot, and a model of interaction was proposed. The model performed online planning during a simulated multi-user HRI task.

  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  Dissertation_JF.pdf 5.10 Mb 00:23:35 00:12:08 00:10:37 00:05:18 00:00:27

Browse All Available ETDs by ( Author | Department )

If you have more questions or technical problems, please Contact LITS.