A joint project of the Graduate School, Peabody College, and the Jean & Alexander Heard Library

Title page for ETD etd-12022013-124632

Type of Document Master's Thesis
Author Gobbel, Glenn Temple
Author's Email Address glenn.t.gobbel@vanderbilt.edu
URN etd-12022013-124632
Title Assisted annotation of biomedical text using rapTAT, an online learning-based tool
Degree Master of Science
Department Biomedical Informatics
Advisory Committee
Advisor Name Title
Michael E. Matheny, M.D., M.S., M.P.H. Committee Chair
Dario Guise, Dr. Ing. Committee Member
Steven H. Brown, M.D., M.S. Committee Member
  • NLP
  • medical informatics computing
  • pre-annotation
  • clinical informatics
  • Bayesian prediction
  • interactive machine learning
Date of Defense 2013-09-23
Availability unrestricted
Clinical providers frequently document patient encounters using unstructured text. Although such text contains a wealth of data useful for improving healthcare, extracting it can be difficult. Natural language processing systems can help with data extraction, but system training and testing still require the costly generation of an annotated document corpus to serve as a reference standard. To reduce this cost, we developed the Rapid Text Annotation Tool (RapTAT), which pre-annotates documents based on user feedback and online machine learning. The tool gradually shifts the manual annotation task from identification and labeling of text phrases to one of review and correction. The tool was evaluated in a use case involving manual or assisted annotation of 404 clinical notes for concepts related to quality of care during heart failure treatment. Notes were divided into 20 batches of 19-21 documents for iterative annotation and training. The annotation rate increased significantly during the review process for RapTAT-assisted but not manual reviewers. F-measure of RapTAT pre-annotations increased from 0.5-0.6 to >0.80 (relative to both assisted reviewer and reference annotations) over the first 3 batches and more slowly thereafter. The tool significantly reduced overall workload by approximately 50% by both decreasing the number of annotations that had to be manually added and helping reviewers to annotate at an increased rate. Assisted annotation did not appear to reduce annotation quality; overall inter-annotator agreement was significantly higher between RapTAT-assisted than manual reviewers. This study demonstrates that online machine learning can substantially decrease the burden of annotating medical text.
  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  Gobbel.pdf 5.59 Mb 00:25:51 00:13:18 00:11:38 00:05:49 00:00:29

Browse All Available ETDs by ( Author | Department )

If you have more questions or technical problems, please Contact LITS.