Sitting

on Pins and Needles: Characterization of Symptom Descriptions in Clinical Notes

Tyler B. Forbush, CRT Salt Lake City VA Health Care System Department of Internal Medicine, University of Utah

Background
The VA continues to study medically unexplained syndromes such as Chronic Fatigue, Fibromyalgia, and Irritable Bowel Syndrome and their symptoms. MUS are common in the veteran population. MUS have much in common with other chronic multi-symptom illnesses, like gulf war syndrome.

Symptoms
Important for diagnosis, assessment of severity and response to treatment, therapeutic decision making, phenotypic classication. *Both at the time of patient care, and for retrospective research

Study Aims
Given that your study requires symptom data, how could you get it on a large scale? Use existing code data (ICD-9-CM) NLP
An o-the-shelf system that uses terminologies A custom system that takes all your time/money

How are symptoms recorded?

Symptom Terms:
written in medical terminology

Subjective Symptom Expressions:

Quotes or capture patients voice Figures of speech Idioms Lay terms

Methods
Cohort included all patients seen in the VA in 2009. 750 documents were randomly selected from all text notes for these patients and grouped into Mental health/social work, and Primary/Specialty 4 health care providers with annotation experience conducted a 2 layer review. Reviewers were trained for each layer until a stable IAA >.80 was achieved. 2 reviewers annotated each note independently, disagreements were adjudicated by a 3rd.

Methods
Layer 1:
Annotate all symptoms and subjective symptom expressions.

Layer 2: Review SSE annotations to

Classify into symptom types Determine if the symptom was code Determine if SSE was also mentioned in words that could be mapped to a terminology. i.e. symptom term

Classication
Somatic:
Loss of somatic function Unpleasant or altered sensation Somatic, non-sensory, not functional

Cognitive / Behavioral / Mental

Cognitive Dysfunction False Perceptions/beliefs Emotional Distress Harm to self/others Behavioral Dysfunction

ICD9-CM Coding
ICD9-CM coding for the encounter was reviewed to see if the SSE was coded
If the SSE was coded, it was considered explicitly coded
Feels like a vice on my head = Code for Headache

If the SSE was a plausible manifestation of the code, it was considered contained by
I dont even want to get out of bed contained by Major Depression code

Restated in Symptom Term

The SSE were compared to Symptom Terms in the same note.
If the SSE was also restated in a symptom term, it may be feasible to use an NLP pipeline that utilizes standard terminologies or dictionaries.
I cant keep food down restated as vomiting

Results
5,031 Symptom Terms and 543 SSE Only 170 notes contained SSE

Results - Classication
Emotional distress, behavioral dysfunction, and loss of somatic function were common.

Results ICD9-CM
Only 2 SSE were explicitly coded, both were headache, and both were restated. 79 were contained by but not restated 36% of SSE is the best you could do using coding if you accept imprecision.

Results - Restated

Limitations
176 of the documents had no coding for the encounter. In some settings, coding may be more inclusive of some symptoms The VA uses ICD-9-CM, ICD-10 may oer better results

Conclusions
Coding wont help you much.
Only headaches were truly coded The best you could get is 36%

NLP systems that extract symptom terms gets you 55%. The rest will require more work.

The Full Story

These subjective symptom expressions contain a depth of information not found in medical terms. 69 y/o patient was referred for Suicidal Ideation. thinking about his deceased wife, he put a gun to his head for a minute, then went to bed

The Full Story

SSE communicate the nature and severity of symptoms.
My knee doesnt feel right
I wish I could just cut my leg o

They often indicate the impact of the symptom on the patient.

Functional status and Quality of life measures
I never make it to the bathroom in time I havent left my room in weeks, Im so freaked out

Future Work
Using NLP to locate phrases that may be a Subjective Symptom Expression and referring them out for human classication, then feeding these back to the system. Preliminary use of these symptom expressions in PTSD and functional status studies

Questions and Comments?

Adi V. Gundlapalli, MD PhD, Miland N. Palmer, MPH RHIA, Shuying Shen, MStat, Brett R. South MS, Guy Divita MS, Marjorie Carter, MPH, Andrew Redd PhD, Jorie M. Butler PhD, Matthew Samore, MD
This work was supported by the Department of Veterans Aairs, Oce of Research and Development, Health Services Research and Development, ProWATCH: Epidemiology of Medically Unexplained Syndromes project # HIR 10-001.