HEALTHCARE ROBOTICS

Sensory Computing and Object Processing Entities

John Bachkosky, Alexandra Boukhvalova, Kevin Chou, William Gunnarsson,
James Ledwell, Brendan McTaggart, Xiaoqing Qian, Nicholas Rodgers, John Shi, Jason Yon
Mentor: Dr. Anil Deane Librarian: Elizabeth Soergel

Introduction Healthcare Companion Robot - ALVORES Mobility Platform

The healthcare industry is being revolutionized by computer-
based technologies, the newest and likely most influential of
which is artificial intelligence (AI), which can assist in patient
consultation, diagnostics, treatment, and other functions1. AI
systems can even improve healthcare operations, as they can
efficiently perform remedial tasks and accurately recommend
diagnoses or treatments using information stored in large
databases2,3. Team SCOPE will advance the state of AI in
healthcare by using object recognition and natural language
processing techniques to create a medical personal assistant.
Object Recognition
Image acquisition
The picture is taken without requiring user input.
Setting the image
The picture is cut into n pieces, called segments.
Uploading the segments
The segments are sent to IBMs Bluemix Services via the
Alchemy API.
Bluemix returns phrases describing each given segments.
Healthcare Applications:
Medication Retrieval
Disease Recognition
Patient Vital Monitoring
Doctors Assistant
Raspberry Pi (blue box) and Arduino (red box) communicate between each
Personal Healthcare
other and the Android tablet to control and move ALVORES. ALVORES
uses proximity sensors (orange box) and motors attached to each wheel
(yellow box) in order to efficiently move around.
Patient Information Database
The platform will steer autonomously towards a desired object.
Front view of ALVORES, Version 1. Shows Android Platform operating according to external input from Android
Tablet running Object Recognition Application. tablet.
The platform has four active states: forwards, backwards, turn
left, and turn right.
Component Integration Proximity sensors interrupt movement when detecting collisions
Platform will use a rack and pinion steering system

Timeline to Thesis Conference

Object recognition software run on three images with the Bluemix output
presented to the right of the input image. Time measurements detail how
quickly IBM Bluemix can identify each image.

Goal: Develop upon already existing software structures and
technologies: Natural Language Toolkit (NLTK) and IBMs Bluemix Text
Processing.
Already Completed / In
Development
Command, Statement, &
Query Processing Programs
Simple database for
statement storage
Natural Language Processing
Future Work
Expand database capabilities
Incorporate visually collected
information from robot into database
Robot capability to answer questions
and complete tasks based on prior
human interaction
References
1. Luxton, D. D. (2015). An Introduction to Artificial Intelligence in Behavioral and Mental
Health Care. Artificial Intelligence in Behavioral and Mental Health Care, 1.
2. Herrero-Zazo, M., Segura-Bedmar, I., Hastings, J., & Martnez, P. (2015). Application of
Domain Ontologies to Natural Language Processing: A Case Study for Drug-Drug
Interactions. International Journal of Information Retrieval Research (IJIRR), 5(3), 19-
38.
3. Tan, H., Xu, Y., Mao, Y., Tong, X., Griffin, W. B., Kannan, B., & DeRose, L. A. (2015,
May). An integrated vision-based robotic manipulation system for sorting surgical tools.
In Technologies for Practical Robot Applications (TePRA), 2015 IEEE International
Conference on (pp. 1-6). IEEE.
Acknowledgements
Text processing through NLTK software on
Special Thanks to: Dr. Anil Deane, Ms. Elizabeth Soergel, Dr. Frank
commands and statements. Verification on
type of query and part of speech tags are Coale, Dr. Kristan Skendall and the Gemstone Program Staff.
displayed.