Christina Magnifico LI804XS Annotated List of Themes Evaluation of Medical Information Retrieval Systems Medical information retrieval systems

are an important aspect of scientific research, but with new systems being developed more frequently than before, each must be evaluated in order to make sure they provide both useful and accurate information to clinicians. Several of the articles reviewed (Demiris, Folk, Mitchell, Moxley, Patrick & Tao, 2003 Grandage, Slawson & Shaughnessy, 2002 Liu & Wyatt 2011) evaluated the literature based on the type of study returned. While Demiris (2003) focused on the number of meta-analysis returned; Grandage, Slawson and Shaughnessy (2002) used a simple equation in order to evaluate the usefulness of articles returned from a medical information retrieval system:

Liu and Wyatt (2011), assessed the number of randomized-controlled trials returned in a search of clinical information retrieval systems and how that affected the usefulness of the results. When evaluating an information retrieval system, Grandage (2002) places increased emphasis on the role medical librarians should take in order to provide the most assistance to clinicians. Other articles (Elliot, Hersh, Hickam & Wolf, 1994 Masic & Milinovic, 2012 Braithwaite, Coiera & Westbrook, 2005) focus primarily on evaluating the entire medical information retrieval system as a whole. Cimino, Ely, Lee, Sable, Shanker and Zhu (2006) evaluated medical information retrieval systems based on the number of documents returned after using a question answering technique, while MacCall (2006) focused

Christina Magnifico LI804XS on assessing information retrieval systems and clinical digital libraries in terms of their facilitation of timely clinical information seeking.

Health Professionals Use of Medical Information Retrieval Systems One of the most frequent themes in the review of the literature was how clinicians, researchers and physicians use medical information retrieval systems. A multitude of studies (Cimino, Hunt & Koziol, 2012 Clevenger & Shelstad, 1996 Coiera, Gosling & Westbrook, 2005 Fauquert, 2012 Grad, Meng, Pluye, Segal & Tamblyn, 2005 Grefsheim & Rankin, 2007 Hersh & Hickam, 1998) focused on IRS usage by the medical community. Though each of the studies sought to identify the way different medical professionals were utilizing medical information retrieval systems, their methodologies were not always the same. Cimino (2012) compared medical information retrieval systems for use in different environments (academic, versus hospital), while Coiera (2005) compared the difference the use of clinical information systems made in pre- versus post-intervention question answering by clinicians. Clevenger (1996) analyzed the retrieval patterns among general surgeons, and Fauquert (2012) discusses the effect the dissemination of information via retrieved records has had on Belgian physicians. The 2005 article by Grad, assessed the impact of clinical information retrieval systems on a family practice. Grefsheim (2007) and Hersh (1998) both sought to assess the information needs of clinicians, while also evaluating how well the medical professionals were able to use medical information retrieval systems.

Christina Magnifico LI804XS Performing Search Queries in Medical Information Retrieval Systems The last theme that was sufficiently studied in the literature was how to perform search queries in medical information retrieval systems. In evaluating the literature, many of the articles (Abernethy, Currow, Fazekas, Sladek & Tieman 2006 Carlson, Cvitan, Krieger, Lavin, McNary, Meyer, Perry, Reese & Spasser, 2005 Meyer & Sarin, 2005 Stoddart & Workman, 2012) described studies on the usage of filters in medical information retrieval system searching. Each article used a different set of filters: palliative care search filters (Abernathy, 2006), evidence-based search filters (Carlson, 2005), anesthesia-related search filters (Meyer & Sarin, 2005), and natural language filters (Stoddart & Workman, 2012). A 2013 study by Mosa and a 2011 study by Smith, both focused on the usage of tags in medical information retrieval system queries. Alpi (2005) published a study on expert searching in public health, using evidence-based search strategies common among public health professionals.

Christina Magnifico LI804XS References

Evaluation of Medical Information Retrieval Systems Grandage, K. K., Slawson, D. C., & Shaughnessy, A. F. (2002). When less is more: a practical approach to searching for evidence-based answers. Journal of the Medical Library Association, 90(3), 298304. Hersh, W. R., Elliot, D. L., Hickam, D. H., Wolf, S. L., Molnar, A., & Leichtenstien, C. (1994). Towards new measures of information retrieval evaluation. Proceedings of the Annual Symposium on Computer Application in Medical Care, 895899. Lee, M., Cimino, J., Zhu, H. R., Sable, C., Shanker, V., Ely, J., & Yu, H. (2006). Beyond information retrieval--medical question answering. AMIA Annual Symposium Proceedings, 2006, 469473. Liu, J. L. Y., & Wyatt, J. C. (2011). The case for randomized controlled trials to assess the impact of clinical information systems. Journal of the American Medical Informatics Association: JAMIA, 18(2), 173180. doi:10.1136/jamia.2010.010306 MacCall, S. L. (2006). Clinical digital libraries project: design approach and exploratory assessment of timely use in clinical environments. Journal of the Medical Library Association, 94(2), 190197. Masic, I., & Milinovic, K. (2012). On-line biomedical databases- the best source for quick search of the scientific information in the biomedicine. Acta Informatica Medica, 20(2), 7284. doi:10.5455/aim.2012.20.72-84

Christina Magnifico LI804XS Patrick, T. B., Demiris, G., Folk, L. C., Moxley, D. E., Mitchell, J. A., & Tao, D. (2004). Evidence-based retrieval in evidence-based medicine. Journal of the Medical Library Association, 92(2), 196199. Westbrook, J. I., Coiera, E. W., & Braithwaite, J. (2005). Measuring the impact of online evidence retrieval systems using critical incidents & journey mapping. Studies in health technology and informatics, 116, 533538. Health Professionals Use of Medical Information Retrieval Systems Fauquert, B. (2012). [From library to clinical decision support systems: access of general practitioner to quality information]. Revue mdicale de Bruxelles, 33(4), 400406. Grad, R. M., Pluye, P., Meng, Y., Segal, B., & Tamblyn, R. (2005). Assessing the impact of clinical information-retrieval technology in a family practice residency. Journal of Evaluation in Clinical Practice, 11(6), 576586. doi:10.1111/j.1365-2753.2005.00594.x Grefsheim, S. F., & Rankin, J. A. (2007). Information needs and information seeking in a biomedical research setting: a study of scientists and science administrators. Journal of the Medical Library Association: JMLA, 95(4), 426434. doi:10.3163/15365050.95.4.426 Hersh WR, & Hickam DH. (1998). How well do physicians use electronic information retrieval systems?: A framework for investigation and systematic review. JAMA, 280(15), 13471352. doi:10.1001/jama.280.15.1347 Hunt, S., Cimino, J. J., & Koziol, D. E. (2013). A comparison of clinicians access to online knowledge resources using two types of information retrieval applications in an academic hospital setting. Journal of the Medical Library Association: JMLA, 101(1), 2631. doi:10.3163/1536-5050.101.1.005

Christina Magnifico LI804XS Shelstad, K. R., & Clevenger, F. W. (1996). Information retrieval patterns and needs among practicing general surgeons: a statewide experience. Bulletin of the Medical Library Association, 84(4), 490497. Westbrook, J. I., Coiera, E. W., & Gosling, A. S. (2005). Do online information retrieval systems help experienced clinicians answer clinical questions? Journal of the American Medical Informatics Association: JAMIA, 12(3), 315321. doi:10.1197/jamia.M1717

Performing Search Queries in Medical Information Retrieval Systems Alpi, K. M. (2005). Expert searching in public health. Journal of the Medical Library Association, 93(1), 97103. Lavin, M. A., Krieger, M. M., Meyer, G. A., Spasser, M. A., Cvitan, T., Reese, C. G., McNary, P. (2005). Development and evaluation of evidence-based nursing (EBN) filters and related databases. Journal of the Medical Library Association, 93(1), 104115. Mosa, A. S. M., & Yoo, I. (2013). A study on Pubmed search tag usage pattern: association rule mining of afFull-day Pubmed query log. BMC Medical Informatics and Decision Making, 13, 8. doi:10.1186/1472-6947-13-8 Sarin, P., & Meyer, M. (2005). Anesthesia Online Research & Training Aid (AORTA): An automated online querying tool for clinicians. AMIA Annual Symposium Proceedings, 2005, 1105. Sladek, R., Tieman, J., Fazekas, B. S., Abernethy, A. P., & Currow, D. C. (2006). Development of a subject search filter to find information relevant to palliative care in the general medical literature. Journal of the Medical Library Association, 94(4), 394401.

Christina Magnifico LI804XS Smith, C. A. (2011). Consumer language, patient language, and thesauri: a review of the literature. Journal of the Medical Library Association: JMLA, 99(2), 135144. doi:10.3163/1536-5050.99.2.005 Workman, T. E., & Stoddart, J. M. (2012). Rethinking information delivery: using a natural language processing application for point-of-care data discovery. Journal of the Medical Library Association: JMLA, 100(2), 113120. doi:10.3163/1536-5050.100.2.009