Description of Proposed Research Project for Education Fellows Consideration

Project Mentors Name

& Complete Contact Info

Beth Bierer, PhD, Director of Evaluation

Cleveland Clinic Lerner College of Medicine, NA25
Phone: 216-444-3283
E-mail: biererb@ccf.org

Specifics of Proposed Research Project

Problem Statement/Issue:

Self-regulated learning (SRL) refers to the extent to which individuals

can monitor and control their thoughts, beliefs, and actions during
learning. Students engaged in SRL typically follow a three-phase,
recursive process that entails: (1) planning analyze tasks and set
goals; (2) performance regulate behavior and emotions; and (3)
evaluation self-reflect using feedback. 1 The education literature
suggests that formative feedback plays a critical role to support the SRL
cycle.2 While this feedback may be internal (e.g., learners thoughts
about progress in relation to desired goals) or external (feedback from
outside sources such as teachers, peers, etc.), the impact of feedback
on learning is expected to improve if students can reconcile external
feedback with internal states.
Of course, educators cannot assume students can simply decipher and
convert external feedback messages of right and wrong into improved
performance without guidance.1 Empirical research clearly demonstrates
that motivation and beliefs (self-efficacy and locus of control) interact
with a students ability to decode and internalize feedback messages to
regulate performance.2-3 Specific strategies (e.g., dialogue with
teacher/mentor) have also been shown to improve internalization of
formative feedback to promote self-regulation. 4 Investigating strategies
which support SRL should contribute to the use of formative assessment
in medicine as more emphasis is placed on self-regulation of
performance and maintenance of certification. 5
The proposed project is important because it explores learners
perceptions of formative assessment feedback within a competencybased educational model which, by definition, necessitates a robust
and multi-faceted assessment system (p. 676).5 Additionally, the
proposed project should illuminate how interconnected factors (e.g.,
motivation, self-efficacy, learning environment, dialogue with teachers,
etc.) shape a medical students interest in and ability to self-regulate
performance, which in turn should inform planning for faculty
development and improve use of formative feedback throughout the
medical education continuum (UME, GME, CME). 2-3,5
We developed a research agenda to gain insights about the quality and
quantity of formative assessment feedback that students cite in their
portfolios.6-8 We also learned, in exit interviews with CCLCM graduates,
that beginning medical students, especially those used to receiving
grades for academic work, struggle with how to use formative feedback
to set learning goals. Some students want extrinsic rewards (grades or
praise) for motivational purposes while others cannot independently
decode formative feedback messages. 9 This revelation from our
graduates led us to wonder how first-year medical students interpret
and internalize formative assessment feedback to improve task-based
performance during medical school. Therefore, this project involves

using a case study design10 to investigate the following research

questions:
Hypotheses/Questions:
(1) How do medical students interpret formative assessment feedback
from different
sources and contexts to self-regulate learning and
judge performance within a
competency-based medical
education program?
self-

(2) What supports medical students development of and engagement in

regulated learning during medical school?

Specific Project Aims:

We hypothesize medical students experience a developmental process
as they become
more adept with interpreting and acting upon
formative feedback in relation to
performance standards
and believe the proposed project will result in a framework to
clarify
this process.
Current Project Status:
(Check all that apply)

X
X
X

New idea
Builds on mentors previous work
Literature review in process
Literature review complete
Research proposal/grant written

Anticipated Project Duration (1-24 months):

IRB approved project

Data already collected
Other:

16 Months (interspersed throughout fellowship)

Fellows Requisite Abilities and Involvement

Needed Competencies:

Technical Skills for Project

X

Quantitative software (e.g., SPSS, SAS, AMOS, etc.)

Qualitative software (e.g., NUDIST, etc.)
Bibliographic software (e.g., RefWorks, EndNote, etc.)
Design/manipulate relational databases
Design questionnaires/inventories/assessment tools
Perform specific analyses _analyze/code interview data
Other ______________________________________

Prerequisite Knowledge Base for Project

Familiarity with literature on formative assessment and self-regulation
Expectations of Fellow:
implementing the

Participate as a member of the research team and assist with

following research protocol:

Assist with development of case study protocol and data collection

methods informed by the frameworks of self-regulated learning, selfdetermination theory, formative assessment, and competency-based
assessment.

Using school records, help identify and recruit six first-year medical
students with varied backgrounds (work experience and college
major) and attributes (gender, race/ethnicity, US citizenship) to
participate in proposed project.

Assist with conducting four digitally recorded, open-ended interviews

with each participating first-year student. These interviews will occur
systematically throughout Year 1 of CCLCMs educational program as

students construct formative portfolio #1 (October), formative

portfolio #2 (January), formative portfolio #3 (March), and
summative portfolio #1 (May).

Anticipated Scholarly
Products for Fellow (list
journals/organizations)

With the principal investigator, observe the 3-4 scheduled meetings

each student has with his/her portfolio advisor and generate detailed
field notes immediately after each meeting.

Meet regularly with the research team to review field notes,

transcripts, and other data and document decisions and emerging
questions/themes.

Use constant-comparative method with support of qualitative

software package to code verified transcripts collected longitudinally
throughout Year 1 of medical school.

List findings from meeting observations and discuss with students

and advisors separately to ensure accuracy.

Repeat protocol with another cohort of first-year medical students

with understanding that protocol may evolve given the flexibility of
the case study as research approach.

Present project as a grand rounds to Education Institute faculty/staff.

Assist with manuscript preparation.

manuscript submission to Academic Medicine

Key References
1. Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and
self-regulated learning: A model and seven principles of good
feedback practice. Studies in Higher Education, 31(2):199-218.
2. Clark, I. (2012). Formative assessment: Assessment is for selfregulated learning. Educational Psychological Review, 24(2):205249.
3. Sargeant, J., Armson, H., Chesluk, B., Dornan, T., Eva, K., Holmboe, E.,
Lockyer, J., Loney, E., Mann, K., & van der Vleuten, C. (2010). The
process and dimensions of informed self-assessment: A conceptual
model. Academic Medicine, 85(7):1212-1220.
4. Durning, S. J., Cleary, T. J., Sanders, J., Hemmer, P., Kokotailo, P., &
Artino, A. A. (2011). Perspective - viewing strugglers through a
different lens: How self-regulated learning perspective can help
medical educators with assessment and remediation. Academic
Medicine, 86(4):488-495.
5. Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R.
(2010). The role of assessment in competency-based medical
education. Medical Teacher, 32(8):676-682.
6. Dannefer, E. F., & Henson, L. C. (2007). The portfolio approach to
competency-based assessment at the Cleveland Clinic Lerner College
of Medicine. Academic Medicine, 82(5):493-502.

7. Dannefer, E. F., Bierer S. B., & Gladding S. (2012). Evidence within a

portfolio-based assessment program: What do medical students
select to document their performance? Medical Teacher, 34(3):215220.
8. Bierer, S. B., & Dannefer, E. F. (2011). Does students gender,
citizenship, or verbal ability affect fairness of portfolio-based
promotion decisions? Results from one medical school. Academic
Medicine, 86(6):773-777.
9. Altahawi, F., Sisk, B., Poloskey, S., Hicks, C., & Dannefer, E. F. (2012).
Student perspectives on assessment: Experience in a competencybased portfolio system. Medical Teacher, 34(3):221-225.
10.Butler, D. L. (2011). Investigating self-regulated learning using indepth case studies. In B. J. Zimmerman & D. H. Schunk (Eds.),
Handbook of self-regulation of learning and performance (pp. 346360). New York, NY: Taylor & Frances.