Evaluation Rubric Final Anne Coustalin

ETEC 565A LMS EVALUATION RUBRIC
ETEC 565A LMS Evaluation Rubric

Anne Coustalin, Yuandi Du, Brandon Nam, Susan Roth
Mary Stephenson
University of British Columbia
2
ETEC 565A LMS Evaluation Rubric
Precis
In 1974, the University of British Columbia (UBC) began its Rural Family Practice
Clerkship Program. It encouraged medical students to gain experience in traditionally
underserviced areas of BC, allowing them to participate in one-on-one training with a rural
physician (UBC, 2014). So successful was the program that in 1997 it became a curriculum
requirement, and in 2014 was fully incorporated into third-year medical school training.
This course apprentices students to a family physician, or team of preceptors, practicing
in a rural community. It is intended to give students an understanding of the total health
care of individuals and their community, including an understanding of the philosophy of
medical care that looks after patients from birth to death and from home to hospital and
back home (UBC, 2014).
Our task is to find an appropriate LMS to accommodate videos of specific clinical skills.
A student will be recorded performing a specific skillsuch as an injection, sutures, or biopsy
and that video will be viewed by two clinical faculty members and four peers. The reviewers
need to be able to provide feedback on the video and annotate their comments in some way. The
four peers will be in the same area as the video subject, as well one of the faculty members.
Other faculty will either be at UBC, or one of the cooperating universities, University of Victoria
or University of Northern BC; the learning shell needs to accommodate some distance. Thus, the
LMS needs to support high video functionality, while preserving privacy and minimizing
expense. It must support other multimedia artefacts, a clinical reasoning framework, and
educational discourse, as well as facilitate communication from different locations. We have

3
divided our rubric into two main sections: Essential Elements and Preferred Elements. Each
section is further broken down into categories, including functional requirements, technical
requirements, data management, and costs. Proposed learning management systems that do not
meet the minimum essential elements score are not considered further.
4
Rubric
Each criterion in the Essential section must achieve a minimum rating of 2 in order for the LMS to be considered further.
Where the rubric has multiple points under one criteria (e.g. Interaction) score 3 only if it addresses ALL points. Where the same
criteria are found under Acceptable and Recommended (e.g. Data Privacy) score 3.
FUNCTIONAL REQUIREMENTS- ESSENTIAL
Not Acceptable Acceptable Recommended Score

Criteria
(Rating of 1) (Rating of 2) (Rating of 3)
Theoretical basis underpins the design

elements. LMS provides access to
Theoretical basis underpins the design Theoretical basis underpins the design
organized content and multiple
elements. LMS provides access to basic elements. LMS provides access to
E- Pedagogical Design interactive tools to engage learners. New
organized material but includes few tools organized content and standard interactive
pedagogical tools are routinely added.
for interaction. tools to engage learners.
LMS provides a variety of modalities to
address different learning preferences.
Provides secure access to class

Provides secure access to class
Provides secure access to class participants email addresses (individually
participants email addresses.
participants email addresses. Provides or groups). Provides a variety of forms of
Provides only asynchronous
some forms of synchronous and synchronous and asynchronous
Communication Tools communication formats and has few
asynchronous communication (threaded communication (chat, instant messaging,
features to aid communication.
discussions, chat). Discussion threads load and threaded discussions).
Discussion threads load slowly and can
quickly and are easy to navigate. Discussion threads load quickly, are easy
be difficult to navigate.
to navigate, and subscribable.
Interface is functional but not intuitively Interface is obvious, self-explanatory, Interface is obvious, self-explanatory,
designed and aesthetically bland. aesthetically interesting. Content intuitive, aesthetically interesting and
Contains distracting elements, visually presented in a simple and straightforward inviting. Content presented in a simple
Design and Layout busy. Little use of design elements (text, manner. Good use of design elements and straightforward manner. Design
colour, font size) to create focal points. (text, colour, font size) to create focal elements (text, colour, font size) well
Media redundancy (visual + narration + points. Some evidence of media utilized to create focal points. No
text) evident. redundancy. redundancy of media.
5
FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Criteria
Students communicate freely with

instructor and peers via the
Students participate in online discussion,
Students can communicate freely with communication tools or forum. Ability to
and make comments to their peers
instructor and peers via the communication participate in online discussion and make
videos.
tools or forum. comments to peers videos.
Learning materials are available to
Learning materials cannot be associated Related textbooks or learning materials
students in LMS but cannot be connected
with LMS. are alongside of the video so that
to the video section.
students will have standards to refer to.
Faculty members communicate with
Faculty members are able to make
Interaction students freely via communication tools or Faculty members are able to make
comments to student videos.
forum, but is difficult to make interaction with comments to student videos and
learning materials. participate in their discussions.
Type of Interaction: teacher-student
interaction, student-student interaction,
Type of Interaction: teacher-student Type of Interaction: teacher-student
student-textbook interaction.
interaction and student-student interaction. interaction, student-student interaction,
student-textbook interaction.
Interaction characteristics: user-
Interaction characteristics: user-
generated interaction, designed
generated interaction. Interaction characteristics: inherent
interactivity.
interactivity, designed interactivity, user-
generated interaction.
Mobile Device Fully integrated with mobile devices; apps

No mobile version. Limited functioning on mobile devices.
Integration available for IOS, Android, and Windows.
Has native video upload/download

Upload of Video Relies on third-party video providers (e.g. Plug-ins available for video services. capabilities. Video available in multiple
YouTube). Video available in one size only. sizes to accommodate available
bandwidth.
6

Criteria
Offers only quantitative assessment Offers quantitative assessment Offers a diversity of qualitative and
opportunities. Basic database-generated opportunities as well as limited quantitative assessment opportunities.
quiz and assessment tool to evaluate qualitative assessment opportunities. Interactive assessment tools that allow
Testing and
students previous knowledge. Limited Multimedia enabled assessment tool students to reflect to previous knowledge.
Assessment Tools
range of questions types (multiple choice, (picture or attachments) to customize Separate tests to allow students to reflect
true/false, short answer and essay students reflection to previous to previous knowledge and enhance
questions). knowledge. skills. Has immediate feedback capability.
Testing and Reporting features limited to numerical Offers limited reporting features. Offers sophisticated reporting features,
Assessment - scores. Student progress is poorly Presentation of student progress is not presenting student progress in various,
Reporting Features communicated. customizable. customizable ways.
Instructor: LMS is difficult to use and Instructor LMS is moderately intuitive Instructor LMS is highly intuitive and
Ease of Use -Instructor requires considerable time investment to and requires a reasonable time requires limited time investment to
become proficient. investment to become proficient. become proficient.
Student: LMS is difficult to use and Student: LMS has an intuitive design
requires heavy time investment for Student: LMS has a functional design layout requiring little to no training in
Ease of Use - Student
proficiency. Appearance is bland and navigable with limited training. order to use. Appearance is inviting to
distracting. Appearance is appealing to user. user.
Video annotation tool is available and

Video annotation tool is not available and
Video annotation tool is available and supported by vendor. Viewers can add
cannot be integrated by third party.
Annotation Capabilities supported by third party. Viewers can video comment or text to video file.
Viewers can append/ comment at end of
add separate file to comment on video. Comment can be accessed in real time
video via webcam/camera.
on the video.
Video editing includes ability to add

Simple editing available. E.g. zoom,
Video Editing No video editing capabilities. slides, stills, and comments to initial
pan, transitions.
video.
7

Criteria
LMS supports a wide range of formats;

LMS supports a wide range of formats; graphics and pictures can be alt -tagged,
LMS has limited ability to provide a wide graphics and pictures can be alt -tagged, fonts can be enlarged, most text is
Accessibility for range of formats due to limited support fonts can be enlarged, most text is compatible with screen readers closed-
Diverse Learners for multiple file formats (text, audio, compatible with screen readers closed- captioning is available for video provided as
video, graphics, etc.). captioning is available for video provided learning objects. Closed-captioning can be
as learning objects. provided for annotated learner video
through voice-to text.
Available during business hours via email, Available with extended hours via email,
Available during business hours via email
telephone and live chat. Central IT support telephone and live chat. Both central IT
Technical Support with long response times. No central IT
is available but no department specific IT support and department specific IT support
support is available.
support is available. are available.
The LMS is compatible with Windows, Mac

The LMS is compatible with Windows, Mac
and Linux with IE, FF, Safari and Chrome
and Linux with IE, FF, Safari and Chrome
The LMS is compatible with Windows browsers supported for their current and
OSs and Browsers browsers supported for their current and
and Mac with IE, FF, Safari and Chrome previous versions
Compatibility previous versions.
browsers supported currently available.
Installation of plug-ins required for use with
No plug-ins required for video editing
video editing application.
application.
The LMS can create a Basic LTI link within

Basic Learning The LMS can access a video editing
The LMS can create an external link to a content page of the LMS so authenticated
Tools application within a content page of the
access a video editing application. users can access a full-fledged video
Interoperability (LTI) LMS.
editing application smoothly.
8
TECHNICAL REQUIREMENTS - ESSENTIAL

Criteria
Integration with student information

Data Management Integrates well with student information Integrates seamlessly with student
system (SIS) is problematic and doesnt
- SIS Integration system (SIS), though functionality is limited. information system (SIS).
allow for full functionality.
Allows for export of data (with personally Allows for export of data ((with personally
Data Management identifiable information removed) to external identifiable information removed) to external
Data is not exportable.
- Exportability data warehouse for use with external data warehouse for use with external
analytics programs. analytics programs.
Basic Learning The LMS can create a Basic LTI link within
The LMS can access a video editing
Tools The LMS can create an external link to a content page of the LMS so
application within a content page of the
Interoperability access a video editing application. authenticated users can access a full-
LMS.
(LTI) fledged video editing application smoothly.
Does not comply fully with BC Freedom Full compliance with BC Freedom of Full compliance with BC Freedom of
Privacy Law of Information and Protection of Privacy Information and Protection of Privacy Act Information and Protection of Privacy Act
Compliance Act (FIPPA) and/or UBC Information (FIPPA) and UBC Information Security (FIPPA) and UBC Information Security
Security Standards. Standards. Standards.
Limited or no data encryption

Data Encryption Allows for encryption of data. Allows for easy encryption of data.
capabilities.
Data De- De-identification of data is possible.

De-identification of data is not possible. De-identification of data is possible.
identification Supports multiple authentications of data.
9
DATA MANAGEMENT - ESSENTIAL
Not Acceptable Acceptable Score

Criteria Recommended (Rating of 3)
(Rating of 1) (Rating of 2)
Personally identifiable information (PII) and Personally identifiable information (PII) and
other sensitive information (personal health other sensitive information (personal health
Personally identifiable information
Data Security - information) is held on UBC servers behind information) is held on UBC servers behind
(PII) is housed in the cloud or on a
Storage UBC firewalls, in UBC data centres in UBC firewalls, in UBC data centres in
server outside of UBC.
compliance with UBC information security compliance with UBC information security
standards. standards.
No additional permissions required.

Data Security - Exhaustive additional permissions
Some additional permissions required.
Permissions needed. Includes tools for digital rights
management.
Automatic Backup No automatic backup system and/or Automatic backup system and disaster Automatic backup system and disaster
and Recovery disaster recovery plan recovery plan recovery plan
Compatible with database management

system. Large funding required but
Large funding required. Institution Cost is appropriate and institution is able to
institution is able to absorb within its
cannot absorb the cost. provide it within its capability.
capability. However, this means cut-offs of
some other projects.
Cost
(Score all three
parts) Students are required to pay Student LMS licenses are covered by There are no end user fees for the LMS and
additional fees to obtain user license. existing tuition. the school incurs no additional fees.
Ongoing maintenance costs covered with

Costly ongoing maintenance costs Affordable maintenance costs within
initial investment for a limited term.
beyond capability of existing budget. existing budget.
Essential Criteria Score Subtotal
LMS must score a minimum of two points on each assessment criteria to be considered further
10
FUNCTIONAL REQUIREMENTS - PREFERRED

Criteria
Access to material on the LMS restricted to

currently enrolled students and links to
Copyright No copyright management tools Access to material on the LMS restricted to
external content are provided. Authors are
Management included. currently enrolled students.
given a choice of Creative Commons for
their content.
Basic calendar available with pop-up Collaborative calendar with pop-up

Calendar No calendar
announcements. announcements.
No file exchange capabilities beyond Supports exchange of smaller files (<10MB)

Supports unlimited file exchange.
simple email transactions. between instructors and students.
File Exchange
(Score both
Supports upload and download of HTML Supports upload and download of HTML
categories)
Does not support upload or download pages, images, flash objects, pdf, and MS pages, images, flash objects, pdf, and MS
of zip files. word files in a single Zip archive. word files in a single Zip archive.
Clinical faculty members and Clinical faculty members and students can
Clinical faculty members and students can
Advanced Search students can search uploaded search by keywords and metadata from
search by keywords or metadata from
Option students videos by title or author clinical faculty members uploaded students
uploaded students videos.
name. videos.as well as by comments
LMS unable to integrate multimedia LMS is able to seamlessly integrate

Other LMS is able to integrate multimedia and
and learning object libraries from multimedia and learning object libraries
Multimedia learning object libraries from limited major
major textbook publishers. from major textbook publishers.
Objects textbook publishers.
11
FUNCTIONAL REQUIREMENTS PREFERRED contd

Criteria
LMS is able to host third party interactive LMS supports interactive tools that can
Interactive
LMS does not support interactive tools. tools. Faculty can create an interactive be created easily within the LMS and
Functionality
clinical reasoning framework. seamlessly disseminated to the students.
The LMS can transfer teaching

activities to its platform. Though some The LMS can transfer teaching activities
Migration of The LMS can transfer teaching activities
limits in its functionality require faculty online. Faculty members need to make
Existing seamlessly. Minimal changes required to
members to adjust their teaching large structural changes in their materials in
Courses migrate existing courses.
activities according to LMS and use order to make the transition.
additional tools outside of LMS.
Training modules are included with LMS Training materials are included with LMS
Training Limited training materials available and
and include printed materials and online and include print, online tutorials, and
Materials not provided with LMS.
tutorials. face-to-face training sessions.
The LMS can add metadata of student

Addition of
The LMS can add a metadata of The LMS can add a set of metadata of video by time frame multiple times by
Metadata or
student's video as a whole. student video by time frame. authorized user (based on roles and
Keywords
permissions).
Single sign on is available with full

Single sign on is available with limited Single sign on is available with most attributes passed via protocols such as
attributes passed via protocols such as attributes passed via protocols such as Central Authentication Service (CAS) or
Single Sign On Central Authentication Service (CAS) Central Authentication Service (CAS) or Security Assertion Markup Language
(SSO) or Security Assertion Markup Language Security Assertion Markup Language (SAML). i.e. Student number, name,
(SAML). i.e. Student number, name (SAML). i.e. Student number, name, email, email, academic year, enrolled courses
and email. academic year and enrolled courses. and enrolled group associated with their
clerkship rotations.
12
TECHNICAL REQUIREMENTS PREFERRED contd

Criteria
Institution level of learning object Department node of learning object

Course areas learning object
Learning Object repository is available to authorized user repository is available to authorized user to
repository of uploaded videos is only
Repository (LOR) to access global repository of uploaded access uploaded videos by their node
available.
videos. access rules and permissions.
The LMS has a cron task to automatically

Group Management The LMS has an advanced group
The LMS has a group builder tool that add students to a group and based on their
Tool builder tool that allows clinical faculty
allows clinical faculty members to add assigned group every 5 weeks.
** Year 3 student members to add students based on their
students based on their assigned LMS can automatically remove students
clerkships are rotated assigned group every 5 weeks.
group. from a previous group and assign them to a
every 5 weeks
new group after 5 weeks.
The LMS manages small to medium The LMS manages medium to large The LMS manages large throughput of
System Performance
throughput of users not affecting client throughput of users not affecting client users not affecting client to server
to server responses. to server responses. responses.
Students can track:

Students can track: Students can track: Completed assignments, progress and
Learning Analytics
completed assignments on a course- completed assignments as well as participation through course, grades across
&Tracking
Students by-course basis. LMS has no grades across courses. LMS has limited courses, relative success, LMS predicts
predictive features. predictive features of uncertain value. success in the course based on predictive
factors.
Instructors can track student engagement,

Instructors can track student learning outcomes, amount of
Instructors can track student
engagement, learning outcomes, student/student interaction, attention to
engagement, amount of
Learning Analytics amount of student/student interaction, feedback, time on task, participation,
student/student interaction,
& Tracking: time on task, participation, patterns of use, demographics, content
demographics, success on
Instructors demographics, content mastery/success mastery/success on assessments, course
assessments
on assessments, risk factors. quality (in relation to metrics), risk factors.
Includes ability to create specialized
queries.
13
DATA MANAGEMENT- PREFERRED

Criteria
Institution can track: Institutions can track:

Learning Institutions can track: overall UBC Faculty of Medicine student
overall UBC Faculty of Medicine
Analytics & overall UBC Faculty of Medicine student retention, retention, students at risk of failure,
student retention, students at risk of
Tracking: students at risk of failure, student grades for each course quality (metrics), effects of training
failure, student grades for each
Institution course, course quality (metrics). and pro-d on overall course quality.
course.
Institutions can track:

Learning Institution can track: Institutions can track: overall UBC Faculty of Medicine student
Analytics & overall UBC Faculty of Medicine overall UBC Faculty of Medicine student retention, retention, students at risk of failure,
Tracking: student retention, students at risk of students at risk of failure, student grades for each course quality (metrics), effects of training
Institution failure, student grades for each course, course quality (metrics). and pro-d on overall course quality.
course.
Minimum standards met? (Did every criterion in the Essential section score two or higher?)
Yes No
If YES, proceed; If NO, LMS should be rejected
Preferred Subtotal
Criteria Score /54
Essential Insert Score

/84
Criteria Score
FINAL LMS
TOTAL /138
SCORE
Evaluation Analysis
Excellent- recommend: 120-138 points

Good - possibly recommended: 100-119 points
Not recommended: below 100 points
14
Rationale
Learning management systems (LMS) are an indispensable technology for today's higher
education institutions and each institution has unique requirements to consider when selecting
one. Essential in the LMS is the functionality to support the teaching and learning needs of the
institutions students and faculty (Cavanaugh, 2014; Abdullateef, Elias, Mohamed, & Zaidan,
2016), as well as ensuring the institution is compliant with data privacy and security legislation.
Incorporated into the above rubric are Bates (2014) SECTIONS framework and
Chickering and Ehrmanns (1996) Seven Principles of Technology as Lever. The following
discussion outlines our rationale for the rubrics evaluation criteria.
E-pedagogy is learning and teaching using digital information and communication
technologies and requires a sound pedagogical foundation where teachers can select content and
strategies to meet specific learning objectives. An e-learning pedagogical foundation is critical to
an effective LMS and identified as essential criteria in our rubric.
In conjunction with a sound pedagogical foundation is a design and layout that facilitates
learning and teaching. Mayer (in Bates 2014) identified twelve principles of multimedia design.
The essence of these principles is that well-designed e-learning content is of little value if it
cannot be communicated effectively to the learner. Therefore, course interfaces should be
obvious, intuitive and self-explanatory so that the users time is spent thinking about the learning
and not about the interface (Gutierrez, 2014). According to Bates (2015), the affordance of
interaction is critically important when selecting a LMS, as students learn best when they are
active in their learning. For that reason, the LMS must be evaluated in terms of the
opportunities it affords for interaction between peers, instructors, and learning materials. In
15
addition, good migration of materials and assessment tools must be in place for students to
further interact with learning materials and reflect on their learning experience.
An effective LMS should not require a heavy time investment in order for a user to
become proficient, thus ease of use is an essential criterion. Content is of little value if the user is
unable to access it or finds the system cumbersome to use. This includes access for users who
require special accommodations, typically due to physical or learning disabilities. Both Bates
(2015) and Brokup (2008) state that accessibility and access for diverse learners are important
factors to consider in ensuring the LMS adequately addresses different types of learners.
Communication tools that facilitate learner-learner and learner-teacher communication
are vital in a digital platform. Chickering and Ehrmann (1996) assert that communication
technologies increase access to faculty, afford resource sharing, and facilitate shared
learning. For that reason, email, computer conferencing, and instant chat, all of which serve to
increase total communication and the opportunities to share work and problem solve, were
deemed essential criteria.
Mobile devices are included under functional requirements as students increasing use of
mobile devices for educational and mobile access to an LMS is particularly valuable for students
who are relatively older and off-campus (Joo, Kim, & Kim, 2016). In addition, a survey by
Boruff and Storie (2014) found over 50% of the 3rd- and 4th-year students and residents
reported using their mobile devices to find drug information, find clinical practice guidelines,
read point-of-care information, do clinical calculations, or perform differential diagnoses
(Results, para 1), and 77% of students used them daily during their clinical clerkships. Mobile
access (always-anytime-anywhere connection) to the LMS is valuable for students who are
already accustomed to using their phones as medical tools.

16
Learning analytics are becoming of increasing importance in online teaching and learning
In creating this rubric we have afforded high value to tracking capabilities and learning analytics
that provide students, faculty and the institution with the information they need to track and
analyze student progress, optimize teaching and learning, and promote student success. We
developed our analytics criteria based on the work of Johnson et al. (2016), Pappas (2014, 2016),
and Sclater (2014).
For the LMS technical requirements, technical supports, integration with SIS, OSs and
browser compatibility are essential criteria. Briefly, seamless integration of data among all
systems allows for efficient and optimal use of LMS. Other technical elements which contribute
to creating a seamless and enjoyable user experience were considered preferred, but non-
essential.
Data security was carefully considered in our rubric. Public post-secondary institutions
such as UBC fall under the BC Freedom of Information and Protection of Privacy Act (FIPPA).
Of particular relevance to our selection of an LMS is Part 3 Protection of Privacy (Assembly,
B. C. L. 1993). We determined that the Faculty of Medicine should under no circumstances
adopt a LMS that did not facilitate compliance with FIPPA regulations and that it must comply
with UBC Information Security Standards (2014). Students in the program record both actors
and real patients, so security and privacy are of paramount importance. The LMS must be able to
manage attempts from hackers to bypass authentication method and the external video annotation
editing tool.
Finally, cost is a major consideration in choosing an LMS. Initial purchase, maintenance,
and support costs all require careful consideration. Hidden costs and high maintenance fees can
significantly affect an institutions budget. Choosing wisely can prevent a costly mistake.
17
Conclusion
Choosing an LMS can be a daunting task. The wide range of available LMS platforms
have different strengths and weaknesses, standard performance levels, infrastructure, and
licensing options. Choosing the wrong platform can be costly in terms of budget and time
commitment. Being clear about an LMS desired features, functionality, and cost can help guide
institutions in the selection process and prevent becoming distracted by an LMS bells and
whistles. It is crucial that a selection strategy or rubric be used to evaluate various systems being
considered. We feel we have created a comprehensive LMS evaluation rubric that is especially
suitable to UBCs Rural Family Practice Clerkship Program but could also be adapted by many
other institutions undertaking this task.

18
References
Abdullateef, B. N., Elias, N. F., Mohamed, H., & Zaidan, A. A. (2016). An evaluation and
selection problems of OSS-LMS packages. SpringerPlus, 5. 248. doi:10.1186/s40064-
016-1828-y
Assembly, B. C. L. (1993). Freedom of Information and Protection of Privacy Act. Retrieved
from: http://www.bclaws.ca/Recon/document/ID/freeside/96165_00
Assembly, B. C. L. (2008). E-Health (Personal Health Information Access and Protection of
Privacy) Act. Retrieved from
http://www.bclaws.ca/Recon/document/ID/freeside/00_08038_01
Baer, L., & Campbell, J. (2012). From metrics to analytics, reporting to action: Analytics' role in
changing the learning environment. Game Changers: Education and Information
Technologies, Educause, 53-65.
Bates, A.W. (2015) Teaching in a digital age: Guidelines for designing teaching and learning.
Vancouver BC: Tony Bates Associates Ltd.
Bates, T. (2014). Choosing and using media in education: The SECTIONS model. In Teaching in
digital age. Retrieved from https://opentextbc.ca/teachinginadigitalage/part/9-
pedagogical-differences-between-media/
Brokop, F. (2008) Accessibility to e-learning for persons with disabilities: Strategies, guidelines,
andsStandards. Edmonton, AB: NorQuest College/eCampus Alberta.
Boruff, J.T., & Storie, D. (2014). Mobile devices in medicine: A survey of how medical
students, residents, and faculty use smartphones and other mobile devices to find
information. Journal of the Medical Library Association, 102(1), 22-30.
doi: 10.3163/1536-5050.102.1.006
19
Cavanaugh, T. (2014). The lms selection process: Practices and considerations. Educause Center
for Analysis and Research. Available at
https://library.educause.edu/resources/2014/7/the-lms-selection-process-practices-and-
considerations
Chickering, A. W., & Ehrmann, S., C. (1996). Implementing the seven principles: Technology as
lever. American Association for Higher Education Bulletin, 49(2), 3-6. Retrieved from
http://www.aahea.org/articles/sevenprinciples.ht
CIHR. (2005). Best practices for protecting privacy in health research (GC Publication No.
MR221-63/2005E-PDF). Ottawa, ON: Public Works and Government Services Canada.
Retrieved September 27, 2016 from: http://www.cihr-
irsc.gc.ca/e/documents/et_pbp_nov05_sept2005_e.pdf
ECAR Working Group Paper. (2015). The predictive learning analytics revolution: Leveraging
learning data for student success. Retrieved September 23 from:
http://net.educause.edu/ir/library/pdf/ewg1510.pdf .
Gutierrez, K. (2014). If you confuse learners, you lose them: 4 steps to effective communication
in eLearning. Available at
http://info.shiftelearning.com/blog/if-you-confuse-learners-you-lose-them-4-steps-to-
effective-communication-in-elearning
Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016).
NMC Horizon Report: 2016 Higher Education Edition. Austin, Texas: The New Media
Consortium. Retrieved from
https://library.educause.edu/~/media/files/library/2016/2/hr2016.pdf
20
Joo, Y.J., Kim, N. & Kim, N.H. (2016). Factors predicting online university students use of a
mobile learning management system (m-LMS). Education Technology Research
Development, 64, 611-630.
Pappas, C. (2014). How learning analytics can improve elearning design and development.
eLearning Industry. Retrieved from https://elearningindustry.com/5-reasons-why-
learning-analytics-are-important-for-elearning
Pappas, C. (2016). 8 types of elearning analytics to obtain from google and your lms. eLearning
Industry. Retrieved from https://elearningindustry.com/types-elearning-analytics-obtain-
google-lms
Sclater, N. (2014). Effective learning analytics: Using data and analytics to support students.
Joint Information Systems Committee. Retrieved from:
https://analytics.jiscinvolve.org/wp/2014/10/08/analytics-systems-centred-around-the-
vlelms-2/
University of British Columbia. (n.d.). Collaborative learning annotation system. Retrieved from
http://ets.educ.ubc.ca/clas/
University of British Columbia. (2014). Year 3 rural family practice clerkship: Preceptors
guide. Retrieved from http://undergrad.familymed.ubc.ca/files/2014/08/2014-
Preceptor_Guide.pdf
University of British Columbia. (2014). Information security standards document. Retrieved
from: https://cio.ubc.ca/sites/cio.ubc.ca/files/documents/standards/IS%20Standards%201-
21.pdf
Warwick Learning and Development Centre. (2007). Pedagogies for e-learning. Available
at http://www2.warwick.ac.uk/services/ldc/resource/eguides/pedagogies/
21

Evaluation Rubric Final Anne Coustalin

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Evaluation Rubric Final Anne Coustalin

Uploaded by

Copyright:

Available Formats

ETEC 565A LMS EVALUATION RUBRIC

ETEC 565A LMS Evaluation Rubric

ETEC 565A LMS Evaluation Rubric

Clerkship Program. It encouraged medical students to gain experience in traditionally

This course apprentices students to a family physician, or team of preceptors, practicing

in a rural community. It is intended to give students an understanding of the total health

care of individuals and their community, including an understanding of the philosophy of

back home (UBC, 2014).

A student will be recorded performing a specific skillsuch as an injection, sutures, or biopsy

educational discourse, as well as facilitate communication from different locations. We have

FUNCTIONAL REQUIREMENTS- ESSENTIAL

Not Acceptable Acceptable Recommended Score

Theoretical basis underpins the design

Provides secure access to class

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score

Students communicate freely with

Mobile Device Fully integrated with mobile devices; apps

Has native video upload/download

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score

Video annotation tool is available and

Video editing includes ability to add

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score

LMS supports a wide range of formats;

The LMS is compatible with Windows, Mac

The LMS can create a Basic LTI link within

TECHNICAL REQUIREMENTS - ESSENTIAL

Not Acceptable Acceptable Recommended Score

Integration with student information

Limited or no data encryption

Data De- De-identification of data is possible.

DATA MANAGEMENT - ESSENTIAL

Not Acceptable Acceptable Score

No additional permissions required.

Compatible with database management

Ongoing maintenance costs covered with

Essential Criteria Score Subtotal

FUNCTIONAL REQUIREMENTS - PREFERRED

Not Acceptable Acceptable Recommended Score

Access to material on the LMS restricted to

Basic calendar available with pop-up Collaborative calendar with pop-up

No file exchange capabilities beyond Supports exchange of smaller files (<10MB)

LMS unable to integrate multimedia LMS is able to seamlessly integrate

FUNCTIONAL REQUIREMENTS PREFERRED contd

Not Acceptable Acceptable Recommended Score

The LMS can transfer teaching

The LMS can add metadata of student

Single sign on is available with full

TECHNICAL REQUIREMENTS PREFERRED contd

Not Acceptable Acceptable Recommended Score

Institution level of learning object Department node of learning object

The LMS has a cron task to automatically

Students can track:

Instructors can track student engagement,

DATA MANAGEMENT- PREFERRED

Not Acceptable Acceptable Recommended Score

Institution can track: Institutions can track:

Institutions can track:

Essential Insert Score

Excellent- recommend: 120-138 points

discussion outlines our rationale for the rubrics evaluation criteria.