A Neural Network Visualization of

Reinforcement Learning Approach for Adaptive

e-learning using Multiple Learner
Characteristics
MSC. Computer Science Project Proposal
By
Dan Oyuga Anne
2019
Introduction
Assessing learning progress is meant to measure and analyze learners’
data to infer competence, meta -competence and confidence(Hlioui &
Alioui, 2016). In also involves measuring effectiveness and efficiency of
learning environment. An overriding purpose for measurement is to
collate information for purposes of making decisions which may be
global; how well the student does compare with other class members,
or local; materials a student has read or the effect of environmental
factors affecting learner characteristics e.g. emotions, cognitive
abilities etc. and their influence on the learners’
performance(Sandanayake, Madurapperuma, & Dias, 2011).
Assessment can also be viewed as a process of identifying changes that
are needed in behaviours or environments and to decide how to
accomplish the goals of the needed change.
Introduction
The presentation of the measurement for consumption is a major factor if
meaningful deductions are to be made from the data collected. There are
many techniques of Data presentations, amongst them is data visualization.
Data visualization is the process of helping people understand patterns,
trends, and insights by transforming data into a visual context(Gupta,
Sampat, Sharma, & Rajamanickam, 2016). This can take different forms
including pivot tables, pie charts, line graphs, column charts etc. In an e-
learning environment, learner characteristics are considered integral factors
that can affect learning outcomes and progress(Rajendran, 2014). However
little success has been achieved in incorporating multiple learner
characteristics into this context (Calvo and D’Mello, 2012). The limited-to-
null adoption of some characteristics such as emotional analysis tools and
affective feedback prevents both learners and teachers from reaping the
benefits of emotion-aware learning.(Feidakis, 2016). In this study we want to
we to model multiple learner characteristics, apply reinforcement to model
adaptive learning, variate the modeled states against performance and
explore the use of Bevier curves in Visualizing this data
Problem Statement
In e- learning environments, the need to the measure learners progress
when putting inconsideration, the learners learning characteristics,
offer learning support and to visualize the data has not been covered in
most researches(Feidakis, 2016). Visualizations of these data need to
be done using the right tools that can give meaningful results. Various
tools have been explored for this purpose(Khan, Graf, Weippl, & Tjoa,
2010). The objective of this work is to investigate The Use of
Reinforcement Learning in modelling multiple learner characteristic for
adaptive learning in an e learning environment.
Objectives of the research
• To Investigate application of reinforcement learning in modeling
learner characteristics
• To Investigate Bevier Curve for use in visualizing e-learner’s multiple
characteristics
• Design and Develop a neural network model to identify and visualize
multiple e-learners learning characteristics state with respect to their
learning performances
• Test the model with existing data
Research Questions
• How does reinforcement learning compare with other methods in
artificial intelligence modeling of learner characteristics?
• Can multiple learner characteristics be extracted during learner’s
active interaction with e-learning environment?
• Can visualization of learner’s performance data be done concurrently
with the learner’s characteristics at time of interaction with e-learning
environment?
• How does training neural network with existing data compare with
real time Data
Research Scope
This project will cover extraction of three learner characteristics; Prior
Knowledge, affective state and learning style using neural networks.
The learner affective states will be extracted from texts using artificial
neural networks. Modelling of the neural networks will be reference
from Microsoft cognitive toolkit which is a grouping of common models
of artificial neural networks. Finally, we will visualize the performance
data together with the varied modeled learner characteristics using
Bezier curves which will also be implemented using CNTK. The data to
be used are research data from ISEAR(Intercultural Study on Emotional
Antecedents and Reactions) for training the Models related to the
affective states and testing their performance and dummy performance
data for Visualization
Literature Review
• Learning theories and adaptive learning
• Learner characteristics and adaptive learning systems
• E-learning models
• Aspects of learner Models
• AI techniques used in adaptive e- learning systems
• Reinforcement in learner models
• Data Visualization and Bezier curves
• Use of CNNTK
Literature Review
Summary of learning theories used in adaptive e-learning systems

Authors Learner characteristics Theory

(Almohammadi & Hagras, 2013b) Learner knowledge Cognitivism

(Deeb, Hassan, & Beseiso, 2014) Learning style Cognitivism

(Fenza, Orciuoli, & Sampson, 2017) Learner knowledge Cognitivism

(Kolekar, Sanjeevi, & Bormane, 2010) Learning style Cognitivism

(Rajendran et al., 2018) Affective states(frustration) Cognitivism

(Malpani, 2011) Learner knowledge(measured by the ability to answer Cognitivism

questions)
(Sabourin, Mott, & J., 2011) Learner effect behaviorism

(C. H. Wu, Chen, Yan, & Lee, 2017) Learner knowledge Cognitivism

(Alshammari, Anane, & Hendley, 2015) Learning style Cognivitism

(Whitehill & Movellan, 2018) Learner knowledge Cognitivism
(Hwang, Sung, Hung, & Huang, 2013) Learning style Cognitivism

(Yang et al., 2016) Learning style and cognitive styles Cognitivism

(S. Y. Chen, Huang, Shih, & Chang, 2016) Cognitive style, gender differences Cognitivism

(C. M. Chen & Li, 2010) Learner context Constructivism

Literature Review
Summary of learner characteristics used in different adaptive systems studies
Authors Title/summary Learner characteristics
(D. Wu et al., 2015) A fuzzy tree matching-based personalized Preferences
E-learning recommender system

The authors propose the use of fuzzy tree structure model to

describe learning activities and learner profiles in order to be able to
recommend learning activities to informal e-learning learners. They
carried out an experiment to test the proposed prototype using
movie data set and found it more reliable than the other
recommender systems that utilized the same data set.
(Tadlaoui et al., Learner knowledge
2018) A learner model based on multi-entity Bayesian networks and
artificial intelligence in adaptive hypermedia educational systems

The authors develop a learner model based on multi entity Bayesian

network and artificial intelligence
(Alshammari, Anane, Adaptivity in E-Learning Systems Learning style, learner knowledge and learner preferences were
& Hendley, 2014) found to be the most used learner characteristics in the learner
The authors carried a comparison of adaptive e-learning system model
based on the learner model, domain model and adaptation model.
(Kanimozhi, n.d.) An Adaptive E-Learning Environment Centred On Emotional behavior
Learner’s Emotional Behavior.

The authors proposed an adaptive system that utilizes the EEG bio
signal approach to detect the learners’ mode and offer adaptive
content depending on the mode.
Literature Review
Summary of existing learner models
NO Authors
1 (Rabat, 2016)
Summary of the study Characteristics in the model Theoretical underpinning
Towards an Adult Learner Personal, cognitive, social, personality Andragogy and self-directed
Model in an Online traits, emotional traits, cognitive and learning
Learning Environment knowledge

Developed a model that takes

2 (Mejia, Gomez, Mancera,
& Taveneau, 2017)
into account the unique adult
learner characteristics
Inclusive Learner Model for
Adaptive Recommendations in
Virtual Education
Demographics, competences, reading No learning theory
difficulties, learning style, cognitive
traits

Develops a models that takes

3 (Huang, Yin, & Liu, 2017)
into account learners with
disabilities
Research on Individualized Basic information, cognitive levels, No learning theory
Learner Model Based on learning style, interest preference
Context-awareness

Developed a learner model

based on various context
including time, device, task,
location, and social
 Conceptual Framework
ISEAR Data
Anger
Disgust
Fear
Affective Predicted Affective
Sadness
States state
Shame
Joy
Guilt

Identify
Relevant Infering
Learning
Learning Management Behavior for learning style Predicted Learning
Style Reinforcement
System (LMS) each learning from behavior Style
Database style

Data Driven
approach

Selecting the relevant features and patterns of behavior

Classifying the occurance of the behavior

Define pattens for each dimension of the learning style

Knowing
Prior Understanding Predicted
Knowlwdge Applying Knowledge State

Bezier Curve
for Perfomance
Visualization
Methodology
Iterative Incremental Software Development Methodology

Research

Iteration 1
Software
Requirements
Design Development Testing Product Feedback
Analysis
Increment 1

Iteration 2

Software
Requirement
Design Development Testing Product Feedback
Analysis
Increment 1

Iteration n

Requirement
Design Development Testing Implementation Maintenance
Analysis
 Methodology
3.1 The requirements analysis
Identification of Input fields input fields.

We will employ both linear regression, genetic algorithm processing and

non-parametric regression to identify the input fields
3.2 Data Preparation.
Data will have to be numeric and so we will convert non numeric data into
numeric quantities. Neural networks require numeric quantities as inputs.
We will normalize the input data by use of mean/Standard scaling to scale
all the variables between 0 and 1.
Methodology
3.3 Design phase
In This step in selecting the NEURAL NETWORKS paradigm is
determining an appropriate number of hidden layers and hidden
nodes for the network structure. To determine the optimal
number of hidden nodes and employ multiple hidden layers we
will use the ratio of 2:3
3.4 Development Phase
We will use Microsoft studio dot net development framework
for developing this application. We will use the Microsoft
Cognitive tool kit to hasten the development of neural networks
 Methodology
3.5 Testing
Our neural networks performance in terms of making predictions, classifications and clustering will be
tested. We are going to do this using the validation test data and training data
Each of our Neural Networks will be tested and validated according to most correct classifications and
closest predictions. We will do sensitivity analysis to help also in demystifying the functionality of the
Neural Networks. Depending upon their importance, some types of outcomes might be weighted
more heavily. In addition to testing for the successful outcome of a Neural Networks, the developer may
also want to test the importance of the inputs.
3.6 Deployment
For Validation and testing we proposed to use averaging of the outputs of networks we developed to
increase the correlation with the training data. We will use the same to select the best Neural Networks
to deploy. We will also deploy the use of rule synthesis, combining the outputs of the Neural Networks
with Boolean operators so as to increase the expected return in learner characteristic identification and
visualization system