I S S U E S A N D IN N O V A T I O N S I N N U R S I N G E D U C A T I O N

Problem-based learning in a new Canadian curriculum

Erlinda T. Morales-Mann RN MA
Assistant Professor, University of Ottawa School of Nursing, Ontario, Canada

and Christabel A. Kaitell RN MPH

Assistant Professor, University of Ottawa School of Nursing, Ontario, Canada

Submitted for publication 27 July 1999 Accepted for publication 11 August 2000

Correspondence: Erlinda Morales-Mann, Assistant Professor, University of Ottawa School of Nursing, 451 Smyth Road, Ottawa, Ontario, K1H 8M5, Canada. E-mail: emorales@uottawa.ca

M O R A L E S -M A N N E . T . & K A I T E L L C . A . ( 2 0 0 1 )

Journal of Advanced Nursing 33(1), 1319 Problem-based learning in a new Canadian curriculum Problem-based learning (PBL) is a method of group learning that uses true-to-life problems as a stimulus for students to learn problem-solving skills and acquire knowledge about the basic and clinical sciences. This article documents the design and implementation of PBL in a second year course in the new curriculum of the University of Ottawa School of Nursing's Generic Program. The learning and teaching experiences of students and facilitators in this PBL course are described. As a way to determine students' perception of their learning using PBL, they were asked to respond to four questions. The most frequently described thinking processes were problem solving, nursing process and group process. When asked to describe the learning they derived from PBL, as differentiated from other instructional methods, students identied group process and problem solving most often. The most frequently identied factors that inuenced performance and learning in PBL were positive attitude and group effort. The factors that affected the facilitators' performance of their role were large group size, insufcient practice of facilitator skills and PBL preparation. To enhance group process, facilitators modelled and shared roles. They fostered student motivation and development through formative evaluation. PBL produced clear benets for students, such as increased autonomous learning, critical thinking, problem solving and communication. For facilitators, PBL was a liberation from the traditional role of `content expert and super consultant'. Keywords: problem-based learning, group process, critical thinking, self-directed learning, problem solving, group dynamics, group communication, group learning, group decision making, nursing education.

Introduction
Health care services require nurses who are critical thinkers, with superior speaking, writing and learning skills, who can solve problems for which there are often no standard solutions and no one correct answer. How are these prociencies achieved? Scientists recommend that a strong theoretical base and domain-specic knowledge are needed to meet the demands of society (Glaser 1988). In addition, cognitive
2001 Blackwell Science Ltd

researchers state that an educator should be as concerned with `how' one teaches as with `what' one teaches (Bruer 1995). Therefore, a change in the way teachers teach, grounded in an understanding of how students learn, merits consideration. In response to these recommendations as well as to the challenges of increasingly complex patterns of health care in Canada and the shift from hospital to home care, the new curriculum of the University of Ottawa School of Nursing
13

E.T. Morales-Mann and C.A. Kaitell

Generic Program was developed. One of the innovations was the inclusion of a two credit seminar in each of the 4 years of the programme. This course was planned to provide an opportunity to integrate knowledge, attitude, and skill in the analysis of practice problems. Problem-based learning (PBL) was chosen as the method for this seminar in the second year. PBL is a method of learning which uses true-to-life problems as a stimulus for students to learn problem-solving skills and acquire knowledge of the basic and clinical sciences. Instructors facilitate learning in student-led group tutorials, but do not provide specic content information (Albanese & Mitchell 1993, Vernon & Blake 1993).

lem-solving, decision-making and clinical judgements. An everchanging society demands its practitioners to be skilful in learning to learn and continually relate nursing practice to developing events 3 (Reilly & Oermann 1992, p. 24).

Background
Change to a PBL curriculum was made by McMaster University School of Medicine in the 1960s, followed by other medical schools in Canada (Neufeld & Barrows 1974). This change was thought necessary because of excessive course content, teaching restricted to lecturing, and poor congruence between evaluation and educational objectives (Abrahamson 1977, Guilbert 1985). Recognized as critical was the need to redesign the curriculum for a more studentcentred and humanistic approach (Des Marchais et al. 1992). In a synthesis of all available evaluative research from 1970 to 1992 comparing PBL with more traditional methods of medical education, Vernon and Blake (1993) concluded that the results generally supported the superiority of PBL, especially in faculty attitudes, student mood, class attendance, academic process and humanism. In medicine, Albanese and Mitchell (1993) examined the effects of PBL compared with conventional instruction and found that PBL graduates performed well or sometimes better on clinical examinations, and were more likely to enter family medicine. Students thought they were more nurtured in PBL and both faculty and students enjoyed PBL more than conventional approaches. At the University of Sherbrooke Medical School, PBL stimulated a higher level of critical thinking seen in test results and tutor evaluations. Generally, instructors found the PBL teaching experience rewarding (Des Marchais et al. 1992). In a review of PBL curriculums, Berkson (1993) reported that the graduate of a PBL curriculum is indistinguishable from his or her traditional counterpart. However, attention to principles of learning and pedagogy, which are PBL strengths, resulted in students' greater engagement in learning, more self-direction and higher levels of satisfaction.
Nursing is a cognitive activity. Although its practice entails numerous psychomotor and affective skills, the underlying activities are prob-

In response, several nursing programmes have adopted PBL in various forms, because it promotes higher thinking skills, and combines theory and practice (Barrows 1988, Creedy et al. 1992, Heliker 1994). In analysing the strengths and weaknesses of PBL, for both teachers and students, Frost (1996) argued that PBL is more likely to bridge the gap between theory and practice, and equip nurses with the skills demanded by society. Heliker (1994) observed that the problems in PBL provide opportunities for critical analysis, problem-solving, reection and motivation for continued learning. The learning stimulated by problem situations `enable students to rene their clinical reasoning, make judgements and enhance their interpersonal and procedural skills in planning and implementing care' (Townsend 1990, p. 60). PBL use in a Canadian undergraduate nursing course was benecial to students who learned community health nursing by analysing increasingly complex problems (Edwards et al. 1998). PBL implementation in one Australian nursing programme produced a role change as students became active, self-directed learners with critical thinking and problem-solving skills (Little & Ryan 1988).

Driving forces for change

Pender (1996, p. 282) recommends that `nurses, as part of their education and practice, spend more time in the community as opposed to the hospitals, for it is in the community that ideas and power exist to fuel health promotion efforts'. The shift to community and home care moves the focus away from nursing specialties to client situations in the context of the family and community. Students in our generic programme include a high number of older adults with university education and life experience. There are more males, new immigrants and minorities than usual and many work to support themselves and their families. Our challenge was to implement a programme whose goal was to help students develop such skills as communication, critical thinking, problem-solving, decision-making and management. These skills are the foundation for effective delivery of care and for collaboration in multidisciplinary teams. For students to develop the skills necessary for effective practice, teaching and learning methods must appropriately foster this development. Given the goals of the undergraduate curriculum and the characteristics of our students, a careful choice of teaching

14

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

Issues and innovations in nursing education

Problem-based learning

processes needed to be made to enhance the quality of learning. Adult learners are known to enhance their learning when given the right conditions to pursue self-direction and self-evaluation (Knowles 1984). Mature learners benet more from learning situations who use prior knowledge gained from life and from previous educational experiences. Instruction designed to enhance the personal efcacy of students, therefore, must include self-regulation of learning through goal-setting and opportunities for self-observation, selfjudgement and self-response (Bandura 1993). Gagne (1985) discussed the contribution of problem-solving to the adult learner's need to cope with life problems. If students are to achieve higher cognitive outcomes and transfer of learning, they must apply deep approaches to learning, critical thinking, and cognition of real-life problems and receive feedback on results (Gagne 1985, Alexander et al. 1991). A powerful motivator for adult learners is learning that is purposeful and contributes to fullling personal and educational goals.

Design of the second year PBL course

The second year curriculum was designed to focus on primary and secondary prevention and address the health needs of families and small groups. PBL was planned for groups of nine or less, to address content areas, group process and conict resolution. In addition, students were to explore elements of professional practice such as ethical decisionmaking, legal implications, collaborative practice and the role of professional organizations. Neuman Systems Model (1994) was selected as the framework best suited to address the health needs of the family. Working within the parameters of learning outcomes and required course content, the two teachers responsible for course design and implementation developed ve problem situations focused on families. Content included families at all stages of development. Nursing interventions relevant to problem situations focused on primary prevention and secondary prevention. Examples of client situations included the following: an elderly client with diabetes undergoing surgery for cataract, a pregnant youth in high school, a child with cleft lip and palate undergoing surgical repairs, a child with ear infections and another with asthma whose parents are tobacco users, and a couple with fertility problems. Prior to dealing with the problem situations, each facilitator with her students was expected to discuss group process including time management and group rules, norms and roles. Group rules were to be reviewed for consistency. At the beginning of the semester, each facilitator was to use the rst two class meetings to guide the introduction, discussion and implementation of PBL, using one of the problems. Each

problem of which there were ve, was allotted four to ve meetings for discussion, from introduction to evaluation. Each facilitator was to introduce a problem to her group at designated times. Three problems were to be discussed in the Fall and two in the Winter semester starting with the least complex. Each group was to use one of the meetings, usually the third week, to review research related to the problem. A record of the problem-solving process plus the collated research materials was also planned. Each problem was designed to enable students to learn specic content. The content areas were health care organization and nancing, legislation and service guidelines, scope of nursing practice, primary health care and health promotion. Evaluation of the knowledge gained was to follow completion of each problem solving exercise. In selecting an evaluation method for the course, the concern was with the student's mastery of the objectives in keeping with the criterion approach that closely matches adult education principles (Cranton 1989). A passfail method of evaluation was deemed supportive of self-direction and critical thinking. The PBL evaluation tools developed by Woods (1994) were adopted. These tools evaluated the student's competency in problem-based learning, student's behaviours in relation to performance as a group member, and student's task and morale role performance. The last 10 minutes of each class was to be set aside for reection and evaluation of group performance. A mid-term and nal selfevaluation were also planned.

Preparation of facilitators and students

Four professors whose teaching experiences ranged from teaching large classes to small groups in clinical settings were assigned as facilitators. Teaching modalities used in large classes were mainly didactic while clinical conferences were frequently student-centred. The facilitators had three tasks: rst, learn about PBL; second, develop an understanding of the facilitator role: and third, acquire facilitator skills. Learning the facilitator role in PBL was a challenge. Although all were experienced in leading clinical conferences, none had any experience as PBL facilitators. Thus, their preparation focused on developing the prototypic behaviours of a facilitator within student-centred group discussions, such as `bringing students out, sharpening their awareness of interests and skills and the use of insight and problem-solving to reach goals and avoid blocks' (McKeachie 1986, p. 66). Central to the facilitator role is the ability of the facilitator to break away from teacher-dominated discussions to those that are truly student-centred. If motivation for and commitment to PBL is to be maintained, every
15

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

E.T. Morales-Mann and C.A. Kaitell

facilitator must appreciate its advantages such as the encouragement of greater student participation and responsibility, and the development of group membership and leadership skills (McKeachie 1986). Only one facilitator had any previous PBL experience, others had minimal knowledge of PBL. Preparation of facilitators for PBL was through participation in a one day workshop led by a PBL expert. In preparation for the workshop, a PBL manual for tutors (Edwards & Hugo, 1996a) and other PBL resources were used. The expert used questioning as she walked the participants through the PBL steps. Participants then worked in small groups to formulate the two parts of a problem situation (main problem statement and additional data). Each of the developed problem situations was examined by the large group for appropriateness and completeness, with the guidance of the PBL expert. At an additional 2-hour session with the PBL expert, all aspects of PBL, facilitator role behaviours, and problem situations were examined. Techniques suggested by Woods (1994), which include `assisting learners to relate subject knowledge to problems and their solutions and extracting experience knowledge' (p. 10), were used as guides. PBL problems and specic directions for their use, such as when problems are introduced, when additional problem data is given to students, and the general conduct of the course were also discussed. Facilitators were directed to devote a few minutes of the rst two class meetings to discuss or answer students' questions about PBL. One of the problems was to be used as exemplar for the PBL process. To foster an attitude of openness in the students, facilitators were encouraged to model the exploration of possible solutions. Initially, some facilitators expressed trepidation at their `lack of expertise' in the content presented in the problems. The PBL expert and the course coordinator reassured the facilitators that their task was to assist students to explore each problem and then apply relevant knowledge derived from research and group discussion to problem-solving. Students were prepared in a 2-hour discussion on PBL with a PBL expert. Before meeting the expert, the students read the PBL Manual for Students (Edwards & Hugo, 1996b) and prepared questions for discussion. As students have to be adept at accessing informational resources to gather relevant, credible information to solve PBL problems, they were given a special orientation to the library.

maximum of nine students in each group, but because of economic constraints, the class of 56 students was divided into ve groups. A total of ve problems were discussed throughout the year. For each of the problems, the students progressed through these steps: (1) understanding the problem, (2) learning about the problem, (3) synthesizing and applying learning to solve the problem and (4) reecting on learning (Woods 1994). The rst problem was used as an exemplar to assist students in understanding PBL. There were two meetings where facilitators received support for PBL implementation through sharing effective management strategies, the use of resources and discussion of problems. Facilitators acknowledged being tempted to step in and wrestle control of student discussion or change its direction. Often, some students wanted to limit exploration, prematurely settling for ill-considered solutions. Facilitators encouraged students not to give in to simplistic solutions or decide prematurely on solution pathways. Through skilful questioning, facilitators guided students to consider additional, relevant perspectives.

Student outcomes
A few students complained that some of the problems lacked authenticity because they did not contain `enough' descriptive data as found in case studies. Owing to the fact that a PBL problem is not required to contain exhaustive patient data, some students reported they had to make too many assumptions about the patient. Other students failed to appreciate the iterative PBL process because they preferred a more linear process. To determine how students perceived their learning using PBL, facilitators asked them to respond to four questions. (70% of 56 students responded). The most frequently identied ideas or themes, in their order of frequency are presented as responses (R) to questions (Q). For questions one and three, several themes were identied from common elements contained in the responses. Following each identied theme are select samples of students' responses. Q1. Describe the thinking processes you used in this PBL course R. Problem-solving: `I used a structured and organized way of analysing and assessing a problem thoroughly.' `We brainstormed, developed hunches, identied causal relationships and hypotheses.' `We developed many perspectives, used our prior knowledge and personal experiences to help solve the problem.' Nursing process: `The Neuman nursing theory was used in the problem-solving.' `We used Neuman as we went through

Implementation of PBL
PBL experts recommend groups of ve to six learners and preferably less than nine (Woods 1994). The plan was for a
16

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

Issues and innovations in nursing education

Problem-based learning

the developmental, physiological, etc. variables and dened intra, inter and extrapersonal stressors.' Brainstorming: `Pinpointing new ideas, expanding on them.' `Brainstorming generated and highlighted new ideas.' Research: `Research into particular areas to nd information pertinent to the problem.' `Pointing directions for research after group discussion.' Critical thinking: `Analysing and assessing what is true about the problem.' `Deducting data not suitable in the context of the situation, ranking data in the order of possibility.' (Note: `Brainstorming' and `research' are not thinking processes; however, students used these words in their response to question one.) Q2a. What knowledge did PBL demand? R. Group dynamics, problem-solving steps, information retrieval, community resources and Neuman's model. Q2b. What skills did PBL demand? R. Research skills, communication skills, team work, critical thinking and reasoning skills, active listening skills, interpersonal skills and organizational skills. Q3. Describe the difference(s) between what you learned using PBL and what you learned using other teaching methods? R. Group process: `Learned teamwork and effective interaction with peers.' `There was growth in communication skills.' Self-directed thinker: `I grew in my ability to be selfdirected, to actively pursue relevant resources and be an

independent learner.' `PBL offered opportunities for reection, increased critical thinking and independent thinking.' PBL structure and process: `It was important to learn and understand the PBL structure and process.' `Realization and acknowledgement that the group will arrive at a number of solutions to a problem even when some learners would clearly prefer one solution to the problem.' Research: `The PBL method provided opportunities and the motivation to learn and research materials related to the problem situation, although some content could have been explored in greater depth.' Learning through other methods: `As compared to other learning-teaching methods, such as teacher-led ones, the PBL method motivated my active participation in my own learning.' `Sometimes, the content learned in PBL was supercial, depending on the group's thoroughness in dealing with the problem situation.' Q4. What inuenced your learning and performance in PBL? Personal: Enthusiasm, motivation, past experience, communication skills and conict among group members. Behavioural: Positive attitude of learners, support given and received by group members. Environmental: Group dynamics, team work, group size greater than nine, active participation of facilitator, time constraints (Table 1).

Facilitator outcomes
In spite of facilitators' preparation for PBL, some described their knowledge of PBL and their facilitator role as inadequate. Facilitators who were `stressed and anxious', may have communicated these to students, thus affecting their

Table 1 Outcomes: categories of students' response to questionnaire (listed in decreasing frequency) Thinking processes used 1. Problem-solving approach 2. Nursing process according to Neuman Model 3. Brainstorming 4. Research 5. Critical thinking Knowledge demanded by PBL 1. Group dynamics/ group work 2. Problem-solving process steps 3. How to retrieve information and community resources Skills demanded by PBL 1. 2. 3. 4. Research skills Communication skills Team work Critical thinking/ reasoning skills 5. Active listening skills Learning derived from PBL 1 Group process 2. Structure and process of PBL 3. Research 4. Active participation in own learning 5. Supercial content on occasion if group is not thorough Factors inuencing learning and performance Personal 1. Enthusiasm and motivation 2. Past experience 3. Communication skills Behavioural 1. Positive attitude of learners 2. Support given and received by members Environmental 1. Group effort/dynamics 2. Team work 3. Group size 4. Active participation of facilitator 5. Time constraints

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

17

E.T. Morales-Mann and C.A. Kaitell

learning. All facilitators expressed the need for intensive individual and team preparation to improve their understanding of PBL. Facilitators stated that the effectiveness of the group was lower when group size exceeded nine students. Quiet students were a challenge. Assigning each student to report on research ndings increased participation. A discussion to remind the group about the various roles that contribute to group process encouraged quiet students to assume task and morale roles. When facilitators modelled the sharing of roles and took on roles, such as an energizer or an encourager role and relinquished the leader role to a student, quiet students often participated more willingly. Student development was fostered through regular formative evaluation of the group process. In addition, the group assessed the content learned from each problem. However, facilitators indicated that the passfail grading method did not distinguish students who demonstrated excellent group work, and superior content knowledge from students at only the pass level. Facilitators observed that about half of the students applied effort until they reached a pass level at midterm, at which point no further improvement was noted. About 25% applied consistent effort and continuously raised their achievement, while another 25% applied little or just enough effort from the beginning to warrant a pass grade. Facilitators unanimously endorsed PBL as an effective learning strategy. The learning outcomes of critical thinking, self-direction and effective communication were satisfactorily demonstrated by students.

The involvement of second year facilitators with PBL in future years can build on their previous successes. There will be two benets: students will have facilitators who are condent and knowledgeable about PBL and group process, and the school will avoid the cost of new teacher orientation. The faculty adopted an evaluation process that assesses students on a passfail basis, based on their competency in PBL and performance as group members (Woods 1994). The passfail grading was identied by both students and facilitators as a problem. The majority of students found that a passfail grade reduced their motivation. A few students indicated that they would apply more effort in the course and consequently learn more, if their grade contributed to their grade point average. They wanted an evaluation that would provide clear and specic evidence of their competence. Because the students invested efforts in PBL, a majority believed that their PBL grade should `count' towards their grade point average.

Conclusion
Clear benets for the students from the use of the PBL format include increased autonomous learning, critical thinking, problem-solving, and communication skills. Facilitators were liberated from the traditional role as `content expert and super consultant' and developed additional skills for facilitating the group's critical thinking, problem-solving and conict resolution. Several recommendations for change and improvement are offered to support and enhance PBL. These include the use of PBL in other seminar courses, a reduction of group size to nine or less, improvement of the preparation of students and facilitators, adoption of a grading system that is more appropriate than passfail and the maintenance of a roster of competent facilitators.

Discussion
Student preparation for PBL was insufcient. Because our students were not previously exposed to PBL, allowing more time in between three meetings for preparation, would have been more effective. Learning in the social context of the group (Piaget 1985), can increase students' comfort and commitment to PBL. Increased preparation will also stimulate prior learning, provide guidance, feedback and successful experience with the method (Gagne 1985). Facilitators will be less anxious if they were given additional preparation in the form of workshops and team discussions. All teachers would benet from more practice with PBL. Students and facilitators expressed their desire to continue the use of this method, having experienced the advantages of PBL. The application of PBL in subsequent years would promote the facilitators' and students' selfefcacy and expertise in PBL, and generalize this ability to solve more complex problems. Group process would be enhanced by limiting group size to nine members or less.
18

References
Abrahamson S. (1977) Disease of curriculum. Journal of Medical Education 53, 951957. Albanese M.A. & Mitchell S. (1993) Problem-based learning: a review of literature on its outcomes and implementation issues. Academic Medicine 68, 5281. Alexander P.A., Schallert D.L. & Hare V.C. (1991) Coming to terms: How researchers in learning and literacy talk about knowledge. Review of Educational Research 61, 315344. Bandura A. (1993) Perceived self-efcacy in cognitive development and functioning. Educational Psychologist 28, 117148. Barrows H.S. (1988) How to Design a Problem-Based Curriculum for the Preclinical Years. Springer Publishing, New York. Berkson L. (1993) Problem-based learning: Have the expectations been met? Academic Medicine 68, S79S85.

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

Issues and innovations in nursing education Bruer J.T. (1995) School for Thought. A Science of Learning in the Classroom. MIT Press, Cambridge, Massachusetts. Cranton P. (1989) Planning Instruction for Adult Learners. Wall and Thompson, Toronto. Creedy D., Horsfall J. & Hand B. (1992) Problem-based learning in nurse education. Journal of Advanced Nursing 17, 727733. Des Marchais D.E., Bureau M.A., Dumais B. & Pigeon G. (1992) From traditional problem-based learning: a case report of complete curriculum reform. Medical Education 26, 190199. Edwards N., Hebert D., Moyer A., Peterson J., Sims-Jones N. & Verhovsek H. (1998) Problem-based learning: Preparing post RN students. Journal of Nursing Education 37, 139141. Edwards N. & Hugo K. (1996a) Problem-based Learning Manual: A Tutor's Guide to PBL. University of Ottawa, Ottawa. Edwards N. & Hugo K. (1996b) Problem-based Learning Manual: A Student's Guide to PBL. University of Ottawa, Ottawa. Frost M. (1996) An analysis of the scope and value of problem-based learning in the education of health care professionals. Journal of Advanced Nursing 24, 10471053. Gagne R.M. (1985) The Conditions of Learning (3rd edn). Holt, Rinehart & Winston, New York. Glaser R. (1988) Cognitive science and education. International Social Science Journal 40, 2144. ducation Guilbert J.J. (1985) Les maladies du curriculum. Revue D'E Me dicale 4, 1316. Heliker D. (1994) Meeting the challenge of the curriculum revolution: problem-based learning in nursing education. Journal of Nursing Education 33, 4547.

Problem-based learning Knowles M. (1984) The Adult Learner. A Neglected Species. Gulf Publishing, Houston, Texas. Little P. & Ryan G. (1988) Educational change through problem-based learning. The Australian Journal of Advanced Nursing 5, 3135. McKeachie W.J. (1986) Teaching Tips: A Guidebook for the Beginning College Teacher. D.C. Heath, Toronto. Neufeld V.R. & Barrows H.S. (1974) The `McMaster philosophy': an approach to medical education. Journal of Medical Education 49, 10401050. Neuman B. (1994) The Neuman Systems Model (3rd edn). Appleton & Lange, Stamford, Connecticut. Pender N. (1996) Health Promotion in Nursing Practice (3rd edn). Appleton & Lange, Stamford, Connecticut. Piaget J. (1985) The Equilibration of Cognitive Structures. University of Chicago Press, Chicago. Reilly D.E. & Oermann M.H. (1992) Clinical Teaching in Nursing Education (2nd edn). National League for Nursing, New York. Townsend J. (1990) Problem-based learning. Nursing Times 86, 6162. Vernon D. & Blake R. (1993) Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine 68, 550563. Woods D. (1994) Problem-based Learning: How to Gain the Most from PBL. Hamilton. W. L. Grifn Printing, Ontario.

2001 Blackwell Science Ltd, Journal of Advanced Nursing, 33(1), 1319

19