PRIME

Volume 3 (2) April 2009 ________________________________________________

Contents
Editorial Lin Norton.....................1 Employability Skills and the University Curriculum: Assessing the Experience of Business Graduates Anita Peleg and Felicity FitzGerald ...............................................3 Should students participate in curriculum design? Discussion arising from a first year curriculum design project and a literature review Catherine Bovill, Kate Morss and Catherine Bulley..............................................17 Students in Transition: The Journey from College to University Carole Roberts and Helen Crabtree...........27 Autonomy, Motivation and IT skills: Impacts on the engagement of Physiotherapy students with eLearning Claire Hamshire, Rod Cullen and Christopher Wibberley....37 Self-Assessment Dialogue: added value? The Student Perspective Sara Eastburn.....49 Contemporary Art and the Level 1 Higher Education Curriculum: Empathy, alienation and educational inclusion. Leigh-Anne Perryman.........59 Enhancing learning? Exploring the use of assessment methods as pedagogical tools to promote effective learning and teaching in early childhood studies Paulette Luff and Rachel Pryor .........................................79 A Non-mathematical Framework for Developing the Conceptual Understanding of Classical Mechanics Peter Vivian....89 Encouraging Reason: A pragmatic approach to dialogic teaching in the primary school. John Smith105 Variation in Student Engagement: A Design Model Ian Solomonides and Anna Reid115 Action research cycles on embedding academic skills: how current pedagogical research can steer curriculum Peter Redding...129 Is There a Theory of SoTL? Janet Parker..145 The Interrelationships between Assessment Marks within a First-Year Undergraduate Programme: Some Implications for Aggregating Marks and Recording Achievement Kevin Rowley and Andy Bell.153 Poster Abstract Transitions to Identity in Student Writers Tony Wailey and Susana Sambade...165 (Re)constructing Babel: Mapping the Dynamics of Teacher Training Beatrix Fahnert, Jeremy Hilton and Joe Maloney167

PRIME Volume 3(2)

Editorial
I am delighted to be writing the editorial for the second special conference issue which has come out of the highly successful second international Pedagogical Research in Higher Education conference (PRHE) http://www.hope.ac.uk/learningandteaching/prhe/index.php held in June of 2008, at Liverpool Hope University in the citys year as European capital of culture. Like its predecessor in 2006, delegates commented on the conferences friendly and supportive atmosphere in bringing together pedagogical researchers and practitioners. Thus it fulfilled its aim of providing inspiration not only from the four keynote speakers: Professors Ron Barnett (University of London, UK), John T.E.Richardson (Institute of Educational Technology, The Open University, UK), Sari Lindblom-Ylnne (University of Helsinki, Finland) and Ference Marton (Goteborg University, Sweden), but also from many of the presenters who are experienced pedagogical researchers in their own right. The theme of the conference was that of curriculum change for learning which stimulated over 75 abstracts, of which 43 were accepted as papers and 13 of which have been written up and accepted for this special issue of PRIME. As you will see from these papers, the PRHE 08 conference covered a wide range of issues under its broad theme including employability skills, student participation in curriculum design, students in transition, e-learning, self assessment and assessment methods, educational inclusion, conceptual understandings, dialogic teaching, student engagement, action research cycles, and last but not least and highly relevant to PRHE is there a theory of SOTL? Throughout these papers run some common themes which are of interest to all who work in higher education. In reading them we might ponder on the nature and purpose of higher education, on how students make the transition into higher education and on how they engage with the curriculum. For example, e-learning is often seen across the sector as innovative and as what students expect from a 21st century university education, but often innovation and authentic learning can come from a more profound letting go of our power as teachers and allowing students to participate in designing the curriculum, playing a greater part in assessment such as self- assessment and rethinking traditional approaches in an effort to be genuinely inclusive. This is for some, a step too far, but if there is sufficient robust evidence that it works, then we have a duty to consider it seriously. Finally, the paper by Jan Parker is worth a special mention for in asking if there is a theory of SOTL, she challenges us to think hard about where pedagogical researchers should situate themselves. At the time of writing this editorial, the dates for the 3rd PRHE conference in 2010 are yet to be finalised, but will be announced on the conference website shortly when I look forward to welcoming you back to Liverpool Hope. Lin Norton Dean of Learning and Teaching Professor of Pedagogical Research

PRIME Volume 3(2)

Employability Skills and the University Curriculum: Assessing the Experience of Business Graduates
Anita Peleg and Felicity FitzGerald London South Bank University, Marketing

Summary
London South Bank University (LSBU) is widely recognised for its strong commitment to the development of skills for employment and the employment success of its graduates. This commitment has seen much accomplished research to identify and develop skills at all levels. In particular, research has focused on the following areas: employers and their graduate recruitment requirements, current students and their needs and current skills provision across the university. This has resulted in the development and delivery of new stand alone skills units and skills embedded units. Specifically, the Marketing Department recently improved its provision at level one by embedding skills into the Marketing curriculum following extensive research among current students. However, little research has focused on LSBU Marketing graduates, their reflections on how they benefited from university education as a whole and more specifically their job search and employment experience during the early years of employment. What were the most important outputs of their university education? How did it meet with expectations? How did their education help their development? Were they adequately prepared for the workplace? What skills were most needed? What helped them gain their first and subsequent employment positions? Our research in progress gives guidance for further curriculum planning regarding specific skills delivery. However, it also reveals that while graduates do appreciate some of the specific skills gained, their overall reflections on the benefits of their university education go far beyond the delivery of specific skills and suggest a much broader contribution of higher education. It is therefore essential for London South Bank, while continuing to deliver skills effectively, to augment further the students critical and analytical thinking through learning and debate at the highest level. This research raises important philosophical and practical issues regarding the purpose of university education and the contentious debate on the role of the university in the development of skills for employability. Moreover, it also raises further questions as to the value of the emphasis on skills education at University level and the problems associated with measuring its effectiveness.

Introduction
Following the Dearing Report (1997) and the Government Skills Strategy White Paper launched in July 2003, skills and employability have become a priority for all universities.

Peleg and Fitzgerald

The Strategy aims to strengthen the UKs position as one of the world leading economies by ensuring that employers have the skills to support the success of their business, and that employees have the necessary skills to be both employable and personally fulfilled. (http:www.dfes.gov.uk/skillsstrategy:2003 accessed 16/5/2008) Moreover, the increasing demands of widening participation, and the international marketplace, have resulted in a wide variation in levels of basic skills at entry. This has further fuelled the demand for skills provision with many students needing skills training and support to help their transition into Higher Education, and beyond. The London South Bank University Core Skills Policy (LDC 2003) puts skills at the heart of the universitys corporate plan and its learning and teaching strategy. The result has been two fold: an increased research output focusing on what specific skills to deliver and how best to deliver them, and, the design and execution of new programs embedding the delivery of these skills into the subject curriculum and across the academic program to develop complex learning (Yorke & Knight 2006 p 568). This increased activity surrounding the delivery of skills has in turn fuelled more intense discourse surrounding the value of these skills and of the role of a university education. This paper reviews this employability discourse and attempts to understand the outcome of the skills development agenda from the LSBU graduate point of view i.e. the graduate fitness for purpose, the value of the degree and the associated skills gained and what gaps need addressing.

Literature Review
What is Higher Education For? According to many classical philosophers of Education, Higher Education is in ruins where education becomes a commodity and schools production lines, educated students the products, and teachers rewarded on the basis of their productivity (Bridges & Jonathan 2003 p 132). The basis of this view begins with Platos ideal of education as a theoretical conception of truth, achieved through advanced philosophical reasoning (Hogan & Smith 2003 p 165). This inspired other classical and contemporary philosophers who all advocate learning for learnings sake as a beauteous intensity (Bearn 2000 p 246), a pledge to the freely accepted responsibility to profess the truth (Derida n.d. as cited in Barnet & Standish 2003 p 219), giving education an elite status. In contrast however, Richard Rorty (1990) refuted the relevance of philosophy to education and emphasised the role of society in guiding education what is held to be true by the society to which the children belong (Rorty 1990 p 42 as cited in Hogan & Smith 2003 p 166). For many, Rortys view is the basis of the demise of the University while for others it is the necessary beginning of social justice where higher education no longer has an elite status but is accessible and relevant to all.

Employability Skills and the University Curriculum

The Death of the University? So what is the evidence of this demise of higher education? Many experts suggest that this focus started with a deepening government involvement in education and more specifically in Higher Education. Once funded by charitable and religious institutions for clearly different purposes, universities were perceived as liberal in nature and thought. However, higher education has always been dependent on those who fund it. He who hath the gold maketh the rules (cited in Padro 2007 p 103). The perception was, however, that the original funding institutions were happy to allow these institutions to develop their own liberal thinking and academic community believing that higher education was a formative process aiming at meaningful freedom producing the transformation of the whole man(Habermas 1978 as cited by Barnett 1990 p 22). Today, the new funding institutions, (namely government and industry bodies) demand the contribution of higher education to the development of human capital, the economy and societal wellbeing. The Government sponsored Dearing Report (1997) expressed these developments by emphasising the importance of higher education in its contribution to the economy concluding that the development of key skills for employability should be the central aim of higher education, a view much supported by industry and demonstrated by the increasing involvement of employers in the education debate. This is echoed by similar government initiatives in the US most recently expressed in the Spellings Commission Report (2006) suggesting improved methods of accountability for University Education particularly with regard to workforce productivity and growth. The ability of a society to produce, select, adapt, commercialise and use knowledge is critical for sustained economic growth and improved living standards. (World Bank 2002 as cited in Padro 2007 p 2) Since Dearing (1997), university funding has been made available for this focus on employability, private for-profit companies have been invited to participate in school and higher education, and Universities have sought deeper involvement from industry in curriculum advice, financing and sponsorship. While this has offered much-needed financial support to cash strapped institutions, many claim that it comes at a price, that price being performativity (Lyotard 1984 p11) where education is distorted by an emphasis on goals, productivity and outcomes. Several philosophers have warned against this global relationship between input and output performativity (Lyotard 1984 p11) where emphasis on excellence and performativity is to the detriment of a proper attention to content and to traditions of enquiry (Readings 1996 as cited in Barnett & Standish 2003 p 218). The imaginative advancement of knowledge is then not facilitated in a regime where those responsible for the management of universities and for the quality of their curricula are agents of performativity. (Standish 2003 p 217)

Peleg and Fitzgerald

Current Research Traditional theory on education clearly conflicts with this modern approach to University curriculum development. Current research continues the employability discourse raising issues on the effectiveness of various methods of skills delivery. In particular the debate focuses on delivery as either bolt on units or embedding skills into the curriculum which suggests a more integrated role played by careers and employability skills (Cranmer 2006 p 169) across the whole academic program, where students engage in complex learning (Yorke & Knight 2006 p 567). These discussions also highlight the complexity of measuring the outcomes of these efforts, raising issues about the point at which it is most effective to measure outcomes, whether before or after graduation (Cranmer 2006 p 173), the importance of both short term and long term outcomes (Harvey etc al 2002 as quoted in Cranmer 2006) without overlooking social and educational background prior to studying at university (Morely 2001) and (Moreau & Leathwood 2006). These approaches to evaluating the role and contribution of the university to student employability continue to challenge researchers in this field. Our research attempts to contribute toward the above mentioned unresolved questions, the value of skills delivered, complex learning and how to measure effectiveness. Despite the very practical focus on delivery and evaluation the resulting discussion continues to question whether developing these skills is the role of the university academic or whether this emphasis comes at the expense of deeper subject knowledge. (Sleep & Reed 2006 p 49) The question therefore is whether resources would be better utilised to increase employment-based training and experience, and/or employer involvement in courses (Cranmer 2006 p 169). However, Yorke and Knight (2003) suggest employability and good learning need not compete for curriculum space and suggest that they should complement each other. Definitions of employability and the skills necessary for employment are of course key in this debate. The Dearing Report (1997) focuses on practical skills such as Learning to Learn, Communication, Information Technology, Information Searching, Career Management and Numeracy, while more recent research with employers (Brennan et al 2001 as cited in Yorke & Knight 2006) and (Brown et al. 2002 as cited in Yorke & Knight 2006) and with graduates (Moreau & Leathwood 2006) suggests that personal skills and job specific skills are much more significant. Studies of what employers say they want of new graduate recruits tend to agree that it is the soft generic abilities and personal dispositions that count. (Yorke & Knight, 2006 p 4) As a consequence questions are raised as to whether the necessary skills can be developed over a students short time at university or whether the necessary skills described as personal qualities and attributes are developed over a lifetime. If the general thrust of employers expectations is taken seriously, can higher education do anything to foster the outcomes that employers desire or are the outcomes interlocked with personality in ways that make them resistant to the education process (Yorke & Knight, 2006 p 569) 6

Employability Skills and the University Curriculum

While the majority of the academic world is concerned by the focus on employability and the role of the teacher, others look at these developments more pragmatically and positively. Developments such as widening participation and access to higher education and the removal of the elitist status of higher education are for many positive results of state involvement in education. The focus on employability skills at all levels of education is, therefore, essential in assisting students to fulfil their potential and earn a decent wage to support themselves and to contribute to a stronger economy and society from which they can also benefit. The aim of the policy is to ensure that all those who have the ability to benefit from higher education have the opportunity to do so. Higher education and the opportunities that it brings should be available to all, regardless of their background (http://www.delni.gov.uk/he-widening-participation.htm accessed 24/7/2008) For those advocating social justice, embracing diversity and equality the university is now a place where diversity and culture is to be celebrated, which provides in partnership with the family that background of culture and social habit upon which a healthy society depends (Robbins 1963 as cited in Barnett 1990 p 97). While much of the literature presented suggests the continued demise of the university in a world where the university is dictated to by the needs of society, Ron Barnett (1990) argues that Universities and academics must begin to accept this situation, to recognise the super complexity of society and the link between higher education and the development of society. He suggests that the university can only preserve academic freedom by undergoing its own critical evaluation and therefore urges the academic community not to close rank but to partake in enlightened and engaged action.taking the fight outwards, engaging with the wider society, and partly on its terms. (Barnett & Standish 2003 p 232) The Western university, insofar as it stood for a total institution, sure of itself and of its powers to delight through its internal discourse, is at an end, its discourses awash with those of the wider society. It is for the university to become supercomplex in character.to live in a creative, persistent, and open endeavour of engagement with all around it (Barnett & Standish 2003 p 233) Barnetts challenge is for the university to reclaim it leadership role by recognising its role as provider of intellectual capital within society (Barnett & Standish 2003 p232).

Research Aims & Outcomes

The above literature review identifies several main areas that would benefit from further research. How can Universities reinstate themselves as the primary producer and disseminator of high level knowledge (Barnett & Standish 2003 p 231) so that they can lead the discourse on education and its practical development? How can they ensure that the employability skills requirements and the academic curriculum work along side each other? What is the value of the current skills provision and how can it be further improved to enhance both employability and academic rigour? Is the emphasis on teaching skills misplaced when so much importance is placed on soft personal skills that 7

Peleg and Fitzgerald

cannot be taught during three years at university? Finally how can we assess the value of skills over the long term? By interviewing graduates this project intends to contribute further to this employability discourse on both an academic and practical level. Graduate perceptions of their education as a whole will provide an important dimension to the higher education debate and perhaps take us a step further toward a consensus on how the university can lead. Identification by graduates of the particular skills acquired will further inform the curriculum with suggestions for improved skills delivery, in particular to assist in the preparation for employment. This research will assess current skills provision, especially its utility in the early years of employment, through a longitudinal program of research that considers the complexities of effective measurement raised by Cranmer (2006), Morley (2001) and Moreau & Leathwood (2006) amongst others.

Research Objectives
To understand and explore the experience of graduates in the first five years of their employment To explore the usefulness and value of skills obtained while at LSBU within the first five years in the work place. To develop ideas for further areas of job search and skills development particularly at level three in preparing undergraduate students for employment. To assess the feasibility of setting up an annual tracking study of LSBU BCIM graduates to measure the value their overall education and of the skills provided for employment.

Methodology
Qualitative in depth interviews are to be carried out with 40 LSBU Business Graduates, where respondents are recruited from a mix of marketing and general business related jobs, private and public sector industries and management levels. The graduates are to be a mix of gender, age and ethnic background, and to represent a variety of achievements in terms of grade classification. Individual qualitative interviews were considered appropriate to allow for in depth and personal discussion of experiences, motivations, achievements and difficulties. This paper reports on the findings of research in progress following the successful completion of phase one of the project. Phase one identified 22 Marketing Graduates carrying out depth interviews of 75-90 minutes with each one. Phase Two will build on these findings by selecting graduates of a different business discipline thus affording a comparison.

Findings
The findings were clear and detailed giving a particularly positive picture of the contribution of higher education. Many graduates found it difficult to unpick the contribution of the different skills to their development. Though the overall consensus 8

Employability Skills and the University Curriculum

was that while skills provision was important and could certainly be improved, the main value of their university degree went far beyond specific skills to a broadening of horizons, analytical and strategic thinking and the ability to look at the whole picture. The degree gave me more than just employability, not only knowledge and theory but developed interests and me as a person. It broadened me, made me see beyond just the subjects, to see the bigger world, the real world, to understand the bigger picture and for me, importantly, it has given me the confidence to talk to anyone about anything (NN graduate 2005) Graduates were also very clear that without personal drive and determination and a very proactive approach to the employment search success can only be limited. As a result their advice to current students was to start the job search early, take it seriously, be proactive, carry out extensive research and think ahead. The Value of Skills Graduates valued the embedding of skills into the curriculum, helping them develop generic presentation and communications skills in a manner relevant to their subjects. Figure One identifies the key factors in graduates successful career development: Knowledge, Subject Specific Skills, Job Search Skills and Personal Attributes 1. Knowledge Key for respondents was the broader subject learning that they had taken into the workplace. A number, for example, had used marketing plans or dissertations either as part of an interview task that then gained them the job or were using these in their jobs. Furthermore, the confidence afforded by having the necessary/correct terminology was also important to communicate both internally and externally. It gave you the language, the jargon, so you can talk on the same level with anyone (JW graduate 2001) Figure 1: The Value of Skills Acquired while at London South Bank
Job Search Skills Communication Skills Research Skills Basic Numeracy Industry Knowledge & Jargon Employm ent Skills Team Work Time Management Planning & Organization Thinking on your feet Skills acquired Presentation Skills a big plus Sourcing & referencing Time Management Organizational Skills Team W ork Information Searching Research Analytical & Critical Thinking Report W riting Basic Numeracy Independent Learning Dealing with diversity

Knowledge gained Understanding of Marketing/Business Marketing Planning & Strategy W hole Picture vs. Fragmented Picture Marketing is NOT advertising Business/Marketing Language Specific Topic Areas

COMMONALITIES
Com munication Skills Presentation Skills People Skills Research Skills Referencing/Sourcing Information Management Independence Planning & Organizing language/jargon Industry Know ledge Basic Maths Analytical Thinking

Summarising & Managing Info. Sourcing Analytical Thinking Personal Attributes Determination Enthusiasm/Energy Charm Outgoing W illingness to learn Experience Proactive Attitude Professionalism Responsibility/Independence

Peleg and Fitzgerald

In particular, graduates emphasized the importance of gaining an overall view of the marketing subject area, understanding how all the component parts worked together. 2. Skills The most important skills, reiterated by all interviewees, were communications skills and, in particular, presentation skills. They valued the emphasis placed in particular on presentation skills throughout their three/four years. This not only ensured ability but gave confidence in all oral communications. Indeed one graduate recounted how vital this had proven to be: first and foremost in helping her gain employment and successful promotion but also what a disadvantage there was to graduates without this ability; she herself was sending three of her team for presentation skills training. Doing all those presentations could be a pain at times as it seemed to be one every day but it made me really good at it as I discovered once I started working and could compare myself to colleagues. Importantly though it was a very useful skill in presenting myself to potential employers ( ND graduate 2003) Also highly valued were the skills involved in analytical thinking; in particular to be able to analyse critically, look at issues from different viewpoints, interpret, diagnose and prognose allowing the extraction and evaluation of key issues and developing practical and strategic recommendations. For marketing I could not have done without the theory but far more important was learning to analyze and do so critically. This has now become my hobby (NN graduate 2003) This analytical ability was often mentioned in conjunction with information management skills, referring to the ability to find key information, evaluate and use it successfully in developing alternative strategies. Finally, team work was considered the most difficult and undervalued skill while at university. The requirement to deal with the problems faced in team work, working with diverse groups, provided valuable training for what was now being encountered in the workplace. Teamwork was not something really valued at the time. In fact it was a real burden .. That said it proved to be a fantastic experience and very good preparation for what happens in the workplace. I just wish the reasons for putting us through all this pain had been really explained as it possibly was the most valuable part of the course ( JS graduate 2004) 3. Job search and employment skills. Several interviewees particularly valued skills that assisted in their job search and initial employment. In particular again, presentations skills and thus all communication skills assisted here. This also included basic numeracy and the ability to understand and present simple data. Most notably research abilities were unanimously identified as essential to the job search process. Perhaps unacknowledged before, but the importance of time management was sheepishly emphasized once in employment. 10

Employability Skills and the University Curriculum

4. Personal Attributes Needless to say respondents were initially reluctant, out of modesty, to sound their own trumpet but as they became more involved in the interview and as a result of gentle probing their real opinions came through. A professional and proactive nature, a willingness to learn and to volunteer for everything plus a determination and enthusiasm always gave them an advantage. Each interviewee emphasized how important it was to take the initiative, to be ready to do more than was required and to be forward thinking. They all agreed that a university education could not teach you these personal skills but the confidence gained from achieving a degree contributed significantly to developing these soft skills and therefore to successful employment. I would say what helped me was being very enthusiastic, being willing to learn and research: learning everything you need to learn about the industry and the company itself. Finding examples to apply, doing things over and over to get it perfect, being flexible and adaptive ( JS graduate 2004) Skills Acquired vs. On- the-Job Training When asked about skills acquired by on the job training there were differences but also many interesting commonalities (fig 2). The on the job training usually focused on skills more specific to the particular role and the particular procedures of the employing organization, such as, specific software programs, performance measuring metrics, reporting formats and organizational systems and procedures. However, commonalities, such as team building, information management, presentation skills and numeracy skills, were often repeated in on the job training which raises the question as to how the university and the work place work together to train and further develop these skills? What more could a degree offer? Finally graduates were asked to identify skills that were missing from their skill set which would have helped them in the early years of their careers. These skills focused in particular on the ability to think on their feet, summarise arguments and respond immediately. This suggests the need for more debating, defending of presentations and written work similar to a Viva Voce format. In addition they focused on their understanding of how the industry operates and how to manage their careers. The points raised are summarized into six main areas: Knowledge of the Working World Many expressed the feeling that despite the extensive and in depth marketing education, they still do not acquire enough practical current knowledge of the Marketing Industry, how the work place operates, the different career paths and what they really involved, the roles within these different job specifications, in particular the difference between agency and client work. It was therefore an extremely steep learning curve when looking for work and deciding what career to follow. Suggestions included more guest speakers, facility visits and closer working relationships with industry to create more opportunities for job shadowing, work experience and short term placements. 11

Peleg and Fitzgerald

Aside from more guest speakers....far greater awareness of differences across the industry sectors particularly in communications..Need to learn what it is really like, perhaps workshops in agencies or sending students for a day or a month and/ or internships. What was really missing was the whole area of agency versus client, the differences and, importantly, the media, media agencies, media channels. These are all key these days given the jobs now available ( MO graduate 2003) Fig Two: Skills Acquired vs. On-the Job Training

Skills Acquired vs. On-the-Job Training

Common Skills
Communication Skills Presentation Skills People Skills Research Skills Referencing/Sourcing Information Management Independence Planning & Organizing language/jargon Industry Knowledge Basic Maths Analytical Thinking

On the Job Training

Specific Technical Skills Organizational Procedures & Systems Performance Metrics Corporate Culture Team Work Client Handing Presentation Skills Project Management Additional Courses - Professional Courses - Self Development Courses - Post Grad

Specific Software Training Some mentioned the need for training in specific software programs used in industry such as: Photoshop, Spreadsheets, Database management. Generic Skills Of the generic skills missing from the curriculum many identified the need to be able to react quickly, think on their feet, minute taking, summarizing arguments and reproducing them immediately. Correct English writing skills were also important with an emphasis on more practice of different writing formats, such as, report writing, minute taking, copy writing, chairing meetings, summarizing. With regards to numeracy all felt they were not prepared enough for simple numerical analysis, number crunching, interpretation and presentation. Career Management Skills In hindsight many felt they would have benefited from more practical sessions to assist them with CV writing, interviewing skills, job selection, research and preparation. Interestingly, however, none of them had used the services of the LSBU careers service either because they did not know enough about its services or they felt they didnt need it at the time, a minority also mentioned some disappointment with the nature of the careers service, with a lack of specific information. 12

Employability Skills and the University Curriculum

Specific Subject Related Topics Additional subject areas such as: Digital marketing, Media and media agencies, Evaluation and control (metrics) and Project management were also suggested to help them understand better the range of roles and career opportunities. Initial motivations and expectations vs. overall contribution of the degree to their development. Unsurprisingly when asked about their initial motivations for university study the immediate response was to enhance their employment opportunities, to get a job with new skills and knowledge. Others also suggested the opportunity for independence and a new social life. However, having completed the degree and now in employment, the overall consensus did not focus on skills acquired or needed but on the wider knowledge gained, the contribution of their studies to their own personal development and confidence and in particular a broadening of horizons. Many felt that this training of the mind, the ability to learn and approach issues critically and analytically was what set them apart from those who had not been at university. This learning to learn, this ability to analyse and reflect was particularly important for their personal development. For many therefore the degree had started off a continuous process of learning and personal development which they continue to engage in. In going into that first job, aside from my own personality I guess, what I took that was probably the most important thing was a willingness to learn. I think just by doing the degree this somehow became part of me and something I value now as continuing to learn is something that is important to me. I dont ever want to stop (AF graduate 2003)

Conclusions and Recommendations

In this research we attempt to contribute further to the discourse on the value of skills delivery and the role of the university in delivering these skills for employability. What is the value of skills acquired at university vs. the soft skills that are part of our personal make up? Graduates reiterated the importance of these personal soft skills that developed with the confidence that university level success and enthusiasm about learning gave them. Our research identified the skills acquired at university that were most valued in the workplace and additional skills that could further enhance their preparation for employment. All graduates recognised the importance of proactively searching for placement and internship opportunities during their studies but requested the facilitation of a clearer understanding of industry and the workplace. To bridge the gap and prepare better students for future employment, closer cooperation between university and workplace is essential if universities are to take up the challenge of Barnetts super complex society (Barnett 2003 p 232) and lead the education discourse. Is the university merely developing human capital or does it have a role in developing learning for its own sake? London South Bank has an open access policy which means 13

Peleg and Fitzgerald

that the effective delivery of skills is essential to help students from diverse educational backgrounds and with diverse abilities improve their study and employability skills. However, our graduates emphasized that this is not sufficient and that the real differentiating factor emanates from a higher level of learning and enquiry, creating a critical and analytical ability that takes their employment success to another level. This suggests that while London South Bank continues to prioritise skills it must not lose sight of the higher level of learning and enquiry that is not only expected of a University but also emphatically applauded by its graduates. Universities like London South Bank have a double duty: to continue to embed skills into its curriculum and to further raise the bar and deliver graduates who are able to engage in analytical and critical enquiry in their quest for continuous learning.

Suggestions for Further Research

Phase Two This research in progress focused on Marketing Graduates at LSBU, the second stage of our research will focus on graduates from a different discipline to afford a comparison. Once the full quota of 40 interviews is complete, a more in-depth assessment of profiles of these different graduates will be undertaken in order to understand better how social and educational background and other personal factors may influence their experience and their perception of the value of their university education. Additional Research This research focused on graduates in the first five years of employment. As all those interviewed were already in employment it may also be interesting to interview additional candidates who are currently involved in the job search process but as yet are not successful. It may also be useful to speak to graduates who have not kept in touch with the university as this lack of contact may be an indication of a different experience and perception. Finally it will be useful to carry out the same research on a continuous basis to track the value of curriculum changes and additional skills delivery and to assess the possible influence of the timing of the research, social and education background and other variables. An additional quantitative study is therefore recommended building on the findings of this research in progress. This quantitative study will assess the value of skills amongst a wider variety of graduates from different disciplines and different education backgrounds prior to entering university. It will also assess students at different points in their careers, including those just completing their education and actively involved in job search.

14

Employability Skills and the University Curriculum

References
Barnett, R. (1990) The Idea of Higher Education Education & Open University Press The Society for Research into Higher

Barnett, R & Standish, P. (2003) Higher Education & The University, in N. Blake, P, Smeyers, R. Smith and P.Standish (eds) The Blackwell Guide to the Philosophy of Education, Blackwell Publishing, chapter 12, pp215-233. Bridges, D, & Jonathan, R (2003) Education & the Market, in N. Blake, P. Smeyers, R. Smith and P.Standish (eds) The Blackwell Guide to the Philosophy of Education, Blackwell Publishing, chapter 7, pp126-145. Cranmer S., (2006) Enhancing Graduate Employability: best intentions and mixed outcomes studies in higher education, 31, (2): 169-184 Dearing Report, National Committee of Inquiry into Higher Education (1997) Higher Education in the Learning Society, Report of the National Committee of Inquiry into Higher Education, The Stationery Office, London. Hogan, P. (2003) Teaching and Learning as a Way of Life The Journal of the Philosophy of Education Society of Great Britain, 37, (2): 207-223 Hogan, P. & Smith R, (2003) The Activity of Philosophy and the Practice of Education, N. Blake, P. Smeyers, R. Smith and P.Standish (eds) The Blackwell Guide to the Philosophy of Education, Blackwell Publishing, chapter 9, pp165-180 Moreau, M. & Leathwood, C., (2006) Graduates Employment and the Discourse of Employability; a critical analysis. Journal of Education and Work, 19, (4): 305-324 Padro, F. (2007) The Key Implication of the 2006 Spellings Commission Report: Higher Education is a Knowledge Industry rather than a place of learning, The International Journal of Learning, 14 (5): 98-104 Sleep, M & Reed, H. (2006) Views of Sport Science Graduates Regarding Work Skills Developed at University, Teaching in Higher Education, 11 (1): 47-61 Yorke, M. & Knight, P.T, (2006) Curricula for Economic and Social Gain, Higher Education, 51: 562-588

15

PRIME Volume 3(2)

Should students participate in curriculum design? Discussion arising from a first year curriculum design project and a literature review
Catherine Bovill1, Kate Morss2 and Catherine Bulley3 1 University of Glasgow, Academic Development Unit 2 Queen Margaret University Edinburgh, Centre for Academic Practice 3 Queen Margaret University Edinburgh, Physiotherapy

Summary
This paper outlines some of the findings from a QAA (Scotland) funded project exploring first year curriculum design (Bovill et al. 2008). Whilst many examples exist of curricula being designed in ways to engage first year students, there are fewer published examples of active student participation in curriculum design processes. In the current higher education context where student engagement in learning is emphasised (Carini et al, 2006), this paper asks more generally whether students should be actively participating in curriculum design. In order to answer this question, several elements of the project findings are explored: student views gathered in focus groups; staff views collected in workshops; and the case studies where students were actively involved in curriculum design. The data are examined for lessons that inform the debate about whether students should be participating in curriculum design, in first year and at other levels. Alongside these findings, relevant literature is critiqued in order to ascertain the desirability and feasibility of adopting curriculum design approaches that offer opportunities for active student participation.

Introduction
In higher education there is currently an emphasis on students becoming more engaged in the learning process (Carini et al, 2006). Indeed, there are suggestions that students should become active co-creators of learning (SFC, 2008; SFC, 2006). This has led to some suggestions for greater student participation in designing specific elements of courses such as assessment (Nicol, 2008). There have also been a handful of specific calls for students to become active participants in the design of the curriculum. Fraser and Bosanquet (2006) outline definitions of the curriculum given by academic staff with one definition describing the curriculum as a dynamic, emergent and collaborative process of learning for both student and teacher (Fraser and Bosanquet, 2006:272). This expands the traditional idea of the curriculum towards a view of the teacher and student acting as co-constructors of knowledge (Fraser & Bosanquet, 2006:275). However, a recent research project examining first year curriculum design found few published examples of the curriculum being co-constructed in this way (Bovill et al, 2008). In the following section the project findings that relate to active student participation (ASP) in curriculum design are outlined. 17

Bovill et al.

First year curriculum design project: what did the findings say?
From 2006-2008, the QAA Scotland funded nine projects as part of their first year enhancement theme. One of these projects focused on first year curriculum design. The first stage of the project was the completion of a literature review of first year curriculum design. Data were also gathered from staff workshops, student focus groups and from case studies that provided examples of first year curriculum design which were engaging students. These case studies were collected from throughout the higher education sector in the UK, with fewer examples submitted from Ireland and the USA. Although this was by no means a comprehensive study, the data gathered from staff in workshops, students in focus groups and from the case studies all supported the view that students should be participating in curriculum design. Respondents reported that where students own experiences become a focus for learning and a basis for curriculum design, students found learning to be more relevant and authentic. Others argued that where students are involved in curriculum design, the enhanced choice can lead to personalisation of their learning experience as well as increased responsibility over their own learning. The most frequently mentioned mode of participation involved student feedback on courses. It is widespread practice in higher education for staff to use feedback to inform curricular modifications. Methods of gathering student feedback commonly included use of staff-student liaison committees, feedback questionnaires, focus groups and the use of electronic voting systems. However, despite many participants reporting that staff are reactive to student feedback and are incorporating changes to curricula on this basis, only three case studies were submitted where staff proactively introduced opportunities for students to participate in curriculum design (Bovill et al. 2008). Different levels and models of participation by students were illustrated in these three case studies. For example, in one module at Queen Margaret University, Edinburgh, first year students brought their own experiences to class and the curriculum was constructed around this. The students also contributed to writing of curriculum materials. In a second case study at University College, Dublin, students were paid to design the virtual learning environment for a module they had just completed. Finally, at Elon University, North Carolina, students were paid to work in collaboration with staff to design a variety of courses (see Bovill et al 2008 for further details). The issue is not straightforward, however; while students expressed a strong desire to be challenged in the learning process, staff who took part in workshops as part of this project had some concerns. They asked whether first year students are sufficiently experienced or appropriately prepared to be designing the curriculum. This led the project team to revisit the literature in an attempt to further address the broad question of whether students should be actively participating in curriculum design.

What does the literature on active student participation in curriculum design say?
Calls for student participation in the curriculum go back as far as Dewey (1916) at the beginning of the 20th Century. Others have concurred with Deweys views that students 18

Should students participate in curriculum design?

should share responsibility for curriculum planning (Aronowitz, 1994, 1981; Shor, 1992; Pinar, 1981; Rogers and Freiberg, 1969). Within more recent mainstream higher education literature, there are a handful of specific calls for students to become active participants in the design of the curriculum. These include, for example, those teaching courses that have an explicit remit to promote active, responsible citizenship (Fisher, 2005; Scandrett et al, 2005; Grudens-Schuck, 2003; Wilkinson and Scandrett, 2003), and those involved in language teaching (Breen and Littlejohn, 2000a). Within the literature, there is a range of rationales for students participating in curriculum design. More generally, active and participatory approaches are thought to enhance and support learning (Kahn and ORourke, 2005; Reynolds et al, 2004; Ivanic, 2000; Brown et al, 1989; Kolb, 1984). Some authors within higher education make greater claims and suggest that ASP changes students lives and through this transformation they may become active and critical citizens who can change their communities (Crowther et al, 2005; Scandrett et al, 2005; Wilkinson and Scandrett, 2003). In common with findings from the first year curriculum design project, in the literature, authors argue that ASP in curriculum design is essential to support learning through, for example: students engaging in authentic, relevant and meaningful learning; breaking down the power differential between staff and students; and students experiencing the freedom to become critical thinkers and critical beings in the world (Barnett and Coate, 2005; Rice, 2004; Freire 2003; Taylor et al, 2002; Mezirow, 2000; Rogers and Freiberg, 1969). ASP in curriculum design also enhances student choice, contributing to learners taking more responsibility for their own learning (Hooks, 1994; Rogers and Freiberg, 1969). However, Reynolds et al (2004) caution that we do not know enough about what is meant by participation. They suggest that there is widespread use of the term participation, partly because it is often viewed as unquestionably positive. Despite the justifications for pursuing ASP outlined above, there are also a number of possible drawbacks to ASP in curriculum design outlined in the literature. ASP can be threatening to students who have come through an education system where teachers have dominated the classroom and students may resist new approaches (Shor, 1992; Rogers and Freiberg 1969). Students may also be sceptical of participatory approaches if they have previous experience of tutors claiming to use participatory techniques in which they have been manipulated to create an impression of involvement for the tutors benefit (Reynolds et al 2004). Participatory approaches have also been criticised for reifying the views of the less powerful - in this case the students (Reynolds et al 2004; Cooke and Kothari, 2001). This often means that an uncritical value is placed on the views of students, whatever their views are. This is potentially flawed in the same way the traditional reification of the tutors stance is flawed. So how do the results from the first year curriculum design project and the arguments within the literature help to answer the question posed?

19

Bovill et al.

Should students be actively participating in curriculum design?

Unfortunately, there is little systematic evaluation of the impact of ASP in curriculum design that helps to answer this question. There is a distinct need for further research in this area. Staff in workshops during the first year curriculum design project were concerned that students might not have enough, or might not have the right kind of, knowledge and skills to participate in curriculum design. Whilst these staff were referring specifically to first year students, this is a broader concern where staff may have years of experience of designing the curriculum and may believe students do not have the expertise to make decisions about curriculum planning that will have substantial impact on their learning. Some students may also feel overloaded with work and that curriculum design is the teachers role (Bovill et al, 2008; Martyn, 2000; Slembrouck, 2000; Shor, 1992). Yet, in other areas of academic life, for example, in relation to academic writing skills or student representation, we do not necessarily expect students to have all the skills they require at the beginning of a process. We offer preparation, training and guidance to students to support them in learning about the elements of academic life with which they must become familiar. Therefore, if we think students should be offered opportunities to participate in curriculum design, we may need offer preparation and guidance in the first instance. Staff involved in curriculum design have varying degrees of expertise and experience. They also define the curriculum differently (Fraser and Bosanquet, 2006). Similarly, students are also likely to have varying experiences and definitions of the curriculum. Time may be required to negotiate shared understandings before setting out on actual design processes. Indeed, the level of negotiation needed may take longer than curriculum design processes that staff are used to: Time is absolutely essential in the empowerment process. We have found that it often takes time for students to develop the confidence - and the language - to express pedagogical ideas clearly. Many seem at first to doubt that we will take them seriously. In most course design projects, a moment comes when students suddenly realise that they are being heard. We have begun to structure our course design projects to include an early and public pointwhen students are making an important decision, such as selecting the textbook. This moment typically changes the dynamic of the design group, empowering students to be active participants and showing faculty the value of listening to students (Felten in Bovill et al, 2008: 88). This process obviously requires significant investment of time, energy and skills, but Michael Apple argues that there exists in curriculum developmentsomething of a failure of nerve. We are willing to prepare students to assume only some responsibility for their own learning (Apple 1981:115). Indeed, this leads to another key implication of student participation in curriculum design that the tutor-student relationship is changed. In co-designing the curriculum, there is a challenge to the predominant understanding of the student-tutor relationship where the tutor holds most of the power and students are subordinate. ASP implies a relationship where the tutor and students are learners cocreating the learning experience through dialogue. As Freire explains: 20

Should students participate in curriculum design?

Through dialogue, the teacher-of-the-students and the students-of-the-teacher cease to exist and a new term emerges: teacher-student with students-teachers. The teacher is no longer merely the-one-who-teaches, but who is himself [sic] taught in dialogue with the students, who in turn while being taught also teach (Freire 2003:63). Similarly, Rogers and Freiberg (1969) argue that the teacher becomes a co-learner in this process. This view of a collaborative student-tutor relationship outlined here relies on collective inquiry and dialogue (Haggis 2006; Grudens-Schuck 2003). This dialogue between the tutor and students implies a new view of the learner as a knowledgeable and critical partner in learning (Darder et al, 2003; Freire 2003; Grudens-Schuck 2003; Shor 1992; Aronowitz 1981). However, it is important to note that ASP does not remove the teachers expertise and the key role they have in facilitating learning (Bartolome, 2003; Breen and Littlejohn, 2000b). Although power is shared between staff and students in this model of a co-created curriculum, this approach is demanding of staff, as demonstrated in the previous quote from Peter Felten. The process of co-creation implies that staff will need to be more selfaware, highly flexible, knowledgeable and sensitive to respond to student learning needs and the direction in which the students want to take the curriculum. This negotiated curriculum design process would also be affected by any professional standard requirements, regulatory frameworks and personal views of how a subject should be taught. This context may constrain the level of student participation in the curriculum that is possible, but there may still be room for creative approaches where students ideas and previous experience are valued and utilised within the curriculum planning process. For many tutors it may be uncomfortable to relinquish control over elements of the curriculum. Numerous authors acknowledge that changing power relations tends to be unpopular with the powerful as it implies a giving up of previous privileges (Gwatkin, 2000; Arnstein, 1969). Similarly in higher education, student-centred approaches and student control over elements of curriculum design are likely to face some resistance from those academics who gain privileges (e.g. status, power, money) from being defined as an expert teacher. On the other hand, Grudens-Schuck (2003) suggests that in courses that are teaching about participation and social justice, adopting ASP in curriculum design reduces cognitive dissonance for tutor and students. The process of co-constructing the curriculum offers opportunities for greater clarity over the expectations of tutor and students about the aims of the curriculum and the potential impacts on learning. It is also likely that the experience of being involved in curriculum design will enhance students awareness of the learning process and how different elements of the curriculum impact on learning, such as: timetabling; setting learning outcomes; setting assessments; and choosing textbooks. Through this process the student gains greater control over their own learning. Another way of involving students in the curriculum design process is to enable their participation at a later stage and therefore capitalise on their experience of a course. In one case study from the first year curriculum design project, students who had completed a course at University College, Dublin were involved in its redesign (Bovill et al, 2008). The advantage here was that students had experienced the course and held useful views as to how the curriculum might be 21

Bovill et al.

redesigned. They also gained experience of curriculum design. The disadvantage in this case was that this design was retrospective and the process did not enable these students to work on the curriculum for a course which they were currently studying their curriculum design impacted on other students who had no influence upon their own curriculum. Having presented a mixed picture from the first year curriculum design project and the literature, what conclusions can we draw?

Conclusions
Questions might be raised as to whether the current higher education context is supportive of ASP in curriculum design. The implied shifts in power and control between tutor and student would require a university which encourages students to act critically and to challenge and question the world in which they live (Barnett, 1997; Haggis, 2006). Yet, many authors have raised concerns that universities are losing their criticality in the face of the recent surge of managerialism and instrumentalism in the UK higher education sector. They suggest that this vision of a critical higher education may be under threat (Barnett and Coate, 2005; Rice, 2004; Taylor et al, 2002; Barnett, 1997). On the basis of previous discussion, we should not assume that ASP is always positive or appropriate. Indeed there is a need for further evaluation and research into the impacts of ASP in curriculum design. There is also a need to examine the feasibility and desirability of ASP in curriculum design in different contexts and to investigate the factors which influence the nature of ASP in curriculum design within these contexts. Nevertheless, if our current goals in higher education include enhancing student engagement in learning, and if students have a desire to be challenged in the learning process, then ASP in curriculum design may be an area which we need to explore further.

References
Apple, M.W. (1981) On analysing hegemony. In Giroux, H.A., Penna, A.N. and Pinar, W.F. (Eds) Curriculum and instruction alternatives in education, Berkerley: McCutchen Publishing. Arnstein, S.R. (1969) A ladder of citizen participation, Journal of the American Institute of Planners, 35 (4) 216-224. Aronowitz, S. (1994) A different perspective on educational equality, Review of Education / Pedagogy / Cultural studies, 16 (2) 135-151. Aronowitz, S. (1981) Politics and higher education in the 1980s, In Giroux, H.A., Penna, A.N. and Pinar, W.F. (Eds) Curriculum and instruction alternatives in education, Berkerley: McCutchen Publishing. Barnett, R. (1997) Higher education: a critical business, Maidenhead: Society for Research into Higher Education and Open University Press. 22

Should students participate in curriculum design?

Barnett, R. and Coate, K. (2005) Engaging the curriculum in higher education, Maidenhead: Society for Research into Higher Education and Open University Press. Bartolome, L.I. (2003) Beyond the methods fetish: toward a humanizing pedagogy, In Darder, A., Baltodano, M. and Torres, R.D. (Eds) The critical pedagogy reader, New York: Routledge Falmer. Bovill, C., Morss, K. and Bulley, C. (2008) Curriculum design for the first year, First Year Enhancement Theme Report, Glasgow: QAA (Scotland). Breen, M.P. and Littlejohn, A. (2000a) The Significance of negotiation, In Breen, M.P. and Littlejohn, A. (eds) Classroom decision-making: negotiation and process syllabuses in practice, Cambridge: Cambridge University Press. Breen, M.P. and Littlejohn, A. (2000b) The practicalities of negotiation, In Breen, M.P. and Littlejohn, A. (eds) Classroom decision-making: negotiation and process syllabuses in practice, Cambridge: Cambridge University Press. Brown, J.S., Collins, A. Duguid, P. (1989) Situated cognition and the culture of learning, Educational Researcher, 18 (1) 32-42. Carini, R.M., Kuh, G.D. and Klein, S.P. (2006) Student engagement and student learning: testing the linkages, Research in Higher Education, 47 (1) 1-32. Cooke, B. and Kothari, U. (2001) The Case for Participation as Tyranny, In Cooke, B. and Kothari, U. (eds) Participation the New Tyranny? London: Zed. Crowther, J., Galloway, V. and Martin, I. (2005) Introduction: radicalising intellectual work, In Crowther, J., Galloway, V. and Martin, I. (eds) Popular education: engaging the academy. International perspectives, Leicester: Niace. Darder, A., Baltodano, M. and Torres, R.D. (2003) Critical Pedagogy: An Introduction, In Darder, A., Baltodano, M. and Torres, R.D. (Eds) The critical pedagogy reader, New York: Routledge Falmer. Dewey, J. (1916) Democracy and education: an introduction to the philosophy of education, New York: The Macmillan Company. Fischer, M.C.B. (2005) The methodology of systemisation and its relevance to the academy, In Crowther, J. Galloway, V. and Martin, I. (eds) Popular education: engaging the academy. International perspectives, Leicester: Niace. Fraser, S, and Bosanquet, A, (2006) The curriculum? Thats just a unit outline, isnt it? Studies in Higher Education, 31(3) p.269-284. Freire, P. (2003) From pedagogy of the oppressed, In Darder, A., Baltodano, M. and Torres, R.D. (Eds) The critical pedagogy reader, New York: RoutledgeFalmer.

23

Bovill et al.

Grudens-Schuck, N. (2003) No beginners: teaching participation at the graduate level, PLA notes, 48 (12)11-14 Gwatkin, D.R. (2000) Health inequalities and the health of the poor: what do we know? What can we do? Bulletin of the World Health Organisation, 78 (1) 3-18 Haggis, T. (2006) Pedagogies for diversity: retaining critical challenge amidst fears of dumbing down, Studies in Higher Education, 31 (5) 521-535. Hooks, B (1994) Teaching to transgress: education as the practice of freedom, New York: Routledge. Ivanic, R. (2000) Negotiation, process, content and participants experience in a process syllabus for ELT professionals, In Breen, M.P. and Littlejohn, A. (eds) Classroom decision-making: negotiation and process syllabuses in practice, Cambridge: Cambridge University Press. Kahn, P. and O Rourke, K. (2005) In Barrett, T., MacLabhrainn, I. and Fallon, H. (Eds) Handbook of enquiry and problem-based learning. Irish case studies and international perspectives, All Ireland Society for Higher Education. Online. Available: <http://www.aishe.org/readings/2005-2/contents.html> (accessed 2 June 2008) Kolb, D.A. 1984. Experiential learning: experience as the source of learning and development, New Jersey: Prentice-Hall. Martyn, E. (2000) Syllabus negotiation in a school of nursing, In Breen, M.P. and Littlejohn, A. (eds) Classroom decision-making: negotiation and process syllabuses in practice, Cambridge: Cambridge University Press. Mezirow, J. (2000) Learning to think like an adult. Core concepts of transformation theory, In Mezirow, J. (ed) Learning as transformation. Critical perspectives on a theory in progress, San Francisco: Jossey Bass. Nicol, D. (2008) Assessment as a driver for transformational change in HE, ESCalate newsletter Number 10 Spring. Online. Available: <http://escalate.ac.uk/4451> (accessed 2 June 2008) Pinar, W.F. (1981) The reconceptualization of curriculum studies, In Giroux, H.A., Penna, A.N. and Pinar, W.F. (Eds) Curriculum and instruction alternatives in education, Berkerley: McCutchen Publishing. Reynolds, M., Sclater, M. and Tickner, S. (2004) A critique of participative discourses adopted in networked learning, EQUEL Symposium and Papers presented at the Networked Learning 2004 Conference, Lancaster University, April 5-7. Rice, C.D. (2004) Enlightened universities: beyond political agendas, Edinburgh: Policy Institute Series: Society No. 4.

24

Should students participate in curriculum design?

Rogers, C. and Freiberg, H.J. (1969) Freedom to learn, (3rd ed.) New York: Macmillan Publishing. Scandrett, E., O'Leary, T. and Martinez, T. (2005) Learning environmental justice through dialogue, PASCAL Conference Proceedings: Making Knowledge Work. Leicester: NIACE. SFC (2008) Final report from Joint Quality Review group to Council. Scottish Funding Council, Online. Available: <http://www.sfc.ac.uk/about/new_about_council_papers/about_papers_17aug07/SFC_07 _113_ANNEX.pdf> (accessed 2 June 2008) SFC (2006) Taking forward learning to improve: a report from the Learning and Teaching Forum, Scottish Funding Council, Online. Available: http://www.sfc.ac.uk/publications/Taking_forward_learning_to_improve_Feb_07.pdf> (accessed 2 June 2008) Shor, I. (1992) Empowering education. Critical teaching for social change, London: University of Chicago Press. Slembrouck, S. (2000) Negotiation in tertiary education: clashes with the dominant educational culture, In Breen, M.P. and Littlejohn, A. (eds) Classroom decision-making: negotiation and process syllabuses in practice, Cambridge: Cambridge University Press. Taylor, R., Barr, J. and Steele, T. (2002) For a radical higher education after postmodernism, Maidenhead: Society for Research into Higher Education and Open University Press. Wilkinson, M. and Scandrett, E. (2003) A popular education approach to tackling environmental injustice and widening participation, Concept, 13 (1/2) 1116.

25

PRIME Volume 3(2)

Students in Transition: The Journey from College to University

Carole Roberts and Helen Crabtree University of Salford, Business School

Summary
The aim of this study is to better understand how the teaching and learning environments in sixth form and further education colleges and in universities differ. It was conducted in a university Business School and eight local colleges (both sixth form and further education). The first part of the study involved in-depth interviews with staff. Findings from these interviews have been fully reported elsewhere (Crabtree et al, 2007) and will be summarised here. They confirmed that the environments are very different. For university staff there is the expectation that students become independent learners whilst in colleges there is an emphasis on staff managing the students learning activities and students being led to learn. The interviews were followed by questionnaires administered to both students in the colleges and a year later to first year students in the Business School. While college students believe they are learning more independently at college than at school it is clear from their expressed learning preferences and the success of the colleges in satisfying these that they are unlikely to be developing the skills needed for success within the HE environment. Indeed the first year university students responses confirm that whilst they believe they have continued to take more responsibility for their own learning since college their current learning preferences are generally not being met. The results have implications for HE teaching and learning policy and practice to facilitate student transition into HE. Whilst the research focuses on business and business-related qualifications, the results are likely to be of interest to all those involved with first year undergraduate students, irrespective of subject or discipline of study.

Introduction
Within universities there has been an increasing emphasis on widening participation which has resulted in a rapid increase in student numbers. This has changed higher education from an elite to a mass education system but as Scott (1995) suggests British higher education has become a mass system in its public structures, but remains an elite one in its private instincts and there appears to be little evidence of a systematic change in teaching and learning policy and practice to accommodate the needs of the changing student population. This failure to adapt creates difficulties for both academic staff and students, with staff expressing concerns about students attitudes to and motivation for learning (see Ottewill and Macfarlane 2003, Hayward et al 2006) and students finding it difficult to adjust to the demands of the higher education learning environment (see for example Lowe and Cook 2003). To some extent this is a problem of transition and a number of authors (e.g. Ozga and Sukhunandan 1998, Cook and Leckey 1999, Byrne and Flood 2005) have suggested that most students entering university may not have appropriate expectations or the necessary skills required for effective learning. In practice students base their expectations and 27

Roberts and Crabtree

learning strategies on their previous educational experiences (Vermunt 1998, Cook and Leckey 1999) and if these are very different from those experienced in the university, new students may struggle to adapt. The results from the first part of the research study, based on interviews with college teachers in sixth form (SFC) and Further Education (FE) colleges and tutors in the university Business School, suggest that the teaching and learning environments in the college and higher education sectors is very different. Detailed results have previously been reported (Crabtree et al 2007). In summary, the findings show that within the college environment, the emphasis on performance management and levels of achievement leads to little opportunity for students to engage in independent study and students being led to learn. In this environment, college teaching staff manage and closely supervise learning and students are provided with regular feedback on their standard of performance. Students are expected to achieve to the best of their ability however the close relationship between staff and student ensures that lots of support and guidance is provided. In contrast, tutors in the university environment expect that students will take an active responsibility for their own learning. They emphasise that their role is to facilitate learning and the students development as an independent learner rather than to teach, and this, in addition to the larger group size found in university, leads to a more anonymous, less supportive environment for students. The purpose of the second part of the study is to investigate the impact that these different teaching and learning environments have on the student experience and the extent to which students are able to adjust during their first year in university.

Methodology
This study used questionnaires to collect data from students studying business related A levels or equivalent vocational qualifications in 4 Further Education colleges, 3 Sixth Form colleges and first year undergraduate students at Salford University Business School. College students were asked to complete these questionnaires during a class in April 2007 and returned them to their tutor in a sealed envelope to ensure confidentiality. In total 192 usable questionnaires were received, almost a 100 percent response rate. Approximately 400 questionnaires were distributed to university students and completed during lectures in April 2008. A total of 163 usable questionnaires were returned. The questionnaires were designed to be completed in about 15 minutes, in order to facilitate completion in class. The final design was informed by a series of pilots using college students from colleges not in the sample who were attending University enrichment days. Demographic data (eg gender, ethnicity) was collected from the students, together with details of their current programme of study and previous educational attainment. In order to explore students attitudes to and experiences of their current teaching and learning environment they were asked to indicate their preference on a 5 point Likert scale between options which reflected the differing teaching and learning practices in college and university these options having been identified in the earlier interview-based study (Crabtree et al 2007). One of the main issues was to design questionnaires which would maintain students interest in order to maximise completion and this was a factor in the choice of a preference scale to explore attitudes. 28

Students in Transition

Eight of these items explored preferences in relation to the role of the teacher and the student. These are shown in Table 1. Table 1: Questions combined to produce the independent learning variable
I prefer to be given all the information I need about a particular subject I prefer to be told where to find the information I need for a subject 1 .. 2 .. 3 .. 4 .. 5 I prefer to be told where to find the information I need

1 .. 2 .. 3 .. 4 .. 5

I prefer to use my own initiative about how and where to find any necessary information I think it is better to work on projects

I think it is better to do exercises/examples or practice exam questions There is no need to read about a subject before it is covered in class I prefer to listen to the teacher in class

1 .. 2 .. 3 .. 4 .. 5

1 .. 2 .. 3 .. 4 .. 5

It is useful to prepare ahead for a class

1 .. 2 .. 3 .. 4 .. 5

I prefer to try things out for myself

I like my progress to be checked regularly by my teacher/tutor I prefer the teacher to tell me how to improve my work I like to know exactly what I am expected to do in class or for an assignment

1 .. 2 .. 3 .. 4 .. 5

I am happy to monitor my own progress

1 .. 2 .. 3 .. 4 .. 5

I prefer to make my own decisions about if and how to improve I prefer to make my own decisions about exactly what is required

1 .. 2 .. 3 .. 4 .. 5

For each student the scores on these eight items were added together in order to produce an independent learning score. Thus the range of possible values on this independent learning variable was between 8 and 40. A low score (between 8 and less than 24) indicated that the respondent generally preferred to be led by the teacher whilst a score in excess of 24 indicated a preference to take more personal responsibility for their own learning. A small number of open questions were also included in the questionnaires to enable participants to give further explanations and to expand on their responses about their experiences and learning preferences. SPPS was used to analyse the quantitative data collected. Tests used included independent sample t-test, Chi squared and regression analysis. Content analysis was used to identify common themes and topics raised in the responses to the open questions.

29

Roberts and Crabtree

Findings
The sample of students from college and the university participating in the study was comparable and fairly representative of the populations participating in tertiary and higher education in these institutions (see Table 2). Table 2: The nature of the sample College students 192 54% male, 46% female 62% white, 38% ethnic minority 51% SFC, 49% FE 60% A-levels 7% vocational A-levels 33% BTEC NC/ND University students 163 60% male, 40% female 50% white, 50% ethnic minority 49% SFC, 28% FE, 23% school sixth form 75% A-level 16% vocational A-level 23% BTEC

Number of students responding Gender Ethnicity College experience College qualifications

Roughly equal numbers of the college students attended sixth form and FE colleges. Two thirds of the respondents were studying for A-level qualifications and one third were taking BTEC National Certificate or National Diploma in Business. In this sample 66% of the respondents reported that they hoped to go on to university at the end of their college course. A higher proportion of university student respondents had studied Alevels and more had studied previously in a sixth form environment (Sixth Form college or school sixth form) rather than in FE colleges. The analysis of findings relating to student learning experiences and attitudes to independent learning is presented in Table 3. Considering first the independent learning variable, results show that both college and first year university students prefer to be teacher led. The mean score for college students was 17.04 and whilst the university students mean score was statistically significantly higher at 18.45 (p<0.001) this is still well below a score of 24 and indicates a tendency towards more reliance on teachers. This contention is supported by the fact that 93% of college students and 83% of university students scored <24. Again whilst the proportion of university students is significantly lower (p=0.02) 83% represents a large majority. College students were more likely to report that their current experience matched their learning preferences than university students. Whilst the mean score on their responses to the item was less than 3 for both groups (2.27 and 2.71 respectively, significantly different at p<0.001), 65% of college students scored 1 or 2 whilst only 47% of university students did so. Similarly a smaller proportion (12%) of college students scored 4or 5 compared with university students (22%).

30

Students in Transition

Table 3: Some results from the sample College students In general my college/university experience matches my learning preferences (1-strongly agree, 5strongly disagree) Independent learning score (possible range 8-40) Mean score: 2.27 + 1.04 University students Mean score: 2.71 + 1.07 Statistical significance Independent sample t test p<0.001 X2 test p<0.001 Independent sample t test p<0.001 X2 test p=0.02 Independent sample t test not significant X2 test not significant

65% agree 12% disagree Mean score: 17.04 + 4.26 range 8-31)

47% agree 22% disagree Mean score: 18.45 + 4.41 (range 9-32) 83% of respondents had score < 24 Mean score: 1.60 + 0.89

93% of respondents had score < 24 At college/university I Mean score: am required to take more 1.72 + 0.99 responsibility for my own learning than I did at school/college 86% agree (1-strongly agree, 57% disagree strongly disagree)

88% agree 5% disagree

In general my current college/university experience matches my learning preferences

1 .. 2 .. 3 .. 4 .. 5

My current college/university experience does not match my learning preferences

Again these results are significantly different (p<0.01). For college students there was a strong correlation between the response to this item and the score on the independent learning variable (p<0.001) indicating the extent to which these students appreciated the fact that they were teacher led. In contrast, there was no significant correlation for university students. Thus while their mean score on the independent learning variable was higher than for the college students - indicating that they had adapted somewhat to the demands of the new, less supportive environment - their current teaching and learning experiences were not matching their learning preferences; they were generally not yet comfortable with the extent of independent learning which university teachers expect of them and the majority still preferred to be instructed and guided by staff. Broad (2006) showed that taking responsibility for their own learning is what college students understand by independent learning.
At college/university I am required to take more 1 .. 2 .. 3 .. 4 .. 5 responsibility for my own learning than I did at school At college/university I am not required to take more responsibility for my own learning than I did at school

31

Roberts and Crabtree

Thus in the responses to the item it was interesting to note that 86% of college students and 88% of university students felt that they were now more responsible for their own learning than in their previous institutions thus illustrating their recognition of the learning journey from school, through FE to HE. When asked about the best and worst aspects of studying at college rather than at school it was evident that many college students valued the increased level of freedom and independence and the teaching methods and support provided (60% and 15% of responses respectively), but at the same time there were concerns from some about the challenging nature and demands of the work required and the lack of support provided (33% and 11.5% of responses). In practice only 25.5% of university students reported that they had had difficulty adapting to the change between college and university, with 74.5% reporting no difficulty. Females (36.5%) were more likely to report difficulty than males (18.6%) (p<0.05) but there were no significant differences noted based on age on entry or ethnicity. Factors which students reported as helping them to adapt included the academic environment (helpful lecturers/tutors, use of the University virtual learning environment, group work, course materials), the social environments (new friends, the relaxed/friendly atmosphere), and personal characteristics (determination, organisation and previous academic experiences). These factors accounted for 38.6%, 32.3% and 13.2% of the responses respectively. Whilst factors reported as not helping included the impersonal environment (the lack of academic support, large lecture sizes), the requirement to learn and study independently and logistics (such as travel and timetable constraints), accounting for 30.1%, 14.3% and 15.9% of the responses. Regression analyses were used to identify factors which were associated with a preference for independent learning for both college and university respondents. These indicated that for both groups there was no significant relationship between the independent learning variable and the type of college attended, the qualification studied at college, gender, ethnicity or year of study (year 12 or 13 for college students and first or repeat year for university students). However there were associations noted between a preference for independent learning and the reason for study being an interest in the subject rather than as a way of enhancing employability (p<0.05 for college students and p=0.011 for university students). In addition college students demonstrated an association between independent learning and the level of attainment on entry (number of GCSEs grade A-C p<0.05). This association with previous academic attainment (GCSEs or tariff points on entry) was however not significant for university students. Results confirmed that university students who scored higher on the independent learning variable were less likely to report that they had had difficulty adapting to the change between college and university (p<0.01). No correlation was observed between a preference for independent learning and the students conception of learning (as building up knowledge by acquiring facts and information or seeing things in a different and more meaningful way).

32

Students in Transition

Conclusions
The regression analysis for college students indicated that students who were already showing a greater preference for independent learning tended to have higher qualifications and an intrinsic interest in the subject. Such students may be considered to be more like what would have been regarded as traditional students before the widening participation initiatives. This may provide some explanation for why the requirement for independent learning in HE appeared to be less of a problem in the past and why many lecturers in HE today often think about the issues in terms of student deficit, basing their teaching and learning strategies on their own experiences and past expectations. The findings of this study thus support the suggestion made by Harvey et al (2006) that an understanding of the first year experience of students in university may be improved if it is seen as a process of transition between college and university rather than a problem of student deficit. Whilst the success of the colleges in satisfying students learning preferences implies that they are unlikely to be developing the skills needed for immediate success within the HE environment, our study confirms that the moves from school to college and thence from college to university are nonetheless steps along the way to greater independence. Similar results showing increasing independence in college students were demonstrated by Broad (2006) who reported that 16-19 year old FE students were aware of the importance of taking control of (and responsibility for) their own learning and thought that they were more capable of doing this than previously. Broads results suggest that independent learning is a skill that can be developed but it takes time to adapt. Our findings indicate that after a period of 6-7 months in the new university teaching and learning environment, students have made some adjustment. However at this stage most students would still prefer to be instructed and led to learn than to learn independently. This suggests that the transition process is likely to be ongoing as students enter the second and final year of their degree. Such changes are probably related to changes in cognitive development which occur throughout a students academic career (Thoma 1993). The findings also confirm previous findings which highlight the importance of social factors and personal characteristics in the process of integration into higher education (eg Roberts et al 2003, Trotter and Roberts 2006, Harvey et al 2006). The study by Trotter and Roberts (2006) stressed the value of programme-based activities which facilitate and promote social and academic integration as part of the early student experience. These activities may help students to adapt by enhancing their confidence and improving their self concept. Indeed Hodkinson et al (2000) argue that the extent to which students are able to assimilate into the overall institutional culture has a direct effect on a students learning behaviour. Their research in a sixth form college suggested that belonging to a tightly bounded community with a positive institutional culture for learning, promoted positive attitudes to learning. Thus achievement may be influenced not just through feeling more confident but more importantly through the experience of belonging to and conforming to a community of practice. Overall this study confirms the importance of understanding the students prior experiences of learning and to be aware of how this will influence their attitudes on entry into university. It suggests that providing the right environment in higher education is an important factor in facilitating student development. We would argue that we should not be dumbing down in order to accommodate the stage of development that students have 33

Roberts and Crabtree

reached when they leave college nor lower our expectations of what students should achieve by the end of their degree programme. year. Rather we should develop an institutional culture where success is expected as the norm and where both staff and students have responsibilities to ensure this. The first year experience is crucial, both in terms of student persistence and achievement. For academic staff this means that in the first year of an undergraduate programme they need to provide students with appropriate challenges but also provide them with sufficient guidance and support appropriate scaffolding - in order to help them to successfully make the transition to HE. This may include structuring assessment to ensure it provides appropriate and timely feedback (which may itself involve considering the effects of semesterisation and the shape of the academic year); making assessment requirements explicit and discussing exemplars to help students understand better what is required and providing clear guidance to the students of their own responsibilities in ensuring success.

References
Broad J. (2006), Interpretations of Independent Learning in Further Education, Journal of Further and Higher Education, 30, (2): 119-143 Byrne M. and Flood B. (2005), A study of Accounting students motives and preparedness for higher education, Journal of Further and Higher Education, 29, (2): 11124 Cook A. and Leckey J. (1999), Do Expectations Meet Reality? A survey of changes in first year student opinion, Journal of Further and Higher Education, 23, (2): 157-71 Crabtree H, Roberts C. and Tyler C. (2007), Understanding the problems of Transition, Proceedings of 4th Education in a Changing Environment Conference, 12th -13th September. Online. Available HTTP: <http://www.ece.salford.ac.uk/proceedings/papers/35_07.pdf > (accessed on 10 June 2008) Harvey L, Drew S, & Smith M.(2006) The First Year Experience: A Review of Literature for the Higher Education Academy. Online Available HTTP: <http://www.heacademy.ac.uk/research/Harvey_Drew_Smith.pdf >(accessed on 11 May 2007) Hayward G, Hodgson A, Johnson J, Oancea A, Pring R, Spours S and Wright S. (2006), Focus groups with higher education institutions in The Nuffield Review of 14-19 Education and Training Annual Report, 267-76. Online. Available HTTP: <www.nuffield14-19review.org.uk/files/documents129-6.pdf > (accessed on 21 March 2007) Hodkinson P and Bloomer M (2000), Syokingham Sixth Form College: Institutional culture and dispositions to learn, British Journal of Sociology of Education, 21(2): 187202 Lowe H. and Cook A. (2003), Mind the Gap. Are students prepared for higher education, Journal of Further and Higher Education, 27, (1): 53-76 34

Students in Transition

Ottewill R and Macfarlane B (2003), Pedagogical Challenges facing Business and Management Educators: Assessing the evidence, International Journal of Management Education, 3, (3): 33-41 Ozga J and Sukhandan L (1998), Undergraduate Non-completion: Developing an explanatory model. Higher Education, 52, (3): 316-33 Roberts C, Watkin M, Oakey D and Fox R (2003) Supporting Student Success: What can we Learn from the Persisters?, Proceedings of Inaugural Education in a Changing Environment Conference, 17th -18th September. Online. Available HTTP: <http://www.ece.salford.ac.uk/proceedings/papers/35_07.pdf >(accessed on 10 July 2008) Scott P. (1995), The Meanings of Mass Higher Education, Buckingham: The Society for Research into Higher Education and Open University Press Thoma G.A. (1993), The Perry Framework and tactics for Teaching Critical Thinking in Economics, Journal of Economic Education, 24, (2): 128-35 Trotter E and Roberts C (2006), Enhancing the Early Student Experience, Higher Education Research and Development, 25 (4): 371-386

35

PRIME Volume 3(2)

Autonomy, Motivation and IT skills: Impacts on the engagement of Physiotherapy students with eLearning
Claire Hamshire, Rod Cullen and Christopher Wibberley Manchester Metropolitan University, Professional Registration Division

Summary
At Manchester Metropolitan University, the concept of the independent autonomous learner is at the heart of institution changes in the learning, teaching and assessment processes and the implementation of an institutional Managed Learning Environment (MLE). In the Physiotherapy programme team we have conducted a mixed methods evaluation of the provision of online resources that aim to facilitate autonomy from the beginning of the programme and are delivered via the WebCT VISTA component of our MLE. Primarily, we investigated facilitators and barriers to uptake and use of these resources by students. Overall, students reported a very positive experience of online activities, with a broad range of factors influencing uptake and engagement. Extrinsic factors related mainly to technical (e.g. home PC setup) and administrative (student enrolment, network access and support) difficulties. These had less impact on our studys metrics than intrinsic factors such as autonomy, motivation and IT skills. Our evaluations have also highlighted a mismatch between the programmes aspirations and student perspectives of autonomy. We have made links between the levels of autonomy, motivation and IT skills of our students and considered ways of addressing these issues within the Physiotherapy curriculum. As a result we are in the process of devising a new induction programme which aims to provide scaffolding that will motivate our students and assist their development as independent autonomous learners.

Introduction
Manchester Metropolitan University is currently engaged in fundamental institution-wide changes to learning and teaching provision. Strategies are being devised around: the principles of student centred learning; the opportunities offered by an institutional Managed Learning Environment (MLE); and the need to make efficiencies in learning, teaching and assessment processes (see Brookes 2005a, 2005b). At the heart of this rethink is the concept of the independent autonomous learner. The National Health Service (NHS) provides a further driver for change in health professional education. Since 1996, when the Information for Health strategy was launched there has been a move to make better use of information technology in all aspects of patient care and staff development (Glen and Cox 2006). The NHS now requires a computer literate workforce able to seek information and communicate through information technology (IT) to enhance clinical practice (Wanless 2002). The ability to use IT effectively is now an important skill for health professional graduates and the pedagogy of health professional education has been adapted to include e-learning (Glen and Moule 2006). Curriculum change and development is linked closely to other institutional issues such as retention and progression and employability as well as a recognition that student needs and expectations are changing. Figure 1 demonstrates the relationship between the 37

Hamshire et al.

curriculum development processes and these issues and emphasises the need to consult with our students about their needs as learners. Figure 1 Curriculum change for learning

Combining traditional face-to-face learning and teaching practice with the use of information and communications technologies such as those supported by most university MLEs is often referred to as blended learning (JISC 2004). In the department of Physiotherapy the move to a blended provision was seen as a way of responding to the change agenda outlined at MMU and in September 2006 the use of WebCT (the main component of the MMU MLE) was incorporated into the learning and teaching of all level 1 units on the BSc (Hons) Physiotherapy programme. For each unit (6 in total) the WebCT provision included a unit introduction page, discussion boards for asynchronous communication, chat rooms for synchronous communication, quizzes to facilitate formative feedback and interactive learning materials divided into weekly study tasks. This study aimed to investigate our students perceptions of WebCT as a learning environment, the barriers and facilitators to their use of WebCT and any impacts that these had on their learning.

Methodology
As we recognise that only the students themselves can articulate the learner experience (JISC 2007), listening to the student voice was central to this study. The methodology was designed to explore students opinions and beliefs and focus on their experiences of working with WebCT, described in their own words. We conducted a sequential exploratory, mixed methods evaluation (Creswell 2003). This involved three basic stages (Figure 2). In stage 1 the emphasis was on initial small scale, detailed, qualitative data collection and analysis targeted at key groups of students. This was used to identify key themes which, in stage 2, informed the development of a larger scale, more quantitative data collection and analysis targeted at the whole cohort of students. Stage 3 required an overall interpretation of the entire analysis. 38

Autonomy, Motivation and I.T. Skills

Figure 2 Methodology

Data Collection and Analysis

Stage 1: Individuals were selected for one-to-one interviews based on WebCT student tracking data from the core unit (Physiotherapy Management 1) between 23 October and 20 November 2006. A purposeful sample of eight students, three high users (over ten logins with at least 1 hour active user time), three low users (1-3 logins) and two nonusers were selected and interviewed. The partially-structured interview schedule began with broad questions, exploring the students use of technology and their perceptions of the programme as a whole and then focussed on specific WebCT issues in a funnel interview (Tashakkori and Teddlie 1998). A thematic analysis based on an analytical framework approach (Ritchie and Spencer 1994) was carried out to identify key themes in the students experience of working with WebCT. Stage 2: The themes derived from stage 1 were used to develop a questionnaire that was sent to the whole cohort (n=120). Eighty-seven questionnaires from the one hundred and twenty sent out were returned (73% response rate). The questionnaire was designed to substantiate and provide clarification of the data gathered by the interviews. The emphasis was on quantitative data via closed question formats; however, open questions were also included providing a further qualitative component. Thematic analysis of the open questions and descriptive statistical analysis of the closed questionnaire questions was carried out. Stage 3: The overall data set comprising of the three elements, qualitative interviews, open qualitative questionnaire questions, and closed quantitative questionnaire questions is shown

39

Hamshire et al.

Figure 3. (The central triangle represents the overall interpretation of the entire analysis). Figure 3 Data collection and analysis

Results and discussion

Barriers and Facilitators to engagement with WebCT resources The key themes that emerged from the combined analysis were that: The availability of useful learning resources that are flexibly and easily accessible encourages WebCT usage by a diverse student population. Autonomy and motivation are a key influence on student usage levels. Lack of computer confidence and competence can be a barrier to use. Students value online resources more than the communication tools. Access issues, broken web links, poor navigation and the use of inappropriate software all discourage use. Some students have preferences for different learning media.

We have summarised and categorised these into extrinsic and intrinsic factors in Table 1. The extrinsic factors (external to the student) relate mainly the technology and technical infrastructure that supports it, while the intrinsic factors (internal to the students) are personal attributes of the student. Table 1 Barriers and facilitators to engagement

Extrinsic Factors
Admin issues 40 Access Software Broken links Technical issues

Intrinsic Factors
Autonomy Motivation IT confidence

Autonomy, Motivation and I.T. Skills

In all cases the factors can be both barriers and facilitators depending on the students experience. For example, if a student has been enrolled correctly in WebCT and can login consistently without problems this facilitates their engagement with WebCT. If on the other hand the student has had intermittent problems logging into WebCT this can become a significant barrier to ongoing engagement. Table 2 demonstrates the spectrum of student perspectives from the partially-structured interviews relating to the intrinsic engagement factors. With close analysis of these responses and those of the other interviewees it has become clear that intrinsic factors played a much more significant role in engagement with WebCT than extrinsic factors. Indeed intrinsic factors were found to have a direct influence on the impact on extrinsic factors for an individual student. From the interviews it was apparent that students with perceived high levels of autonomy and motivation and good IT skills were able to easily overcome extrinsic barriers to engagement. However, students who were perceived, through the interviews, to be less autonomous and motivated and/or were less confident IT users often found the extrinsic barriers they encountered insurmountable. Overall as with many other studies into the provision of online resources that supplement traditional teaching (e.g. Sharpe et al, 2006) the majority of the students perceived teaching provision in WebCT positively, and reported a positive user experience throughout the unit: 90% (n=78) reported that it was easy to learn how to use the system. 80% (n= 70) reported that it was easy to find their way around in WebCT. 93% (n= 81) reported that it easy to access WebCT. 97% (n= 84) reported that they liked being able to get lecture notes and access web links online. 71% (n= 62) reported that learning this way was convenient. 70% (n=60) reported that learning this way helped them to study. 79% (n= 69) reported that the course area was necessary for the unit.

This positive response was encouraging as the majority 86% (n=75) of the students were first time users of WebCT. In general students felt WebCT was necessary for the unit; found learning this way convenient; and liked being able to get lecture notes and access web links online at any time.

Autonomy in the Physiotherapy curriculum

Physiotherapists, as educators, place great value on students developing independent learning skills. Kell and Van Deursen (2003), in a study of physiotherapy students, suggested that there is an obligation to ensure that our graduates have a desire to be educationally self-directed. The Chartered Society of Physiotherapy (CSP) curriculum framework is in accord and states that Physiotherapy programmes should include opportunities that encourage students to learn independently (CSP 2002). 41

Hamshire et al.

Table 2 Spectrum of student perspectives

WebCT User Level High user

Autonomy
I think as a mature student and spending a bit of time working and doing other things my organisation is certainly better now than it would have been when I left school. It doesnt daunt me the fact that a lot of it is selfdirected. In fact I quite like that. At the beginning it was quite a shock to like do work and not necessarily be going over it and sometimes if you get a bit behind or something its hard to catch up because youve always got things for the next time. We were panicking like mad saying we want just one set textbook you know.

Motivation
I spend my Tuesdays and Thursdays, full day from nine to about half three then take a break and then from four until six just working through the study tasks that Ive got to do or looking what I need to be doing longer term with my assignments and things like that. Sometimes its hard when the studies are looking up things, you get home and you look in your study pack and youre like look in book such and such and youre like at home and you dont have the book and you think, thats it Im going to have to go back to it because, I mean, I cant do it and Ill have to catch up later. Its my fault I should look earlier to see what I have to do. Once Im really into something that Im interested in Ill read it all day but just procrastinating, and theres always something to be done other than study, like tidying your room or tidying the kitchen, or go out to town, which you dont have to do but I tend to do instead, that tends to get in the way. I dont know, I need to sort that out.

IT Skills
Id say Im very confident. I used to use the computer a lot with my work as well before I started here so yeah I know my way around a computer pretty well. I use it obviously for Uni, for research and for WebCT and obviously to keep up with emails, Internet shopping, general poking around like that. We never had to use the computer before really. So I always much preferred doing assignments and that kind of thing handwritten. I much preferred it, it flowed better and here its like you must do it by computer and like in the beginning I was all panicky about it. I did everything by hand and then typed it up at the end.

Low user

Non user

Ive found it quite hard to adapt to the way that everything is geared towards a DIY attitude. I lost it at times. Basically its self orientated learning where the student is left to do the work. It has its merits definitely and in one way Im being forced into this new way of learning, even though Im kind of being dragged kicking and screaming.

Yeah communication with friends at home as a data base for finding out things and as a way of getting information from sources that would want to get in contact with me, like bank or airline, the email for this interview, stuff like that. Just for dummies like me that wouldnt be really good with the computer if you could just make it (WebCT) easier to understand how to use.

The ethos of the BSc (Hons) Physiotherapy programme at MMU reflects this and has high aspirations in terms of autonomy from the start. Table contains excerpts from the level 1 programme handbook emphasising the need for students to take personal and professional responsibility in the context of their learning and to be highly reflective in their practice. High levels of autonomy are implicit within the programme from the start of level 1. 42

Autonomy, Motivation and I.T. Skills

Table 3 Autonomy in the Physiotherapy curriculum

Source Programme Design

Articulation
The students learning is set within a professional context and the curriculum is designed to enable students to increasingly take responsibility for their own professional development. Throughout the programme there will be a strong emphasis on learning through reflection. Students will be encouraged to reflect on their experiences through the use of reflective logs/diaries and reflective writing assignments. They will be able to communicate accurately and have the qualities needed for a position that requires them to exercise personal responsibility.

Programme Delivery

Level 1 Descriptor

Autonomy is however a complex issue. Ecclestone (2000) considers autonomy as having three different levels (Procedural, Personal and Critical) which reflect increasing student independence from the tutor (see Figure 4). Our own experience indicates that the solution to the development of autonomy in students, is not as simple as providing e-learning resources as part of blended learning provision. The stage 1 interviews suggest that there may be differences in the way that students experience and interact with online learning. One of the factors identified as influencing this interaction was the apparent degree of readiness that individual students showed for self-directed learning. Although the use of online resources within a programme may encourage self-direction in some students, assumptions cannot be made that this is the case for everyone. The degree of control that students want to take over their learning process depends upon their individual personality, ability and attitude (Fisher et al 2001) and also the stage that they are at within their degree programme. Our challenge in terms of autonomy is to help our students develop skills that move them through procedural levels of autonomy up to critical levels of autonomy. Figure 4It is clear from Table 3 that the programme documentation is aimed at levels of autonomy at the higher critical level of the autonomy spectrum. However, if we look at Table 2 it is clear that some of our students express their autonomy at the lower procedural level i.e. Ive found it quite hard to adapt to the way that everything is geared towards a DIY attitude. I lost it at times. This mismatch presents us with a challenge in terms of curriculum and programme design. One of the rationales for the use of e-learning resources is that it can encourage active, self-directed learning (Glen 2005 and Santy and Smith 2007). Peacock & Hooper (2007) have linked the use of online resources to the promotion of independent, active learning within a number of studies and McKimm et al (2003) concur with this view. Our own experience indicates that the solution to the development of autonomy in students, is not as simple as providing e-learning resources as part of blended learning provision. The stage 1 interviews suggest that there may be differences in the way that 43

Hamshire et al.

students experience and interact with online learning. One of the factors identified as influencing this interaction was the apparent degree of readiness that individual students showed for self-directed learning. Although the use of online resources within a programme may encourage self-direction in some students, assumptions cannot be made that this is the case for everyone. The degree of control that students want to take over their learning process depends upon their individual personality, ability and attitude (Fisher et al 2001) and also the stage that they are at within their degree programme. Our challenge in terms of autonomy is to help our students develop skills that move them through procedural levels of autonomy up to critical levels of autonomy. Figure 4 Levels of autonomy

Linking autonomy, motivation and skills in curriculum development

Our research leads us to believe that the development of autonomous, independent, selfdirected learners, although highly complex, is something which can be enhanced by targeting key areas of the curriculum. We have summarised our thoughts in Figure 5. It has become apparent to us that the extent to which a student is an autonomous, independent, self-directed learner is a function of their level of autonomy and their level of intrinsic motivation (want or need to learn). Although closely linked, autonomy and motivation are not synonymous. For example a student who possesses critical level autonomy may not be motivated by tasks that are seen as not relevant directly to their studies. At the same time a highly motivated student may simply not have the experience or required skill set to enable autonomous learning. In either case the curriculum needs to be designed to provide appropriate scaffolding to enable students to develop higher levels of autonomy and provide the necessary motivational stimuli. 44

Autonomy, Motivation and I.T. Skills

In terms of autonomy the curriculum can target the students learning, academic and IT skills providing them with an understanding of their own learning needs and a skill set that empowers them as individual learners and within a community of practice in physiotherapy. By developing a curriculum that is relevant to physiotherapy students in the work place, that sets out and communicates a clear purpose for the constituent parts of the programme and that involves students broadly in all aspects of their learning, including assessment, we can provide motivation for all of our students.

Figure 5 Linking autonomy, motivation and skills in the curriculum

We also see a clear role for the institutions MLE (including WebCT VISTA) in our future curriculum developments. As reported by others (e.g. McKimm et al (2003)) we feel that independent and active learning can be encouraged through web based programmes where this is embedded thoroughly in the curriculum design and that by utilising integrated, interactive course materials, educators can improve learning and make that learning more enjoyable and meaningful for learners.

Application of our findings to the level 1 induction programme

In order to ensure these findings have an impact on the curriculum, consideration of how induction can improve the use of online resources by all students was necessary this has lead to reflection on the re-design of induction. A key element of such a re-design will be to embed the materials and resources provided via WebCT into the students working practice before they arrive, during their critical first week at university and in ongoing support throughout level 1. This phased induction will be based around small group activities, linked and integrated with the programme and level one unit learning outcomes. We will provide students with flexible access (via WebCT VISTA) to a broad range of resources relating to programme administration, campus orientation and social 45

Hamshire et al.

aspects, as well as academic skills learning materials. This induction will help students manage their transition to Higher Education, reduce anxiety, provide a focus for skills development and enable them to communicate easily with their tutors and support staff. Such a framework, we believe, will build a foundation for the development of student autonomy. The core skills element will be tied closely to in-class activities, particularly those related to assessment. The purpose of this is to increase the relevance of the core skills resources to specific learning activities and provide just enough information in just enough time (JEIJET) to support the students in their work. As students become more familiar with these resources, in appropriate contexts, we anticipate that we will foster ongoing independent use of the resources. One of the most important roles will be that of a designated key contact from the teaching team. They will be responsible for bridging the gap between what takes place in the classrooms and lecture theatres and the resources available to support those activities in WebCT. In simple terms this will mean the key contact facilitating the JEIJET principles of easy start by dropping into teaching sessions and directing students to supporting resources at the times when they are most useful. The role of the key contact is, to some extent, to emphasise the relevance of key resources to inexperienced learners and provide the motivational stimuli for their use. We anticipate that linking assessment activities to the core skills resources available within WebCT will be a major benefit in this respect. Funding has been obtained from the HEA Subject Centre for Health Sciences and Practice to develop such an induction programme. The project will develop and deliver a blended learning induction programme at level 1 that targets key learning skills in Physiotherapy.

Conclusions
Our work has highlighted to us the importance of detailed consultation with students in the planning and design of blended learning resources. We also recognise that there is to some extent a mismatch in the programme aspirations and some of students self perceptions in terms of autonomy. As our programme recruits a diverse student body, their readiness and levels of motivation for learning in HE vary greatly depending on their prior learning experience. This has an impact on their levels of autonomy. The challenge is to embed scaffolding within curriculum design to enable all students to develop appropriate independent learning skills as they progress through the whole programme.

46

Autonomy, Motivation and I.T. Skills

References
Brooks, J. (2005a) Vice-Chancellor's Discussion Papers I-II (Academic Direction & University Development), Manchester Metropolitan University. Brooks, J. (2005b) Vice-Chancellor's Discussion Papers III-IV (The Shape and Function of the Faculties & Roles and Responsibilities of Directorate), Manchester Metropolitan University. Creswell, J.W. (2003), Research design: qualitative, quantitative, and mixed method approaches. 2nd ed. California: Sage publications. Ecclestone, K. (2000) Assessment and Critical Autonomy in Post Compulsory Education in the UK, in, Journal of Education and Work, Vol. 13, No. 2. Fisher, M., King, J.and Tague, G. (2001) Development of a self-directed learning readiness scale for nursing education. Nurse Education Today. Vol 21 pp 516-525. Glen, S. (2005) E-learning in nursing education: lessons learnt? Nurse Education Today Vol, 25 Issue 6. pp 415-417. Glen, S. and Cox, H. (2006) E-learning in nursing: The context. in Glen, S.and Moule, P. E-learning in nursing. Hampshire, Palgrave Macmillan, Chapter 1. Glen, S and Moule, P. (2006) Preface. in Glen, S. and Moule, P. E-learning in nursing. Hampshire, Palgrave Macmillan, Preface. Joint Information Systems Committee (JISC) (2007) In their own words. Exploring the learners perspective on e-learning. Online Available HTTP: <www.jisc.ac.uk> (Accessed 1 October 2006). Joint Information Systems Committee (JISC) (2004) Effective practice with e-learning. Online. Available HTTP:<www.jisc.ac.uk> (Accessed 17 April 2006). Kell, C. and van Deursen, R. (2003) Does a problem-solving based curriculum develop life-long learning skills in undergraduate students? Physiotherapy. Vol 89, No.9, pp 52330. McAtominey, D. and Cullen, W.R. (2002) Effective e-Learning with VLEs, Netskills Workshop Materials. Online. Available HTTP:<http://www.netskills.ac.uk/> (Accessed 3 March 2007). McKimm, C., Jollie, C. and Cantillon, P. (2003) ABC of learning and teaching, web based learning. BMJ Vol 326, pp 870-873. Peacock, S. and Hooper, J. (2007) E-learning in physiotherapy. Physiotherapy Vol 93, No 3, pp 218-228.

47

Hamshire et al.

Ritchie, J. and Spencer, L. (1994) Qualitative data analysis for applied policy research in Bryman, A. and Burgess, R.G. (eds.) Analyzing qualitative data. London: Routledge, Chapter 9. Sharpe, R., Benfield, G., Roberts, G. and Francis, R. (2006) The undergraduate experience of blended e-learning: a review of UK literature and practice. The Higher Education Academy. (Online) Available HTTP:<www.heaacademy.ac.uk> (Accessed 7 August 2007). Tashakkori, A. and Teddlie, C. (1998) Mixed methodology: combining the qualitative and quantitative approaches (Applied social research methods, No. 46). California: Sage publications. The Chartered Society of Physiotherapy (CSP) (2002) Curriculum framework for qualifying programmes in physiotherapy. London: The Chartered Society of Physiotherapy. Wanless, D. (2002) Securing our future health: taking a long term view, Final Report, H M Treasury. London. Online Avaliable HTTP:<http://www.hmtreasury.gov.uk/Consultations_and_Legislation/wanless/consult_wanless _final.cfm> (Accessed 25 April 2006).

48

PRIME Volume 3(2)

Self-Assessment Dialogue: added value? The Student Perspective

Sara Eastburn University of Huddersfield, School of Human and Health Sciences

Introduction
Brown (1999) argues that assessment is integral to the learning process of an individual, and that assessment procedures must be well informed and fit-for-purpose (p. 4). These values are fundamental to the assessment, learning and teaching strategies within global higher education along with recognition that assessment is one of the key drivers to optimise student learning (Race, 2005). In addition, involving students in the assessment process can deepen their learning experience and develop a greater reflective insight (Race, 2001). Self-assessment is described by Race (2005) as a means by which students can both better prepare for assessment and better demonstrate their learning (p. 94). Reflective practice is a necessary skill of all graduates; fundamental to this is self-assessment. Within graduate education per se awareness of ones own learning, in terms of both achievements and ongoing needs, is fundamental to 1) working autonomously, 2) lifelong learning and 3) collaborative working. Self-assessment is suggested by Taras (2001) as a means by which confidence and independence may be fostered. A self-assessment dialogue document (SADD) is a means by which a student is encouraged to reflect on their learning from undertaking a piece of work at the point of assessment. This not only fosters breadth and depth of reflection, but allows dialogue between the student and assessor (tutor or peer) that feeds forward either summatively, formatively (described by Irons (2008) as a powerful and constructive learning tool) or a combination of both into the students learning continuum. In addition, a dialogue-approach may allow an opportunity for the student to clarify and verify with the tutor what is being said (Nicol and Macfarlane-Dick, 2006) crucial if learning is to be maximised as students often do not understand the feedback being given to them. Tutor feedback is viewed as a crucial component to student learning and there is clear evidence that self-assessment and tutor discussion enhances student learning in comparison to self assessment alone (Taras, 2001). There is some debate around the relationship between learning opportunity and summative grading (Taras, 2001; 2002), but even if awarded a summative mark, the formative feedback from the student-assessor dialogue enriches the students learning experience (Irons, 2008). Ultimately such activity may enhance student retention and develop skills of reflection and criticality.

Aims
The overall aim of this research was to establish the perceived value of a SADD to assessment, learning and teaching strategies from both the student and tutor perspective within an undergraduate module. 49

Eastburn

This paper will present the undergraduate students perceived value of a SADD to their assessment and learning strategies. Research Question What is the added value of a SADD to a students assessment and learning experience?

Methodology
A 2-phase questionnaire approach was used to collect data from students. A core module within a pre-registration healthcare programme at a large British university provided the vehicle for this research and the sample consisted of all students enrolled on this module during 2005-2006 (n=36). The module was summatively assessed by a synoptic statement based around a placement learning journey which summarised: 1) an analysis of how the learning originated, 2) a key learning need relevant to clinical practice (including an annotated bibliography clearly linked to the learning need), and 3) a focussed action plan for the future. In addition, students were asked to complete a SADD at the point of assessment which should be reflective, insightful, specific and complete. A summative mark of 5% was allocated to the SADD with the remaining 95% of marks being afforded to remaining assessment components.

Data Collection and Analysis

Phase 1 of the questionnaire was administered after submission of the summative assignment, including the SADD, and phase 2 was administered after the assignment, including the SADD, had been graded and returned to the student. Preceding the distribution of both questionnaires students were given an information sheet which explained the aim of the research, that their involvement was voluntary and independent from the module itself, that responses were anonymous, and that their decision to participate in the research would not influence the feedback or grade that they received for the assignment. Additionally, the information sheet explained that return of completed questionnaires implied consent to use the data, that they could withdraw their data at any time without reason, and that their data may be used for learning, teaching, assessment and research purposes. The questionnaires were distributed by the researcher on both occasions and all students were asked to return their questionnaire (completed or blank) to a single box. The researcher remained available to answer questions during the data collection process. Data were analysed using SPSS 12.0.1 and qualitative content analyis. Ethical approval was granted by the host university.

Results and Discussion

69% (n=25) of students completed phase 1 of the questionnaire, and 72% (n=26) of students completed phase 2. 50% (n=2) of tutor participants gave consent and were available to take part in the focus group. Table 1 shows that 92% (n=23) of students enjoyed the module but only 16% (n=4) enjoyed the assessment. Statistical tests show no association between the ways in which students responded to the two questions. 50

Self Assessment Dialogue

Table 1: Enjoyment of the module and enjoyment of the assessment?

Did you enjoy the assessment? Yes Did you enjoy the module? Total Yes No 4 0 4 No 17 2 19 Don't know 2 0 2 Total 23 2 25

Pearson Chi-Square p=0.709

Only 8% (n=2) of students indicated that the assessment related activity they enjoyed the most was completing the SADD, whilst 24% (n=6) of students indicated that this was the assessment related activity that they least enjoyed. Table 2 shows additional qualitative comments. Table 2: Comments on completing the SADD
I Enjoyed Completing the SADD the Most I Enjoyed Completing the SADD the Least

It was the easiest and least complicated part. Allowed me to reflect It couldnt be wrong and was the quickest to do

After completing essay had this to do as well. Time consuming and more made up than reflective Difficult to critique yourself Didnt think it gave me any particular benefit I find it difficult to rate my level of success Bit pointless. Didnt add anything to learning made things up to fill up the boxes The SADD was long winded, a lot of questions to answer

The results in Table 3 suggest that students who perceived the SADD to help them view their work differently also felt that it helped them view it more critically. This notion is widely supported in the literature. Figure 1 illustrates some of the qualitative comments from students in relation to the differently and more critically questions. Table 3: Did the SADD make you look at your work differently and more critically?
Did the SADD make you look at your work more critically? Yes Did the SADD make you look at your work differently? Total Yes No Don't know 6 2 3 11 No 0 9 1 10 Don't know 2 1 1 4 Total 8 12 5 25

Pearson Chi-Square p=0.015

51

Eastburn

Of particular interest are the 2 comments (**) that very clearly echo the students summative-driven rather than learning-driven vision of the assessment process. Figure 1: Qualitative comments on Differently and More Critically

have not looked at the piece of work again I completed the assignment and then completed SADD had no bearing on the assignment ** Dont think would have thought about those sort of questions if werent getting marked on it It was just an extra piece of work I tacked onto the end

It made me think about what I had done more Allowed me to reflect on what is good and not so good about my essay Partly forced me to re-read essay Good for practice on self assessment Perhaps, it helped me to evaluate my work

Differently

Searching through document for something I could put in could have spend time actually changing the work rather than saying if I could do it again More Critically Quickly done 5 minutes before handed in ** I just gave the answers that I thought would get me the best marks It was the last thing I did as an after thought

Good practice for future pieces of work I could decide which bits I had done well and done not so well Made you realise what you had done well and what was not so good Highlighted areas I need to improve upon Made me read the essay again and think about good and bad points

Students were asked in what ways they thought completing the SADD had made a difference to their university-based study or clinical practice. Figure 2 highlights the comments, the majority of which were positive. When asked whether the SADD added value to their overall learning and assessment experience 48% (n=12) of students said no, 32% (n=8) said yes, and the remainder did not know. Qualitative comments to this question included I think it was a waste of time which could be spent more wisely on actually doing the work which concurs with the notion that students perceive assessment as verification of learning rather than a learning 52

Self Assessment Dialogue

opportunity in itself and it was just made up because it had to be handed in to get 5% extra which presents an interesting view of the SADD being extraneous to the assessment task (in spite of it being very clearly part of the summative assessment task itself). Figure 2: Qualitative comments from students Negative Connotation Made it more time consuming It hasnt changed my thinking yet (my emphasis) Positive Connotation Reflect to a higher level Gives confidence about self critiquing Develops own reflection skills Taught me that I need to reflect more Encouraged me to follow through learning needs Gave me more autonomy within my learning Allowed me to reflect on the essay Know good and bad points about me for next essay Identify learning Increased reflection Encouraged me to reflect more Made me re-read my work Practice of self assessment Improvement of critical thinking Allowed me to be more self critical I can also see which areas I am better at

More positive responses included it allowed me to be more self critical in looking at work and identify mistakes and it depends if it helps for future assessments the former of which shows a reflective insight into the work and the latter implies transferability between different pieces of assessment. It was considered that the students who indicated that they had enjoyed the module might also score the other questions more favourably. However, the Mann-Whitney U test showed no significant difference between the summed score of the other questions, nor the individual responses to the other questions, when compared with whether the students had enjoyed the module or not. However a high level of significance (p=0.012) was seen by the Mann-Whitney U test in the difference between responses to the remaining summed questions and whether the students enjoyed the assessment or not (Figure 3) suggesting that students who enjoyed the assessment were more likely to give a favourable response to other questions.

53

Eastburn

Figure 3: Box and whisker plot showing the summed score of the remaining questions and enjoyment of assessment
10

No

Total Scores sum

Yes
5

Did enjoy the assessment? Did youyou enjoy theassessment?

In addition, the Mann-Whitney U test showed a significant difference (p=0.043) between whether students enjoyed the assessment and whether they felt that the SADD added value to their overall learning and assessment experience. Table 4: Completion of the SADD and tutor feedback
Did you find the feedback from the tutor on the SADD helpful? Yes Do you think you completed the SADD to best of your ability? Total Yes No Don't know 12 5 1 18 No 0 1 1 2 Don't know 2 0 0 2 Total 14 6 2 22

Table 4 shows that 46% (n=12) of students indicated that they both completed the SADD to the best of their ability and received tutor feedback, and found the tutor feedback helpful. Additionally, 21% (n=5) of students felt that they had not completed the SADD to the best of their ability but they felt that the tutor feedback was helpful to them. Overall, 82% of students who received tutor feedback on the SADD found the feedback helpful. Unfortunately, students were not asked to elaborate on what aspect of the tutor feedback was most helpful. (NB. 4 students who indicated that they completed the SADD to the best of their ability did not receive tutor feedback). A diverse range of comments in relation to whether the students had completed the SADD to the best of their ability included I took my time in completing it and attempted to be as critical and reflective as possible. My mark reflected the effort I had put in, [I] maybe wasnt as truthful as I could have been looking back and it did not seem relevant. The impression was that you simply got the marks for handing it in. 54

Self Assessment Dialogue

Students were asked to identify ways in which the tutor feedback was helpful to them, comments included the highlighting of mistakes and suggestions for improvement, confirmation of self-believed strengths and weaknesses, provision of encouragement and increased confidence. Students were also asked to identify ways in which the tutor feedback could have been better. Two comments were could have expanded a little more to my feedback to help me understand my mistakes and more in depth on areas to improve corroborating Irons (2008) who reports that students need greater, more specific and detailed feedback. Table 5: Actual and potential demonstration of reflection through the SADD
Do you think the SADD is a useful tool through which to demonstrate reflection? Yes Do you think you demonstrated through completing the SADD your ability to reflect on a piece of work? No Total

Yes

21

21

No Missing data Total Pearson Chi-Square p=0.007

2 1 24

1 1 2

3 2 26

Table 5 shows a highly significant association between actual reflection and potential reflection using the SADD. 100% (n=21) of students who felt that they had demonstrated their ability to reflect via the SADD felt that it was a useful tool by which to do this and, of the 3 students who indicated that they felt that they did not demonstrate reflection well through completing the SADD, 67% (n=2) felt that the SADD was a useful tool through which this could be done. Thus 92% (n=24) of all students felt that the SADD was a useful tool with which to demonstrate reflection. Table 6: Identification of and planning strategies by which to meet learning needs

Do you think the SADD is a useful tool through which to plan strategies to meet your ongoing learning needs? Yes Do you think the SADD is a useful tool through which to identify your ongoing learning needs? Yes 14 No 6 Total 20

No Total Pearson Chi-Square p=0.003

0 14

6 12

6 26

77% (n=20) of students (Table 6) thought that the SADD was a useful tool through which to identify their ongoing learning needs, of which 70% (n=14) also thought it was an 55

Eastburn

effective way to plan strategies to address those needs. Thus a high level of significance was seen between students who felt that the SADD was useful to identify ongoing learning needs and also strategies to meet the needs. Table 7: Planning strategies by which to meet learning needs and timely tutor feedback
Do you think the SADD is a useful tool through which to gain tutor feedback in a timely manner? Yes Do you think the SADD is a useful tool through which to plan strategies to meet your ongoing learning needs? No Total

Yes

11

14

No Total

8 19

3 6

11 25

Of the 56% (n=14) of students who felt that the SADD was a useful tool through which to plan strategies to meet ongoing learning needs, 79% (n=11) also thought that it was a useful tool through which to gain timely tutor feedback (Table 7). There is a statistically significant positive association (Table 8) between the students perception of the SADD being a useful tool to better prepare for assessment and plan strategies to meet ongoing learning needs. Table 8: Preparation for assessment and planning strategies
Do you think the SADD is a useful tool through which to plan strategies by which to meet your ongoing learning needs? Yes Do you think the SADD is a useful tool through which to be better prepared for assessment? No Total

Yes

11

15

No Total Pearson Chi-Square p=0.020

3 14

8 12

11 26

Interestingly, 32% (n=8) of students deemed the SADD useful for gaining timely tutor feedback but not to plan strategies to meet ongoing learning needs (Table 7). Students were not asked to elaborate further about the skills they felt necessary to plan strategies to address their learning needs. This is an area for further investigation.

56

Self Assessment Dialogue

These results imply that planning for and resolving of learning needs feeds forward into the assessment process such that students are better prepared, and that students do not see pieces of assessment isolation.

Conclusion
This research has shown that the SADD is a useful tool by which students may demonstrate refection. Additionally, the majority of students (77%) felt the SADD was a useful tool to identify ongoing learning needs and 70% of those felt that it was also a useful tool to plan strategies to address those needs. Thus, the remit of the SADD could be wider than pure reflection and it may practically support the ongoing learning process. More than 80% of students who received tutor feedback on the SADD found the feedback helpful and beneficial to their learning experience but this research has shown that it is essential that the feedback given to students is done so in a language that they understand if that feedback is to be useful (supported by Nicol and Macfarlane-Dick, 2006 and Irons, 2008). It has also illuminated that students require specific and detailed feedback if their learning opportunities are to be maximised. Finally, this research has shown that the investment of time and effort from the student is essential if learning from using the SADD is to be optimal and that it is essential that using the SADD is viewed as part of, not outside of, the [summative] assessment task.

References
Brown, S. (1999) Institutional Strategies for Assessment in Brown and Glasner (Eds) Assessment Matters in Higher Education. Choosing and Using Diverse Approaches. Buckingham: Open University Press. Irons, A. (2008) Enhancing Learning through Formative Assessment and Feedback. Abingdon: Routledge. Nicol, D.J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31, (2), 199-218. Race, P. (2001) A Briefing on Self, Peer and Group Assessment. Learning and Teaching Support Network Generic Centre. York: LTSN. Race, P. (2005) Making Learning Happen. A Guide for Post-Compulsory Education. London: SAGE Publications Ltd. Taras, M. (2001) The Use of Tutor Feedback and Student Self-assessment in Summative Assessment Tasks: towards transparency for students and for tutors. Assessment & Evaluation in Higher Education, 26, (6), 605-614. Taras, M. (2002) Using Assessment for Learning and Learning from Assessment. Assessment & Evaluation in Higher Education, 27, (6), 501-510. 57

PRIME Volume 3(2)

Contemporary Art and the Level 1 Higher Education Curriculum: Empathy, alienation and educational inclusion.
Leigh-Anne Perryman The Open University, Centre for Research in Education and Educational Technology

Summary
Can a Tracey Emin bed or a Grayson Perry pot be a more productive object of study than a Raphael Madonna or a John Constable landscape for some Level 1 university students, allowing them to make meaningful connections between the artworks and their own lives? Eliot Eisner (1984; 1994; 1998; 2002), one of the most influential voices in art education, has long argued that studying visual art can help us to discover the contours of our emotional selves, enabling us to have experiences we can acquire from no other source. However, various art educators and writers (e.g. McFee, 1986; Lippard, 1990; Cahan & Kocur, 1996; Chalmers, 1996; Boughton & Mason, 1999) have observed that visual art education in the West is still dominated by a culturally exclusive canon of western artworks and that this limits the extent to which socially, culturally and ethnically diverse students can benefit from and engage with the study of art history, leaving them feeling alienated and disempowered. Calls to abandon the canon in the name of inclusion are often voiced with reference to school art education but are applied less frequently to a higher education context. This paper details one of the first phases of a PhD research project intended to address this imbalance by exploring whether including contemporary art in the Level 1 undergraduate curriculum has the potential to reduce the barriers to learning faced by the ever-more diverse range of students entering higher education in the 21st century. An online questionnaire was used to survey 420 undergraduate students about their experiences of studying contemporary art in a short Level 1 Open University course. Early research findings have implications beyond the discipline of art education, indicating that while the western canon may indeed have the power to exclude on race, socio-economic, gender and age-related grounds, just replacing canonical curriculum content with a different kind of visual art (for example contemporary art) is not a one size fits all solution to minimising educational exclusion. Significantly, it appears that there is an age-related divide in adult students feelings about contemporary art, in that while younger students can relish its challenging form and content, finding the subject matter relevant to their own lives and enjoying the emotional demands of studying some of the most controversial artworks, some older students preconceptions about contemporary arts lack of worth prevents them from any productive engagement with it. However, the research findings also indicate that it is possible such preconceptions can be a starting point for a meaningful engagement with contemporary art when explored and addressed through a pedagogy featuring meta-cognitive strategies and reflective writing, offering students a framework within which to locate and make sense of their reactions to shocking and controversial contemporary art and the skills to work with the multiple interpretations and open-ended meanings it commonly involves.

59

Perryman

Introduction and Rationale

This makes me very angry. Shes having a laugh at our expense. How can you compare this with something by Constable or Raphael? I didnt sign up for this course to be taught about this sort of rubbish. Ralph, aged 68, Derbyshire I absolutely love it This really made me think about how subjects from everyday life, from my world, could be expressed through art, could be worthy of being art. Traceys bed is like my life.messy! Jo, aged 24, Essex. The comments above refer to Tracey Emins installation My Bed (http://www.saatchigallery.co.uk/artists/artpages/tracey_emin_my_bed.htm) and were made by two undergraduates studying the British Open Universitys Level 1 short course Making Sense of the Arts, which introduces students to art history via a study of artworks by artists who have been nominated for the Turner Prize (Awarded annually to an artist who has made an outstanding contribution to art in Britain in the last 12 months). Arguably, the differences of opinion shown in these comments highlight a dilemma increasingly facing curriculum designers and teachers within all areas of higher education how to reconcile the interests, motivations, values and needs of the ever-more diverse range of students entering our universities, in the interests of educational inclusion. This need is widely voiced as a priority area for curriculum development. Melanie Nind (Nind, 2005: 5), for example, identifies the need for the curriculum to make connections with learners perspectives to start from, and value, what learners bring, while the author and social activist Bell Hooks (1994: 8) has long asserted that an engaged pedagogy, in which everyones presence is acknowledged is essential to generating the excitement needed for meaningful and effective learning. Much inclusion-related research has been carried out in the name of multicultural education, focusing on ethnic diversity, for example Geneva Gays extensive exploration of ways in which ethnically diverse students success might be improved through culturally responsive teaching' (Gay, 2000). Some of this research into multicultural education has focused on the visual arts, with various art educators and writers (e.g. McFee, 1986; Lippard, 1990; Cahan & Kocur, 1996; Chalmers, 1996; Boughton & Mason, 1999) observing that visual art education in the west is still dominated by a culturally exclusive canon of western artworks and suggesting that this is compromising educational inclusion by denying diverse students the chance to benefit from studying the visual arts. However, empirical studies in this area tend to be located within primary and secondary school settings (e.g. Hooks, 1995; Dash, 1999; Young, 1999; Freedman, 2000; Knight, 2006), with relatively little written about ethnic exclusion in higher education art study. There is even less research into other types of exclusion in undergraduate art education such as that resulting from barriers to learning connected with students sexuality, socioeconomic background and age. Arguably though, the dilemma of how to reconcile diverse learners needs is particularly pertinent within higher education, and in emotionally-rich subjects like the visual arts, where adult learners values, expectations and existing knowledge can collide with the artworks that they study, sometimes resulting 60

Empathy, Alienation and Educational Inclusion

in intense feelings of empathy and at other times in feelings of alienation and disempowerment, as the comments above suggest. The project discussed in this paper addresses issues that are common to all humanities disciplines where a canon of preferred works tends to dominate the curriculum. It is intended to contribute to pedagogical research in inclusive education by exploring whether including western and non-western contemporary art in the Level 1 undergraduate curriculum (rather than the more commonly found western canonical art) might help to reconcile diverse students needs through a pedagogy of recognition (Slee, 1999: 200), resulting in a curriculum within which students can recognise their own experiences and identities (Nind, 2005: 5), while also assessing the extent to which Mayers (2008: 77) assertion that: Contemporary art is about now! Its about figuring out who we are, who we are becoming, and how to live, know and actWhat could be more relevant? applies to adult learners.

Methodology
Research design The research project discussed here comprises a single instrumental case study (Stake, 2005) the Open University Level 1 short course Making Sense of the Arts, which introduces students to the humanities through the study of poetry, history and art history. The course has no entry requirements, is designed for students who have not studied before or who may have studied a long time ago, and lasts for a maximum of 20 weeks. Students have an allocated tutor and all tuition is conducted by telephone and through written feedback on two formative assignments. (An optional online forum also offers students the opportunity to discuss the course with their peers.) Upon successfully completing a summative End of Course Assessment (ECA) students gain 10 credit points at Level 1. The art history section of Making Sense of the Arts the main focus of this research project - is unusual in including contemporary art (art produced after 1980 by Turner Prize nominees) in a Level 1 course. The mixed model approach The Making Sense of the Arts case study is located within a sequential (Creswell & Plano Clark, 2007 ) mixed model (Tashakkori & Teddlie, 1998) research design in which both quantitative and qualitative research questions, data collection techniques and analysis techniques have equivalent status (Tashakkori & Teddlie, 1998: 15) throughout the research process (see Appendix 1). While mixed methods research designs have plentiful detractors (e.g. Denzin & Lincoln, 2005: 9-10) they are becoming ever more popular, with numerous advantages being identified (see Greene, Caracelli et al., 1989; Tashakkori & Teddlie, 1998; Bryman, 2006; Greene, 2008). In the context of this study, the choice of a mixed model strategy was intended to achieve a more rigorous, more comprehensive and more complete understanding of adult students experiences of art education than would be possible using qualitative or quantitative methods alone.

61

Perryman

The survey The findings reported in this paper are the result of one of the earliest phases of the overall research project - an online questionnaire comprising four sections: Section 1: Various questions gathering information about students age, gender, disabilities (if any), ethnicity, previous educational qualifications (if any), and previous art study (if any). Section 2: Likert-scale and open-ended questions intended to gain information about students reasons for studying Making Sense of the Arts, the outcomes they hoped to achieve, the extent to which they felt they had achieved these outcomes, and any factors negatively impacting on their study experience. Section 3: (i) Open-ended questions intended to gain information about students initial feelings about the prospect of studying contemporary art in Making Sense of the Arts and the extent to which those feelings changed after studying the art history section of the course; (ii) Likert scale-style questions intended to gain information about students feelings about the effectiveness of the various components of the art history section of Making Sense of the Arts Section 4: Semantic differential scales intended to gained information about students reactions to 17 contemporary and non-contemporary, canonical and non-canonical artworks (see Appendix 2), some of which also featured in Making Sense of the Arts. The online survey format was chosen for both economic and logistical reasons. Firstly, the cost of printing over 800 copies of a fairly lengthy full-colour questionnaire appeared prohibitive and the online version offered accessibility advantages in allowing participants to magnify the images featured in the survey. It also offered reactive routing options, allowing students to skip certain sections of the survey if appropriate for example, if a student indicated that they had completed the course but had not studied the art history section (The Making Sense of the Arts assessment strategy allows this, as the discipline of art history is not a compulsory option in the only summative assignment in the course) they were automatically taken from Section 2 to Section 4 of the questionnaire. Furthermore, with a large sample the time advantages of using an online questionnaire rather than a postal one were very attractive, the data being immediately available for analysis. All students in the November 2007 cohort (n=420) were contacted by email in the first instance and invited to complete the questionnaire (The research is currently being repeated with a second cohort of students (n=440), in advance of conducting semistructured telephone interviews with 20 students chosen from both cohorts). It was hoped that the use of personalised email invitations would minimise the problems of identity verification and spoof respondents that are often identified as undermining the validity of self-selecting online samples (see Roberts & Parks, 2001; Hewson, Yule et al., 2003: 44). Internet access is not compulsory for Making Sense of the Arts students and recent research (UK Online Centres, 2007) indicates that there is still a digital divide in the UK, with 75% of people counted as socially excluded also being digitally excluded. For this reason, students who had not indicated a preferred email address (and had been 62

Empathy, Alienation and Educational Inclusion

emailed via their default OU address) (n=67) were also posted a paper version of the questionnaire two weeks after the initial email had been sent (As the questionnaire responses were anonymous it was impossible to check whether these students had already completed the survey). While this procedure introduces validity-related issues concerning the mode effects resulting from the use of different administration methods (see de Vaus, 2007: 131) it was hoped that these would be outweighed by the benefits gained from avoiding sample bias. More importantly, it was considered ethically indefensible to exclude students with no Internet access from participating in a research project intended to address educational inclusion. The decision to use semantic differential scale questions to collect data about students responses to a range of artworks was inspired by recent use of the scales within the field of empirical aesthetics (e.g. Martindale, Moore et al., 1990; Locher, Smith et al., 2001; Silvia, 2005; 2006; Silvia & Brown, 2007; Tan & Tollenaar, 2007) where they are commonly used to explore the link between personality traits, previous experience of art study and artistic preference. (For a summary of such studies, see Furnham & Walker, 2001: 998-999; Martindale, 2007.) This research has relevance in the context of educational inclusion, highlighting the possibility that barriers to learning associated with the study of visual art may be closely connected with personality differences in addition to differences in students preferred learning styles - a common focus of educational research. It will be further explored in the next phase of this research project. Originally devised by Osgood, Suci and Tannenbaum (1957) the semantic differential scale is a tool which measures peoples affective reactions or attitudes to stimulus words, concepts or images in terms of ratings on bipolar scales defined with contrasting adjectives at each end (e.g. happy-sad or simple-complex). Typically, seven point scales are used (although some studies have used five and six-point scales). The present study uses a five point scale chosen to reduce the time burden on respondents and to encourage spontaneity of response. The adjectives for this scale (see Figure 1) were selected via a multi-stage process. First, a focus group comprising 5 non-art trained and 5 art-trained participants were asked to freely discuss the 17 artworks featuring in the questionnaire. A prototype scale was then designed, featuring the 20 most popular adjectives/adjectival phrases relating to the three basic dimensions of response identified by Osgood (1975) as being applicable to any concept or stimulus namely Evaluation (e.g. worth studying-not worth studying), Potency (e.g. weak-powerful) and Activity (e.g. complex-simple). Adjectives relating to a fourth dimension Berlynes (1974) Internal State dimension (e.g. comfortingdisturbing) - were also chosen. A selection of researcher-chosen antonyms were then added to the scale which was formatted so that negative and positive polarity was randomised for left-right position, thereby avoiding the possibility of biasing respondents opinions by presenting potentially ambiguous (i.e. neither negative or positive) adjectives on a side of the scale that is clearly intended to represent one polarity. The scale was then piloted with 50 Open University humanities students before its use in the current survey.

63

Perryman

Figure 2: Semantic differential scale for Raphaels Madonna of the Meadows

Findings and Implications

Analysis of the survey data is still in its early stages, with data collection for the second cohort of Making Sense of the Arts students still ongoing. However, the initial findings from the first cohort of students already appear significant. What follows is limited to discussion of open-ended answers and statistical frequencies; a fuller qualitative and quantitative analysis will take place once the data from the second cohort of students has been collected.

64

Empathy, Alienation and Educational Inclusion

A 69% survey response rate resulted in a final sample of 289 students, aged between 18 and 84, with a 71%/29% female/male gender balance (this is typical for OU arts courses). Students previous educational qualifications ranged from no qualifications (38%) to qualifications at postgraduate level (5%) and 17% of respondents had studied art or art history prior to studying the subject in Making Sense of the Arts. As previously mentioned, the course assessment strategy allows students to pass the course without studying the art history section and this was the case for 11% of students in this first cohort. Students were predominately White British and only 9% declared a disability. Initial feelings about studying contemporary art An initial exploration of the collected data clearly indicated that the majority of students surveyed had been concerned about the prospect of studying contemporary art. The following comments, taken from an open-ended question asking What were your initial feelings about studying contemporary art in Y160 are typical: Terror! I thought I would hate it. I did not believe I would enjoy it especially when I saw type of art I was to study. Quite apprehensive. It wasn't what attracted me to the course. I wouldn't have thought that my "modern art isn't really art" attitude would change much. Of all the art history to choose, the Turner Prize must be the worst possible topic. I just couldn't drum up any enthusiasm for it I was sad that it had to be 'contemporary art'. I began with the preconceived notion that modern art is meaningless. I felt it was an ordeal to be endured. I was disappointed that the course would not concentrate on older works. Initially I considered most contemporary art a 'scam' I was quite apprehensive. I had visited the Tate Modern various times and found some of the work very strange and hard to understand. I looked forward to beginning a completely new (for me) field of study and felt let down by the works we were expected to study. Trepidation - I would describe myself as more interested in Turner than the Turner Prize. FrustratedOld art is great artthats why theyre called the Old Masters. Theres no comparisonArtists like Constable were real artists and people like Damien Hirst are just conmen.

65

Perryman

One students comment that my only exposure to the Turner Prize was through the media so I approached this section of the course with great of trepidation and cynicism highlights one of the challenges of teaching with contemporary art the frequent negative portrayal in the media of artists such as Tracey Emin and Damien Hirst, and of the Turner Prize itself, and the possibility that this has a negative impact on some adult students motivation to learn about contemporary art. Walker (1999: 12; cited in Addison & Burgess, 2003: 111) talks of the medias utterly predictable knee-jerk reactions and populist attacks on contemporary art and artists, leading to manipulation of the readers' emotions and encouragement of philistine attitudes and aggressive feelings. The problems such media responses pose in terms of adult learners study of contemporary art can be better understood in the context of a number of pedagogical theories. For example, Knowles (1985) theory of andragogy, while having several weaknesses (Schapiro, 2003) offers some pertinent insights into adult learners motivations and their need to be confident in what they have chosen to learn. Wlodkowski (1999: 74), building on Knowles ideas, explains that adults want to learn what they find meaningful and of some significant value to their personal or professional life. Judith Korosciks (See Koroscik, 1982; 1990; 1990a; 1992a; 1992b; 1993; 1994; 1996; 1996a) notion of conservative tendencies - the inclination of novices to approach learning by confirming preconceived ideas and personal biases, which sometimes reflect peer group consensus (Efland, 2002: 119) also appears relevant in the context of adult learners initial negativity about the prospect of studying contemporary art. Visual culture theorist Kerry Freedman (2003: 83) makes a similar point about the possibility that the misconceptions about art that learners might gain outside the classroom can be a barrier to their learning. Not all students were negative about the prospect of studying contemporary art, however. For example, one student revealed that she was over the moon and that it was knowing we were going to study the Turner Prize that made me sign up for the course. Another student confirmed that they were excited about the prospect of learning more about contemporary art and about unravelling some of its "mystique". A third student mentioned initial feelings of openness, willingness to find out and excitement. Changed feelings Continuing the positive note, the questionnaire responses did indicate that studying Making Sense of the Arts had led to a change of heart amongst many of the most sceptical students, with 59% revealing that their feelings about the value of contemporary art had changed since studying the course and 71% indicating that the art history section of the course was more enjoyable than they had expected it to be. Again, the following comments are fairly typical: I enjoyed the Art History VERY much and was pleasantly surprised at how much my attitudes to contemporary art have changed. I felt as though this section had the biggest impact on my confidence, as I feel I could hold my own now in a discussion on contemporary art.

66

Empathy, Alienation and Educational Inclusion

I am less judgemental and am now prepared to study and analyse each art work. I still have a feeling that some of the artists are conning us and that there is an 'Emperors new clothes' aspect to the gullible observers. I was absolutely amazed when in one of the exercises I found myself being positive about modern art. Me! The ECA art history subject Afrodizzia was a modern piece which included elephant dung in its media...At first I thought it was a gimmick but now I understand more about it and really enjoyed the piece and discussing it. I want to go back to Tate Modern and have another look - I am sure I will still feel the same about some but perhaps I will be able to appreciate them more - even if I don't like them! The course has helped me to look at things in a completely different way. I now realise that I dont have to like or even totally understand a piece of art work to be able to talk about it or think about it. Particularly encouraging, in the interests of educational inclusion, was the fact that 66% of students said they were now more confident about discussing contemporary art with other people and several students commented that they felt they had gained access to a world that had previously seemed closed to them. An age-related divide As has already been seen, the reasons for students initial negativity about studying contemporary art ranged from concerns that they would not be able to understand works whose meanings appeared obscure to feelings that such artworks were not art at all and were therefore not worth studying. Interestingly, their explanations appeared to be divided along age-related lines, with students aged over 50 tending to be the most cynical about whether Turner Prize art was worth studying. While a detailed analysis of the semantic differential scale data has yet to be conducted, an early exploration of the data also highlighted particularly significant correlations between students age and their reactions to the 17 artworks featuring in the survey with students aged over 50 again tending to be the most negative about the contemporary artworks, more frequently judging them to be Not worth studying, Pointless and Offensive , as shown in Figure 2 - a graph comparing students responses to Tracey Emins My Bed and Raphaels Madonna of the Meadows. Judgements related to skill and emotional impact Kozbelt (2004: 157) observes that two components frequently emerge in the discussion of artistic quality: technical skill and originality and analysis of the semantic differential scale data did indeed indicate a link between students judgements regarding the skills shown in an artwork (the adjective pair Skilled-Lacks Skill) and whether it was worth studying. Once again however, this was much more common amongst students aged over 50.

67

Perryman

Figure 3: Chart comparing students reactions to Raphaels Madonna of the Meadows and Tracey Emins My Bed.

Students aged under 50 (and especially students aged under 30) appeared to be less likely to link the artists apparent skill with the worth of an artwork and were more likely to base such a judgement on the emotional power of the work (using the adjective pairs Emotionally intense-Lacks emotional impact, Intimate-Remote and WeakPowerful), rating an artwork to be worth studying when they perceived it to be emotionally engaging and stimulating. These correlations will be further explored in the next phase of the research. Foregrounding the monster A commonly expressed objection to teaching with contemporary art at all educational levels is that students may be offended or upset by the challenging and controversial themes that are often addressed and the use of formal elements that are deliberately designed to shock and offend. Artist Mark Hutchinson confirms that in secondary education teachers often shy away from most contemporary art because they consider it too difficult, an art full of monsters, replete with vulgarity and coarseness (Hutchinson, 1998: 144). British art educator and writer Lesley Burgess, writing about the challenges of teaching with controversial art in the secondary school curriculum, confirms that by refusing to engage with potentially problematic practices educators may be missing an opportunity to confront important personal, social and cultural issues (Burgess, 2003: 68

Empathy, Alienation and Educational Inclusion

108). She proposes that by foregrounding the monster (Burgess, 2003: 120) and addressing difficult, controversial or complex issues, contemporary art is actually particularly effective as a stimulus for students achievement of a wide range of learning outcomes. Interestingly, it was apparent from the semantic differential scale data that students judgements about whether an artwork was worth studying were not necessarily dependent on their feeling comfortable about it, or finding it beautiful or pleasing. For example, it was not uncommon for a student to give an artwork a high score for being Worth Studying while also indicating that they found it repulsive, offensive or unpleasant. Carole Beckers work is of interest here. Becker (1994: 119), a protg of Hebert Marcuse, recommends that art educators should revisit Marcuses belief that to be effective, art must exert its capacity for estrangement and should challenge societys assumptions through 'the demands of intellectual and visual rigour and/or the heightened recognition of pleasure and pain (Becker, 1994: 119-120). Marcuses assertion that art only becomes effective (and perhaps also affective) when its content is embodied in an aesthetically challenging form that [pushes] the viewerto a more complex, more emotional, or revelatory understanding of the problems posed by the work' (p121) merits further exploration in the context of the Making Sense of the Arts survey data, especially in terms of exploring the possibility that a pedagogy employing meta-cognitive strategies and reflective writing could give students an access point to and a way of making sense of such aesthetically challenging art. Silvia and Brown (2007: 100), exploring the emotions that lead people to reject, censor, and deface works of art, differentiate between anger and disgust, reporting that anger was associated with appraising a picture as incongruent with ones values and as intentionally offensive, and disgust was associated with appraising a picture as incongruent with ones values and as unpleasant (Silvia & Brown, 2007: 100). Freedman (2003: 65), discussing adult students responses to visual culture, highlights the impact of students expectations in informing their reactions to art, explaining that expectation is an emotional state tied to knowledge, often knowledge of form and that people who see a work of art that is apparently unrelated to anything they have seen before might respond as if it is threatening. Future analysis of the Making Sense of the Arts survey data will therefore explore in more detail possible correlations between students responses to the scales Worth Studying-Not Worth Studying, OffensiveInoffensive, Comforting-Disturbing, Repulsive-Attractive and Unpleasant-Pleasant and whether any correlations appear to be linked with students age and/or gender. The link between students expectations and the form of artworks which they find to be unpalatable will also be explored in interview. Looking at pedagogy The Study Diamond Some of the most positive student comments referred to the pedagogical approach featuring in Making Sense of the Arts and, more specifically, a meta-cognitive framework named the Study Diamond (see Figure 3) which is used to guide students learning within the disciplines of poetry, history and art history. Throughout the course students are prompted to build their interpretations of humanities texts by making links between the four points of the Study Diamond, namely: 69

Perryman

Techniques (the form of a text) Effect (its emotional impact) Meaning (the apparent connotation(s) of a text) Context (including reflexive consideration of students own subjective context). While conceived in its current format by the Making Sense of the Arts course authors, the Study Diamond has a solid ancestry in cognitive and pedagogical theory, providing a framework within which disparate artworks can be analyzed and then compared. Potentially, a meta-cognitive framework such as the Study Diamond can also facilitate students learning by slowing down their looking (Perkins, 1994; cited in Efland, 2002: 118), prompting them to take more time to look for relevant details and, as suggested by student questionnaire responses, can counteract the effects of Korosciks (1982; 1990; 1996) conservative tendencies (mentioned earlier). Figure 4: The Study Diamond

Making Sense of the Arts co-author and Study Diamond co-designer Tim Baugh explains that:

70

Empathy, Alienation and Educational Inclusion

As a tool for learners and teachers the Study Diamond provides a range of techniques aimed at raising awareness of the need to view reflective and analytical processes as inseparably linked in successful learning and teaching. One main area of focus is the balanced and dynamic integration of students personal responses, views and perspectives with the views of apparent experts and with those of their peers, especially those views and responses differing from their own. This, in turn, allows students to address the issue of subjectivity as a partial explanation for the multiplicity of meanings offered by any particular text. (Baugh, 2008) Charman and Ross (2006: 30), addressing the challenges posed by the process of interpreting contemporary visual art confirm the value of meta-cognitive frameworks, arguing that: Approaching the process of interpretation with a toolkit of thinking skills is particularly useful with regard to contemporary visual art, in which meanings can be contradictory, multiple and are certainly open-ended and unstable. In the light of such open-endedness, teaching the skills of interpretation benefits from a structured approach and method. The Study Diamond was almost unanimously seen as a positive impact on students study of art history, with 82.2% of students stating that it had a Very Positive (44.8%) or Positive (37.4%) impact on their studies. Students comments were illuminating: Working with the study diamond made me approach the artworks in a positive and inquisitive manner and so I derived much more benefit from the study than I would have done without it. The art history wasn't as intimidating as I had expected once I got into it. The study diamond was worth the entry price. I will use that for everything from now on genius. The study diamond gives a framework to study when it might have been difficult to know quite how or where to start. The course actually opened up art to me and the study diamond has provided me with a really useful means of viewing art at a personal level. The next research phase will include further exploration of the impact of the Study Diamond on students learning, through interviews with students and through analysis of their writing in response to the course activities and assignments.

What Next?
The data collected thus far appears to give a clear indication that some Making Sense of the Arts students continue to feel negatively about contemporary art after studying the course, bemoaning (amongst other things) its ugliness, its offensive subject matter and artists apparent lack of skill. Silvia and Brown (2007) point out that essentially no 71

Perryman

research has been done on negative responses to art. Their own application of the appraisal model of aesthetic emotions (see Lazarus, 1991; Roseman & Smith, 2001; Ellsworth & Scherer, 2003 and Silvia, 2005) to anger and disgust in response to visual art is a recent exception, exploring both how negative aesthetic emotions happen and how similar negative emotions differ from each other. The next stage of the Making Sense of the Arts research project will therefore use semi-structured interviews and analysis of students writing to further explore the implications of Silvia and Browns work (and other research into appraisal theories of emotion) in informing a better understanding of the barriers to learning faced by adult students of the visual arts and the ways in which they might be minimised. Pedagogy-related issues will be a particular focus, for example the significance of meta-cognitive frameworks such as the Study Diamond in offering students a way of making sense of their negative emotions and of integrating them with formal and contextual analysis and the consideration of others perspectives to achieve a considered and well-argued interpretation of any artwork they might encounter. More extensive statistical analysis of the survey data collected from the two cohorts of Making Sense of the Arts students will also take place, including multilevel modelling intended to help identify the relationship between students emotional reactions to canonical and non-canonical, contemporary and non-contemporary art, their feelings about its aesthetic properties, and their views about whether such art is worth studying. The next stage of the Making Sense of the Arts research project will also explore the possibility that some adult students value judgements about the worth of canonical noncontemporary art (and the worthlessness of contemporary art) might prevent them from any meaningful engagement with the latter, irrespective of the pedagogical context within which it is presented. Art educator Tom Gretton (2003) addresses this possibility, observing that although high culture is often seen as reproducing not inspiring ideals and transcendent values, but ethnocentricity, patriarchy, and the norms of bourgeois individualism (Gretton, 2003: 179) the canon continues to be reproduced. Teachers therefore have two choices, he suggests: to jointhose who denounce and reject the canon or to accept that its definition and reproduction meets some powerful cultural needsand find acceptable ways of dealing with it (Gretton, 2003: 179). Gretton proposes that the latter is preferable and that engagement with a loose canon can produce relevant knowledge, transferable understanding and cultural empowerment (Gretton, 2003: 183), explaining that as students begin to feel some sort of cognitive and cultural power over the objects they study, they will develop a sense that the canon and its values belong to them, are theirs to play with as they see fit (Gretton, 2003: 186). Hopefully the [Making Sense of the Arts research will reach some useful conclusions about whether Grettons views can be reconciled with those of art museum educator Melinda Mayer (2008: 77), who declares that when we teach with contemporary art, the potential is present for learning that is centred not in the classroom, but in all the worlds beyond it and students efforts to negotiate their relationship to those worlds. What could be more relevant? Above all, it is hoped that the project will contribute to knowledge about the ways in which higher education might achieve an arts curriculum which is transformative, (Banks, 2001); empowering (Gay, 2000) and emancipatory, resulting in better understanding of interconnections among individual, local, national, ethnic, global and human identities; and acceptance of knowledge as something to be continuously shared, critiqued, revised and renewed (Gay, 2000: 35) an arts curriculum 72

Empathy, Alienation and Educational Inclusion

in which both empathy and alienation in response to visual art can be reconciled in the interests of educational inclusion.

References
Addison, N. & Burgess, L., (Eds). (2003). Issues in Art and Design Teaching. London: RoutledgeFalmer. Banks, J. A. (2001). 'A curriculum for empowerment, action and change'. In C. E. Sleeter (eds). Empowerment through multicultural education. Albany: State University of New York Press: 125-141. Baugh, T. (2008) Interview. [Interview with Leigh-Anne Perryman, 2 February 2008.]. Becker, C. (1994). 'The Education of Young Artists.' In H. A. Giroux & P. McLaren (eds). Between Borders: Pedagogy and the Politics of Cultural Studies. New York: Routledge. Becker, C. (1994). 'Herbert Marcuse and the Subversive Potential of Art'. In C. Becker (eds). The Subversive Imagination: Artists, Society, and Social Responsibility. New York: Routledge. Berlyne, D. E. (1974). Studies in the New Experimental Aesthetics. Washington DC: Hemisphere Publishing Corporation. Boughton, D. & Mason, R., (Eds). (1999). Beyond Multiculturalism: International Perspectives. Berlin: Waxmann. Bryman, A. (2006). 'Integrating quantitative and qualitative research: how is it done?' Qualitative Research 6(1): 97-113. Burgess, L. (2003). 'Monsters in the playground: Including contemporary art'. In L. Burgess & N. Addison (eds). Issues in Art & Design Teaching. London: RoutledgeFalmer: 101-121. Cahan, S. & Kocur, Z. (1996). Contemporay art and multicultural education. New York: The New Museum of Contemporary Art and Routledge. Chalmers, G. (1996). Celebrating Pluralism: Art Education and Cultural Diversity. Los Angeles, CA: The Getty Education Trust. Charman, H. & Ross, M. (2006). 'Contemporary Art and the Role of Interpretation: Reflections from Tate Modern's Summer Institute for Teachers'. International Journal of Art & Design Education 25(1): 28-41. Creswell, J. W. & Plano Clark, V. L. (2007 ). Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage.

73

Perryman

Dash, P. (1999). 'The teaching of art to Afro-Caribbean students in the UK'. In D. Boughton & R. Mason (eds). Beyond Multicultural Art Education: International Perspectives. Berlin: Waxmann: 135-150. De Vaus, D. (2007). Surveys in Social Research (5th Edition). London: Routledge. Denzin, N. K. & Lincoln, Y. S., (Eds). (2005). The Sage Handbook of Qualitative Research (3rd Edition). Thousand Oaks, CA: Sage. Efland, A. (2002). Art and Cognition: Integrating the Visual Arts in the Curriculum. New York: Teachers College Press, Columbia University. Eisner, E. (1994). Cognition and Curriculum Reconsidered. New York: Teachers College Press. Eisner, E. (1998). The Kind of Schools We Need. Portsmouth, NH: Heinemann. Eisner, E. (2002). The arts and the creation of mind. New Haven, CT: Yale University Press. Eisner, E. W. (1984). 'Alternative Approaches to Curriculum Development in Art Education'. Studies in Art Education 25(4): 259-264. Ellsworth, P. C. & Scherer, K. R. (2003). 'Appraisal processes in emotion'. In R. J. Davidson, K. R. Scherer & H. H. Goldsmith (eds). Handbook of affective sciences. New York: Oxford University Press: 572-595. Freedman, K. (2000). 'Social Perspectives on Art Education in the U. S.: Teaching Visual Culture in a Democracy'. Studies in Art Education 41(4): 314-329. Freedman, K. (2003). Teaching Visual Culture: Curriculum, Aesthetics and the Social Life of Art. New York: Teachers College Press. Frick, A., Bachtiger, M. T. & Reips, U. D. (2001). 'Financial incentives, personal information and drop-out rate in online studies'. In U. D. a. B. Reaps, M. (Eds.) (eds). Dimensions of internet science. Lengerich: Pabst Science Publishers: 209-220. Furnham, A. & Walker, J. (2001). 'The influence of personality traits, previous experience of art, and demographic variables on artistic preference'. Personality and Individual Differences 31(6): 997-1017. Gay, G. (2000). Culturally Responsive Teaching: Theory, Research & Practice. New York: Teachers College Press. Greene, J. C. (2008). 'Is Mixed Methods Social Inquiry a Distinctive Methodology?' Journal of Mixed Methods Research 2(1): 7-22.

74

Empathy, Alienation and Educational Inclusion

Greene, J. C., Caracelli, V. J. & Graham, W. F. (1989). 'Toward a Conceptual Framework for Mixed-method Evaluation Designs'. Educational Evaluation and Policy Analysis 11(3): 255-74. Gretton, T. (2003). 'Loaded Canons'. In N. Addison & L. Burgess (eds). Issues in art and design teaching. London: RoutledgeFalmer: 178-187. Hewson, C., Yule, P., Laurent, D. & Vogel, C. (2003). Internet Research Methods. London: Sage. Hooks, b. (1994). Teaching to Transgress. New York & London: Routledge. Hooks, b. (1995). Art on my Mind: Visual Politics. New York: New Press. Hutchinson, M. (1998). 'Of Monsterology'. In D. McCorquodale, N. Siderfin & J. Stallabrass (eds). Occupational Hazard: Critical Writing on Recent British Art. London: Black Dog. Knight, W. B. (2006). 'Using Contemporary Art to Challenge Cultural Values, Beliefs, and Assumptions'. Art Education 59(4): 7-39-45. Knowles, M., & Associates (1985). Andragogy in action. San Francisco: Jossey-Bass. Koroscik, J. (1982). 'The effects of prior knowledge, presentation time and task demands on visual art processing.' Studies in Art Education 23(3): 13-22. Koroscik, J. (1990) Novice-expert differences in undestanding and misunderstanding art and their implications for student assessment in art education. Paper presented at symposium, Novice-Expert Paradigms for Student Assessment of Art Learning, Boston Koroscik, J. (1990a). 'The function of domain specific knowledge in understanding works of art.' In (eds). Inheriting the theory: New voices and multiple perspectives on DBAE. Los Angeles: J. Paul Getty Trust. Koroscik, J. (1992a). 'A comparative framework for designing visual arts curricula.' Arts Education Policy Review 94(1): 17-22. Koroscik, J. (1992b). 'Research on understanding works of art: Some considerations for structuring art viewing experiences for students.' Kasvatus: Finnish Journal of Education 23(5): 20-25. Koroscik, J. (1993). 'Learning in the visual arts: Implications for preparing art teachers'. Arts Education Policy Review 94(5): 20-25. Koroscik, J. (1994). 'Blurring the line between teaching and research: Some future challenges for arts education policymakers.' Arts Education Policy Review 96(1): 2-10.

75

Perryman

Koroscik, J. (1996a). 'What potential do young people have for understanding works of art?' In A. Kindler (eds). Child development in art. Reston, VA: National Art Education Association: pp. 143-164. Koroscik, J. S. (1996). 'Who Ever Said Studying Art Would Be Easy? The Growing Cognitive Demands of Understanding Works of Art in the Information Age'. Studies in Art Education 38, (No.1): pp4-20. Kozbelt, A. (2004). 'Originality and Technical Skill as Components of Artistic Quality'. Empirical Studies of the Arts 22(2): 157-170. Lazarus, R. S. (1991). Emotion and adaptation. New York: Oxford University Press. Lippard, L. R. (1990). Mixed blessings: New art in a multicultural America. New York: Pantheon Books. Locher, P. J., Smith, J. K. & Smith, L. F. (2001). 'The influence of presentation format and viewer training in the visual arts on the perception of pictorial and aesthetic qualities of paintings'. Perception 30: 449-465. Martindale, C. (2007). 'Recent trends in the psychological study of aesthetics, creativity and the arts'. Empirical Studies of the Arts 25(2): 121-141. Martindale, C., Moore, K. & Borkum, J. (1990). 'Aesthetic Preference: Anomalous Findings for Berlyne's Psychobiological Theory'. The American Journal of Psychology 103(1): 53-80. Mayer, M. M. (2008). 'Considerations for a Contemporary Art Curriculum'. Art Education 61(2): 77-79. McFee, J. K. (1986). 'Cross-cultural inquiry into the social meaning of art: implications for Art Education'. Journal of Multicultural Research in Art Education 4(1): 6-15. Nind, M. (2005). 'Models and practice in inclusive curricula'. In M. Nind, J. Rix, K. Sheehy & K. Simmons (eds). Curriculum and Pedagogy in Inclusive Education: Values into Practice. London: RoutledgeFalmer: 1-10. Osgood, C. E., May, W. H. & Miron, M. S. (1975). Cross-cultural Universals of Affective Meaning. Urbana, Illinois: University of Illinois Press. Osgood, C. E., Suci, G. J. & Tannenbaum, P. H. (1957). The measurement of meaning. Urbana: University of Illinois Press. Perkins, D. N. (1994). The Intelligent Eye: Learning to think by looking at art. Santa Monica, CA: Getty Center for Education in the Arts. Roberts, L. D. & Parks, M. R. (2001). 'The social geography of gender switching in virtual environments on the Internet'. In E. a. A. Green, A. (Eds.) (eds). Virtual Gender: Technology, Consumption and Gender. London: Routledge: 265-285. 76

Empathy, Alienation and Educational Inclusion

Roseman, I. J. & Smith, C. A. (2001). 'Appraisal theory: Overview, assumptions, varieties, controversies'. In K. R. Scherer, A. Schorr & T. Johnstone (eds). Appraisal processes in emotion: Theory, methods, research. New York: Oxford University Press: 319. Schapiro, S. A. (2003). 'From Andragogy to Collaborative Critical Pedagogy: Learning for Academic, Personal, and Social Empowerment in a Distance-Learning Ph.D. Program'. Journal of Transformative Education 1(2): 150-166. Silvia, P. J. (2005). 'Emotional responses to art: From collation and arousal to cognition and emotion'. Review of General Psychology 9: 342-357. Silvia, P. J. (2006). 'Artistic training and interest in visual art: Applying the appraisal model of aesthetic emotions'. Empirical Studies of the Arts 24: 139-161. Silvia, P. J. (2007). 'An Introduction to multilevel modeling for research on the psychology of art and creativity'. Empirical Studies of the Arts 25(1): 1-20. Silvia, P. J. & Brown, E. M. (2007). 'Anger, disgust, and the negative aesthetic emotions: Expanding an appraisal model of aesthetic experience.' Psychology of Aesthetics, Creativity and the Arts 1(2): 100-106. Slee, R. (1999). 'Policies and practices? Inclusive education and its effects on schooling'. In H. Daniels & P. Garner (eds). Inclusive Education: Supporting Inclusion in Education Systems. London: Kogan Page: 194-206. Stake, R. (2005). 'Case Studies'. In N. K. Denzin & Y. S. Lincoln (eds). Handbook of qualitative research. Thousand Oaks, CA: Sage. Tan, E. S. & Tollenaar, A. (2007). 'Aesthetic Communication through Posters'. Empirical Studies of the Arts 25(1): 21-39. Tashakkori, A. & Teddlie, C. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage. UK Online Centres. (2007), Understanding Digital Inclusion, Available from http://www.ukonlinecentres.com/downloads/UK_Online_digitalinclusion.pdf [Accessed 20 February 2008]. Walker, J. A. (1999). Art and Outrage: Provocation, Controversy and the Visual Arts. London: Pluto. Wlodkowski, R. (1999). Enhancing adult motivation to learn. San Francisco: JosseyBass.

77

Perryman

Young, B. (1999). 'An African-American Perspective on Multicultural Art Education'. In D. Boughton & R. Mason (eds). Beyond Multicultural Art Education: International Perspectives. Berlin: Waxmann: 21-32.

78

PRIME Volume 3(2)

Enhancing learning? Exploring the use of assessment methods as pedagogical tools to promote effective learning and teaching in early childhood studies
Paulette Luff and Rachel Pryor Anglia Ruskin University, Faculty of Education, Early Childhood Studies

Summary
This paper highlights aspects of our experience as exploratory practitioners researching the use and the value of two modes of assessment (web-based discourse and sketchbook learning journals). We report our developing understandings of ways in which these can support and extend students learning. On our Early Childhood Studies (ECS) courses we emphasise approaches to young childrens education informed by socio-cultural theories. This promotes a view of learning which stresses the importance of shared meaning making and the co-construction of knowledge. Accordingly, we draw upon the Vygotskian concept of pedagogical tools, mediating and extending knowledge construction, and emphasise a close relationship between means of assessment and student learning. The study uses a case study design, apt for monitoring and explaining educational practices. Most data gathering is integrated into the module programmes with the sketchbook journals and the web-based discourse assessments themselves forming important sources of qualitative data, together with staff and students reflection on the processes. Evidence, from our initial analysis of findings, indicates that sketchbook learning journals can provide a means for students to capture, synthesise, reflect upon and critique their learning. Similarly, web-based discourse (as an element of blended learning) encourages collaborative study and offers opportunities for formative assessment, through e-tivities designed to support students reflective and critical thinking about inclusive policies and practices in early childhood settings. The paper concludes by summarising ways in which engagement with research-informed teaching has supported developments and improvement and deepened our understanding of learning and teaching processes.

Introduction
This paper arises from an exploration of assessment practice undertaken during the past academic year by two lecturers who are relatively new to teaching in higher education and to undertaking pedagogical research. The main value of this self-reported study was, therefore, to enable us to reflect upon our own practice, however we hope that this account of our work will resonate with the experience of other educators and contribute to a shared understanding of the relationships between learning, teaching and assessment. The paper outlines our conceptualisations of assessment and some key insights relating to the methods of assessment explored, before giving a brief description of the study and discussing some key findings regarding the relationships between assessment and student learning. We conclude by highlighting aspects of our own learning and some of the issues raised by the project.

79

Luff and Pryor

Understandings of assessment
Our teaching and research espouse three key principles of socio-cultural theory. Firstly, recognition that learning is a social process, embedded in historical and cultural contexts (Vygotsky, 1978, 1981; Rogoff, 1998); thus requiring course content to match and build upon students prior learning and experience and offer opportunities for shared knowledge construction. Secondly, a belief that effective learning occurs within zones of proximal development (Vygotsky 1978, 1986); with the attendant responsibility to create learning environments and assessment opportunities which afford an appropriate level of intellectual challenge and provide support for students to achieve their potential. Thirdly, that human learning is mediated and extended by the use of tools; implying that forms of assessment can assist and shape our thinking and actions (Vygotsky, 1978; Kozulin, 2003; Cowie and Carr, 2004). In accordance with this theoretical standpoint, our aspirations for assessment are that it should: enable lecturers and students to build positive relationships; foster positive dispositions for learning; offer opportunities for students to engage with subject knowledge in order to construct understanding; promote literacies and critical skills; and be of academic, professional and personal value (Goodfellow and Lea, 2005; Dunn et al, 2004; Taras, 2002; Maier and Warren, 2000; Ashcroft and Foreman Peck, 1994). Assessment and e learning Jacques and Salmon (2007) define interactive on-line learning and web based discourse as based on written communications between student to student and student to lecturer. The student posts contributions onto a virtual learning environment (VLE) in response to activities set by the lecturer. The interactive and participative element is in the response to postings by others and participants make sense of the material through this interaction. Maier and Warren (2000) argue that this form of teaching and learning may lessen the tutor-student bond but increase student independence and interdependence between one another. Ashcroft and Foreman-Peck (1994:92) agree and comment that this form of open pedagogy and interaction gives students opportunities for exploration and enquiry and allows them to come back to the questioner at a stage further than they were when the question was posed. They point out that a rational addition to this pedagogy is a form of assessment that is a learning experience in its own right. In response to this argument the web based discourse was informed by Salmons (2000: 29) Five Stage Model of Facilitating Online Learning where e-tivities are used to scaffold learning, moving up in stages, from individual participants establishing their on line identities. to where course related group discussions occur and the interaction becomes more collaborative. Weller (2007) argues that the web based discourse encourages students to share and comment on each others postings which reinforces the belief that learning is a social construct encouraged by peer to peer interaction and learning from peers, as the students share and comment upon each others remarks. Both Weller (2007) and Macdonald (2006) agree that e-learning helps to develop independent self directed students and lays the foundations of lifelong and collaborative learning as it provides the opportunity for a range of teaching and assessment strategies. It can enhance student learning by allowing more opportunities for reflection than in perhaps a wholly face to face environment. Macdonald (2006:2) defines blended learning as 'the introduction of online media into a course while 80

Enhancing Learning?

at the same time retaining face to face contact and other traditional approaches to supporting students'. She argues that this can encourage students to learn and construct their own understandings through learning and assessment activities, helping to relate theory and practice to experiences that they have had. Learning journals as a form of assessment Making links between theory and workplace practice is also a feature of learning journal style assignments. These are common in vocational subjects, such as health, social care and education, as they provide students with opportunities to reflect upon their experience and develop informed knowledge (Brookfield, 1995; Dart et al, 1998, Heath, 1998; Moon, 2006). In an analytical review of assessment practice, Hounsell et al (2007) found that writing assignments such as portfolios and journals were used in order to promote deep and reflective learning. The key advantages of this type of assignment included the documenting of learning over time; the construction of a resource for future learning; improvement in students written communication skills; and motivation towards selfdirected learning (ibid). Others have, similarly, identified the benefits of journal style assignments in fostering depth of engagement with subject content knowledge (Dart et al, 1998; Park, 2003) and critical and self-aware approaches to learning (Haigh, 2001). On courses where the majority of students are female, a form of assessment which encourages the formation of connections between prior and new knowledge and allows the integration of affective and cognitive learning may be particularly conducive to womens ways of knowing (Clifford, 2002; Belenky et al, 1997). Robinson et al (2007) describe sketchbooks as playgrounds for ideas in which sources of inspiration can be highlighted, questions and interests can be explored, learning journeys can be recorded, and thinking can be supported. The use of a visual approach allows ideas to be expressed in the form of mind maps and diagrams and encourages presentation and critique of images in addition to written text. The preparation and presentation of sketchbook learning journals parallels processes of documenting young childrens learning in order to make their thinking visible and enable reflection upon learning (Guidici et al, 2001; Rinaldi, 2006). The use of sketchbooks as a tool for learning and research thus offers students a space in which they can investigate the early childhood curriculum and develop and present their theories, based upon fieldwork in early childhood settings and their reading.

The research project

We approached this research from an exploratory practitioner perspective (Campbell and Norton, 2007: 1) actively questioning the value of two modes of assessment and conducting an enquiry with the intention of furthering our understanding of student learning and of developing our own teaching practice. The sketchbook learning journals and web-based discourse, in the context of the modules in which they are embedded, were viewed as two parallel case studies. A case study was selected as a useful means of investigating an area of interest in depth and detail in order to discover meaning and draw conclusions to inform educational practice (Merriam, 1998; Stake, 1995; Sanders, 1981; Reichardt and Cook, 1979).

81

Luff and Pryor

Our research was conducted with groups of early childhood studies students, and most of the data collection activities were integrated with our teaching. Throughout the academic year, aspects of students' learning behaviour were noted and discussed; focus group discussions were conducted to improve understanding of participants experiences; and the content of assessed work was analysed. In addition, a questionnaire was administered to the students undertaking the e-tivities and four of the students who had completed the learning journals took part in more interviews, which were videotaped. The findings, below, are based upon content analysis of data from these sources. Where student comments are used, these are quoted in italics. The e-tivities case study was carried out with two groups of year 3 students taking the Inclusive Practice in Early Childhood module. The first group were undergraduates in the final year of their Early Childhood Studies degree. The second group were mature students undertaking the Early Years Practitioner Status award, studying on a part time basis while working during the rest of the week. Assessment for the module was in two parts, consisting of participation in web based discourse and a conventional written paper. The aim was to encourage students to share and develop ideas, using discussion as a springboard to examine and challenge the issues and implications of inclusion in the early years. The discourse was led and supported by three e-tivities, a literature review, a web based research question and a final group writing activity drawing on the information and discussion that arose from the first two activities. These shared e-tivities then provided a basis for the individual written assignment. Second year BA students studying a Curriculum in Early Childhood module undertook a different, low tech, style of assignment. This took the form of a sketchbook learning journal, in which students represented their thoughts about different aspects of the early childhood curriculum (philosophical approaches, environments and resources, adult roles, playful approaches to learning and curriculum content). This learning was recorded in a visual manner, using images, charts, mind maps and text, together with 3000 words of critical commentary. This was completed during two full semesters and submitted for summative assessment at the end of the second semester.

Some findings
Web-based discourse One of the advantages of blended learning, where the e-moderator is a lecturer who also meets face to face with students, is that a rapport can be established before the web-based discourse begins. This was of particular benefit to the mature students, who were unfamiliar with the technology and needed help to access the WebCT virtual learning environment. For the undergraduate students, the weekly face to face seminars were used as a reminder to carry out the e-tivities and to help recognise the value of the discourse as information and research for the final written section of the assessment. Some of the students commented that as face to face contact time was readily available, the web based discussion appeared less attractive. The mature students, who only met as a group for one day every week, remarked how the discourse extended their discussion time together, how it improved communication, group cohesion and support. One student commented that the use of ICT had: 82

Enhancing Learning?

'increased communication and the sharing of ideas, perceptions and beliefs which also developed teamwork. There is always something to be learnt from working with other people. I understand that I am better at co-ordinating a project than leading one. It is very satisfying when a colleague recognises your strengths and is able to support you'. Macdonald (2006) argues that blended learning can provide challenges for students as learning on line may require more independence, self direction and autonomy in relation to their study. It may be that mature students are better equipped to study independently, skilled at time management and more likely to use reflective skills. The advantage of blended learning is that the face to face element can help to maintain and reinforce those skills needed to develop understanding and construct meanings. Instead of receiving information from a lecturer, the web based discourse should encourage students to find information and help them reflect and find meaning. As one student commented; 'it has been a very stimulating way to learn and I liked the fact that we as a group, support and develop our own practice and knowledge. (There is) Much more access to information as each person brings their own experience and learning'. Another said; 'I found that before using the discourse I was perhaps quite narrow minded or naive within my opinions and attitudes, whereas through participating in the discourse I have been able to gain a deeper understanding of the wider aspect of diversity, in addition it has made me reflect on my own views and working practice and to review the policies and procedures within my setting'. The comments demonstrated that the process of sharing information and knowledge, and contributing to the discussion, consolidated and improved understandings. Weller (2007) comments that assessment is the driver that motivates students and, interestingly, the mature students remarked that they would have preferred the web based discourse element to be fine graded rather than pass/fail as they valued the lecturers opinion of the standard and academic level of their discussion. These students had just completed a Foundation Degree and were concerned about progressing to higher education study at level 3, with its increased expectation of analysis and evaluation. This contrasted with the approach taken by the majority of undergraduate students, some of whom posted one sentence answers using mobile phone text language. Protocols were developed during the first semester, explaining the style of writing expected during the discourse and the difference between the use of the discussion forum and informal e mail chat. The protocols had to be clear as to how often the lecturer would log on, and the length and academic level of the responses, so that levels of commitment were clear to both parties from the beginning. The mature students remarked that they thought the e-moderator should have contributed more frequently to their commentaries and they were disappointed to find that, at times, there had been no response. It became apparent that praise and encouragement was essential to encourage motivation and enhance student confidence.

83

Luff and Pryor

Sketchbook learning journals The sketchbooks allowed all students to record and store information, thus creating a record of the learning that took place during the module. This contrasts with other forms of assessment in which just a small part of the module content becomes the focus for an assignment. Students who engaged with the module content throughout the academic year demonstrated a building of knowledge. One student described her journal becoming sort of like a Bible, and many expected to return to their sketchbooks for future reference. A mature student, who is intending to set up her own nursery, expressed an intention to display the sketchbook on the premises to explain her educational philosophy to colleagues and parents. Park (2003: 193) reports that learning journals encourage students to focus on the whole of their course and construct their own synthesis of the course material. Likewise, in their sketchbooks students brought together ideas from lectures, seminars, fieldwork and reading in order to create their own arguments and present their developing views about the curriculum for young children. Documenting learning in this way, over time, also assisted students in seeing relationships between different parts of the module: You can see the links between ideas as you go backwards and forwards; and to consolidate their learning: in an essay you can get away with writing something even if you dont fully understand it but I really know about everything I have put into my sketchbook. You cant fault it as a learning tool. There is evidence in most of the sketchbook assignments of genuine enquiry, analysis and critique. Students gained confidence in discussing theory through analysing their fieldwork, looking at practice in early childhood settings, and linking this with reading and research. Many students ideas changed and became more sophisticated throughout the module. Most typically they moved away from a simple understanding of the early childhood curriculum as a set of guidelines and standards to be followed and gained an appreciation of curriculum as dynamic and multifaceted; the process which emerges from the childs interests and allows each individual child to develop essential skills for life through their interaction with adults, the environment and their play activities. Some students reflected explicitly on this change, one wrote; I discovered that my knowledge of the curriculum was basic and limited. It is far more complex than I first thought .... Interestingly, a student who is already a confident critic of the English education system began to moderate her view and admitted to discovering that; curriculum does not necessarily equal oppression! The usefulness of this type of assignment as a tool to support learning is indicated by students who intend to continue working with this type of learning journal. One student who did not pass the module at the first attempt, but who only needed to add further 84

Enhancing Learning?

evidence of having met one learning outcome, chose instead to prepare a completely new sketchbook. She saw it as a way of deepening her knowledge of Montessori method in preparation for her final year major project. Another student, also looking ahead to her major project, asked whether she could submit a sketchbook journal as an appendix to a study of childrens creativity. A third student said that she would use a learning journal in order to gather, organise and reflect upon material to be included in her project. Our professional learning from this project Our findings indicate that sketchbook learning journals and web-based discourse can be used as tools in support of our efforts to teach in order to make learning possible (Ramsden, 2004). We have begun to understand how assessment can extend thinking and now more fully appreciate the benefits of integrating assessment with students learning throughout a taught module, rather than setting an assignment in order to test knowledge and understanding once the module has been completed. Whilst we have cited our positive findings above, we have also taken into account negative feedback from students (for example, concerns about workload and anxiety in interpreting tutor expectations) and will use this to inform our presentation of these assignments in the future. It is apparent that, in an assessment culture dominated by prescribed learning outcomes, more innovative forms of assessment may open spaces for self expression and independent learning and offer both the student and the lecturer greater flexibility in their approaches to learning. However the study has shown that skills of self-direction need to be carefully nurtured, enabled in part by the guidance on boundaries, expectations and levels of commitment given during the course. Mature students may be at an advantage over the younger students, using their lifelong learning skills to better equip them to use the skills of independent study. Responses suggest that students favour quite explicit guidelines for their work and that lecturers need to be sensitive to this and balance encouragement of independent, creative thought with structures that provide a basis from which students can take responsibility, so that student confidence is maintained and developed. As with any reflective teaching and exploratory research, we are left with many questions and much scope for future enquiry. Four particular issues arose in relation to both webbased discourse and sketchbook learning journals. Firstly, whether the assessment tasks were equitable? There was evidence that some students were more privileged than others, for example in having unrestricted access to computers and fast internet connections or more time and money to spend on their learning journals. Secondly, we began to question whether we, as lecturers, were well equipped with the literacies required to assess innovative tasks. In the most artistic of the learning journals, for example, much was expressed through strong images and it was difficult to assess these visual statements. Likewise, the e-tivities raised a debate about language with an expectation that the students should use academic voices conflicting with the styles of on-line communication that the undergraduate students were more accustomed to. Associated with this were the third and fourth problems: the extent to which lecturers should allow and encourage student autonomy, or direct the tasks and lead learners towards specific learning outcomes; and the challenge of engaging reluctant participants and enabling them to see the value and relevance of the assessment tasks. 85

Luff and Pryor

Both assessment approaches were based upon the principle that learning is a social process achieved in contexts that are meaningful, collaborative and reflective, which foster motivation and positive dispositions towards study. To achieve these ideals, learning and assessment activities need to be thought about and structured carefully so that they are embedded into the course, with clarity of purpose, and thus become an effective means of enhancing learning.

References
Ashcroft, K. and Foreman-Peck, L. (1994) Managing Teaching and Learning in Further and Higher Education, London: Routledge-Falmer. Belenky, M.F., Clinchy, B.M., Goldberger, N.R. and Tarule, J.M. (1997) Women's Ways of Knowing (2nd edn.) New York: BasicBooks Brookfield, S.D. (1995) Becoming a Critically Reflective Teacher. San Francisco, CA: Jossey-Bass. Campbell, A. and Norton, L. (2007) Learning, Teaching and Assessing in Higher Education. Exeter: Learning Matters. Clifford, V.A. (2002) Does the use of journals as a form of assessment put into practice principles of feminist pedagogy? Gender and Education 14 (2): 109-121. Cowie, B. and Carr, M. (2004) The Consequences of Socio-Cultural Assessment, in Anning, A., Cullen, J. and Fleer, M. (Eds.) Early Childhood Education, London, Thousand Oaks, New Delhi: Sage. Dart, B.C., Boulton-Lewis, G.M., Brownlee, J.M., and McCrindle, A.R. (1998) Change in knowledge of learning and teaching through journal writing, Research Papers in Education, 13 (3): 291-318. Dunn, L., Morgan, C., OReilly, M. And Parry, S. (2004) The Student Assessment Handbook: New Directions in Traditional & Online Assessment, London: RoutledgeFalmer. Giudici, C., Rinaldi, C. and Krechevsky, M. (2001) Making Learning Visible, Reggio Emilia: Reggio Children. Goodfellow, R. and Lea, M. R. (2005) Supporting writing for assessment in online learning, Assessment and Evaluation in Higher Education, 30 (3) June 2005 261-271. Jacques, D. and Salmon, G. (2007) Learning in Groups. A Handbook for face-to-face and online environments, (4th ed.) London. Routledge. Heath, H. (1998) Keeping a reflective practice diary: a practical guide, Nurse Education Today, 18 (18): 592-598.

86

Enhancing Learning?

Haigh, M.J. (2001) Constructing Gaia: using journals to foster reflective learning, Journal of Geography in Higher Education, 25 (2): 167-189. Hounsell, D., Falchikov., N., Klampfleitner, M., Huxham, M., Thomson, K. and Blair, S. (2007) Innovative assessment across the disciplines, An analytical review of the literature, Final Report. York: Higher Education Academy. Online. Available HTTP: http://hca.ltsn.ac.uk/assets/York/documents/ourwork/research/Innovative_assessment_LR .pdf (accessed 14 June 2008). Kozulin, A. (2003) Psychological Tools and Mediated Learning, in Kozulin, A., Gindis, B., Ageyev, V.S. and Miller, S. (Eds) Vygotskys Educational Theory in Cultural Context, Cambridge: Cambridge University Press. Macdonald,J.(2006) Blended Learning and Online Tutoring: A Good Practice Guide, Hampshire: Gower Publishing. Maier, P. and Warren, A., (2000) Integrating Technology in Learning and Teaching a practical guide for educators. London. Kogan Page Merriam, S.B. (1998) Qualitative Research and Case Study Applications in Education. San Francisco: Jossey-Bass. Moon, J (2006) Learning Journals (2nd edn.) London and New York: Routledge. Park, C. (2003) Engaging students in the learning process: The learning journal, Journal of Geography in Higher Education, 27 (2): 183-199. Ramsden, P. (2004) Learning to Teach in Higher Education (2nd edn.) London: RoutledgeFalmer. Reichardt, C.S. and Cook, T.D. (1979) Qualitative and Quantitative Methods in Evaluation Research, Thousand Oaks, CA: Sage. Rinaldi, C. (2006) In Dialogue with Reggio Emilia, London and New York: Routledge. Robinson, G., Hulston, D. and Mountain, A. (2007) Think inside the sketchbook a playground for ideas, Cambridge: Teknoy Press. Rogoff, B. (1998) Cognition as a Collaborative Process. In Kuhn, D. and Siegler, R.S. (Eds.) Handbook of Child Psychology, Vol. 2 (5th Edn.) New York: John Wiley. Salmon, G. (2000) E moderating. The key to teaching and learning online, (2nd edn). London: Taylor and Francis. Sanders, J.R. (1981) Case Study Methodology: A Critique, In W.W. Welsh (Ed) Case Study Research in Educational Evaluation. Proceedings of the Minnesota Evaluation Conference. Minneapolis: Minnesota Research and Evaluation Center. Stake, J. (1995) The Art of Case Study Research, London, Thousand Oaks, CA: Sage.

87

Luff and Pryor

Taras, M (2002) Using Assessment for Learning and Learning for Assessment, Assessment & Evaluation in Higher Education, 27 (6) 501 510. Vygotsky, L.S. (1978) Mind in Society: The Development of Higher Psychological Processes. Cole, M., John-Steiner,V., Scribner, S. and Souberman, E. (Eds.). Cambridge, MA: Harvard University Press. Vygotsky, L.S. (1981) The Genesis of Higher Mental Functions. In Wertsch, J.V. (Ed.) The Concept of Activity in Soviet Psychology. Armonk, NY: M.E. Sharpe. Vygotsky, L.S. (1986 / 1934) Thought and Language. Kozulin, A. (Ed.) Cambridge, MA: MIT Press. Weller, M. (2007) Virtual Learning Environments. Using choosing and developing your VLE, London: Routledge.

88

PRIME Volume 3(2)

A Non-mathematical Framework for Developing the Conceptual Understanding of Classical Mechanics

Peter Vivian Blackpool and The Fylde College, School of Art & Design in collaboration with Coventry University

Summary
The existence of a maths problem with the teaching of engineering science is well documented and shows little sign of abating. Obviously, engineers need mathematics to prove the efficacy of their designs but there is also a perception of the need for mathematics as a means of teaching the conceptual nature of engineering. This paper questions this assumption and suggests that we make more effective use of the existing, well developed, semantic language of classical mechanics. It is the aim of this research to develop a scheme of graphic communication that will enable students to develop their awareness of the basic concepts of classical mechanics without the use of mathematics, to develop their spatial abilities through graphic manipulation and to aid their application of mathematics. This paper outlines a framework to enable educators and students to take advantage of this scheme either online or in the classroom.

Introduction
This research arose out of attempts to teach the basics of classical mechanics to BA Technical Illustration students without recourse to mathematics. These students were interested in the subject but did not want to do any mathematics, perceiving the mathematical approach to be physics and of no interest to them. Research has shown that this anecdotal evidence is a small example of what is called the 'maths problem' and it is perceived by academics to be a major problem in the education of engineers (Kent & Noss 2003: 24). Kent & Noss (2003: 18, 30) state that 'no one dissents' from the use of mathematics to communicate the conceptual basis of mechanics but do themselves question (Kent & Noss, 2003: 28) whether mathematical fluency is a prerequisite to learning the principles of engineering in general. Ironically, practising engineers do not appear to agree with the academic community seeing mathematics as numerical manipulation and calculation rather than as a language for conceptual communication (Kent & Noss, 2003: 18). The university academic community is also disappointed with the level of understanding of mechanics of their new students, which Lee et al refer to as a 'mechanics problem' (2006: 1), but this is also seen as part of the 'maths problem' (Lee et al 2006: 37). However, Lee et al (2006: 37) also argue that the HE academic community cannot expect the schools to change their approach to teaching - a sentiment that has been reiterated to good effect in this conference (PRHE 2008). One of the solutions proposed by Kent & Noss (2003: 38) is the use of multimedia, who referring to it as 'the use of mathematics before understanding mathematics'. Multimedia 89

Vivian

enables students to investigate cause and effect much in the same way as traditional laboratory work and personal experience. Modern computer game technology can be successfully adapted to this purpose (Darling et al 2008: 9). Another variation on the laboratory exercise is to develop physical examples that demonstrate the effect of principles (Ji & Bell 2008). As this paper will show, the strategy of relying on multimedia has a basic flaw when considering the development of conceptual understanding. Multimedia is a depiction system that can be used to explore cause and effect but it cannot be used to explain the nature of the relationship between the cause and the effect. That requires a description system and traditionally this is considered to be mathematics. For example, the adaptation of a computer game (Darling et al 2008) for educational purposes necessitated the inclusion of mathematical graphs, i.e. a description system, to aid conceptual understanding.

Graphic communication
A number of studies have attempted to categorise graphic representations based on characteristics of graphic communication and as a result imply spectrums of variability from figurative pictures to arbitrary text. Tyman (1979) argued for 6 methods of configuration and 4 modes of symbolization resulting in 24 categories. Richards (1984) used modes of depiction, correspondence and noun-verb space resulting in 8 categories. Lohse (1990) attempted to identify 'homogeneous' clusters and came up with 5 major groups. Englehardt (2002), as well as identifying types of correspondence and expression, identified 10 primary types of representation with a further 6 hybrid types. When it comes to the form of representation commonly referred to as diagrams, the above studies failed to identify a form of communication called semasiographic writing systems (Sampson 1985: 28-32). For example, the vector diagram, which presumably would be categorized by Engelhardt (2002) as some form of 'link diagram' owing to its use of arrows, is not actually a diagram but a semasiographic writing system. Shimojima (1999) was concerned with defining the difference between text, i.e. the arbitrary representation of spoken language, and graphic representations. Identifying this distinction is made problematic because there are non-textual graphic representations that are semasiographic writing systems and work just like text, though being designed for a specialized communication, they are narrower in scope. Norman (2000) tried to differentiate diagrams from description systems and depiction systems, again failing to recognize that according to the Oxford English Dictionary, a diagram is a form of depiction system and that many graphic representations today, 90

Conceptual Understanding of Classical Mechanics

which are often called diagrams, are either a mixture of schematic depiction and symbolic description (e.g. an engineering drawing), or a symbolic system in their own right (e.g. vector diagram, flow diagram, organization chart). It is normal for graphic representations attempting to communicate a message in a direct or literal manner to include text in a supportive role. Horn (1998) is quite explicit about this role of text in graphic communication but again fails to explicitly identify the semasiographic nature of the other graphic systems. Figure 1: The two types of graphic communication

To conclude, graphic communication is usually a mixture of two distinct types of graphic communication (See figure 1): Depiction systems: These are used to represent the physical form of objects. The most figurative being photographs. Drawings and paintings have less detail, diagrams and schematic representations even less and geometry is used to represent basic shapes. Description systems: These are used to transmit conceptual information. These writing or symbol systems are interpreted by literate readers competent in the graphic language. In figure 1, I have chosen an image of Chinese to represent writing systems to make an important point. Assuming that you cannot read Chinese, you will still recognise this as a writing system though you will not be able to read it. However, the use of symbol systems in graphic communication is on the increase but it seems that we may not recognize them to be semasiographic writing systems. Take for example the road sign in figure 2.

91

Vivian

Remove the text and you may still recognise that turning left leads to the airport and turning right leads to a motorway. The coloured boxes and symbols are part of the road sign symbol system. Many of the symbols have to be placed in specific spatial positions relative to other symbols and the physical surroundings. In sentences, words have to positioned in the correct spatial position relative to other words in order to make sense. This is called the grammar of the writing system. The road traffic symbol system also has a grammar. Figure 2: A common example of hybrid graphic communication

Next, remove the symbols as well and we are left with a diagram a schematic representation of the roundabout. This too has conventions that have to be learned. These show that you have to go round in a clockwise direction and only the 1st, 3rd and 5th roads are exits. It is perhaps fortuitous that this research, focussing as it does on the graphic communication within a single community by a member of that community, was able to identify the existence of a distinct semasiographic language.

Graphic communication in classical mechanics

The study of the textbooks of elementary mechanics over the last 200 years showed a shift from relying on text supported by some drawings (figurative, diagrammatic and geometric) to mathematics supported by text and the free body diagram, itself a combination of diagram, symbol system and text. 92

Conceptual Understanding of Classical Mechanics

For much of the 19th century, communication relied on textual description often without the aid of pictures. When used, the pictures were in separate plates and mainly geometric in nature though there were attempts at more figurative illustration. As printing technology evolved, the pictures were merged with the text, arrows started being added to geometric depictions to indicate direction and simple mathematical expressions were being introduced into the textual description. By the 20th century, there was more reliance on algebraic equations and for linear motion at least, the geometric representation of velocity was evolving into the velocity-time graph. Sections of explanation are now followed by examples for the student to solve, with the answers at the back of the book. During the first half of the 20th century, the format of the modern text book was established. Mathematical explanations, supported by text, are followed by worked examples and there are problems for the student to solve. The picture, heavily abstracted of detail, was used to depict a physical context for the subsequent mathematical explanation. By the latter part of the 20th century, the use of vectors in schematic depiction and mathematical description had become firmly established. This is clearly a crude overview but it serves to show the general trend away from textual description supported by geometrical representation towards mathematical and textual description supported by diagrammatic depiction. The use of pictorial representation of scenarios and experimental equipment had also virtually disappeared by the early 20th century. Figure 3: The graphic communication mix in classical mechanics

This research has identified two basic types of graphic communication, which are now illustrated in the context of classical mechanics (See figure 3): 93

Vivian

The form of physical objects are communicated using depiction systems. As we move from photograph, though drawing to diagrams and finally geometry, detail is being abstracted out of the depiction, leaving only that which is required for the purpose of the communication. Concepts, having no physical form, cannot be depicted and have to be explained using some form of description system. As we move from text, through mathematics to a symbol system, the language is losing scope. The identification of a Classical Mechanics symbol system, complete with a grammar, is another outcome of this research. In the teaching of the concepts of classical mechanics, the primary means is now through the use of mathematics, a description system, supported by Free Body Diagrams, a depiction-description system hybrid.

Hypothesis of this research

This distinction between depiction and description systems of graphic communication is fundamental to this research. Description systems are able to communicate about concepts because the participants share a semantic understanding of the symbols and their syntax i.e. they share a common language. Participants can also communicate using both systems in an indirect form of correspondence i.e. metaphor, analogy and simile; allegory; metonymy and synecdoche. Discussion of this aspect of graphic communication is outside the scope of this paper. As Sampson (1985: 48) observed, the earliest writing systems evolved to make communication more effective for specific purposes. Such languages may be incomplete or restricted by today's standard of text but that does not detract from their usefulness. In engineering terms, they were 'fit for purpose'. The research into text-books over the last 200 years has identified a number of exemplars of graphic explanation that could be further developed. Similarly, the symbol system of classical mechanics, together with the use of diagram and geometry, was evolving into a very effective language for explaining the concepts of classical mechanics. This evolution appears to have ceased as mathematics took over. The quest for solutions to the resulting 'maths problem' and the quest for a 'new symbolisms' using multimedia (Kent & Noss 2003: 6) may renew that evolution. It is the hypothesis of this research is that there is a better mix (see figure 4) for teaching the basic concepts of classical mechanics. Conceptual explanation should rely primarily on the use of an enhanced form of Free Body Diagram, combining diagram, geometry and the symbol system, presented in the form of a storyboard. This can be done in sketch form or by using vector-based drawing software. Note that such software is based on the same vector concepts it is describing. When the physical scenario is absent, context can be provided by the use of figurative depiction in the form of digital photography. i.e. in the classroom, in text-book or on-line. 94

Conceptual Understanding of Classical Mechanics

When the tutor is absent, further explanation can be provided by text or audio. i.e. in text-book or on-line.

Figure 4: Suggested graphic communication mix

A computer is not essential, though modern digital technology enables this form of communication to be freely and easily reproduced. There is also evidence to suggest that students today are happier to use digital drawing software than conventional hand drawing equipment (Arthur, 2007). It should also be noted that this software enables the use of new, more meaningful, graphic manipulations that are not possible with paper and pencil. This approach should be a more effective graphic language for teaching and learning the concepts of classical mechanics because that language had been evolving for precisely that purpose. Being narrower in scope, it is much easier to learn than mathematics and being more isomorphic with the subject, it should be easier to use than mathematics.

Newton's second law of motion

Take for example, the statements of Newton's 2nd Law using text, mathematics and the symbol system (see figure 5). Newton's statement translates into mathematics as an apparently simple equation (figure 5b). However, students will only appreciate the full scope of this statement if they understand that <F> and <a> represent vectors, what the <=> sign means in terms of vectors, what multiplying a vector by a constant <m> does to a vector and that the existing velocity of the mass is irrelevant. 95

Vivian

The law can be written using the Classical Mechanics symbol system (figure 5c). Once again, the reader has to understand the meaning of the various symbols but here the significance of their spatial arrangement, i.e. their grammar, is visually obvious - they all have to be in-line. The image is explicit about the spatial relationship between force and acceleration, and their shared line of action. Figure 5: Representations of Newton's 2nd Law

Fortuitously, vector arrows are so isomorphic that if they are drawn to scale, they can be used to make calculations. The length of the arrow is a metaphoric representation of the magnitude of the property the arrow represents. Graphic calculation reached its peak at the latter half of the 19th century with Karl Culmann and Graphic Statics but there is no suggestion in this research that we should go back to those methods of quantification. However, the scaling of the arrows can be used for conceptual explanation and qualitative analysis. In this case, doubling the length of one arrow, doubles the length of the other.

Isomorphic symbol systems

For a symbol system to be isomorphic, the relationship between its symbols would need to be the same as the relationship between the concepts they denote. No symbol system can be 100% isomorphic, though some systems are more so than others. Text is not isomorphic. In Mathematics, the relationship between the symbols have isomorphic properties. Figure 6 is designed to illustrate the significance of isomorphism in symbol systems. Text is first translated into mathematics. The mathematics can be manipulated to derive a new relationship. The new relationship can then be translated back into text. Note that text, bearing no direct relationship to physical reality, cannot be manipulated in this fashion. It is this ability to derive new knowledge by the manipulation of the symbols that makes mathematics such a powerful a tool for science.

96

Conceptual Understanding of Classical Mechanics

Classical Mechanics is more isomorphic than Mathematics - its symbols are motivated and the physical spatial relationships between them correspond better with reality. Before demonstrating the isomorphic properties of the Classical Mechanics symbol system, an important aspect of the thesis of this research needs to be presented.

Figure 6: Translation and manipulation of a symbol system

In order to translate a reality into a mathematical model, the student needs to be sufficiently fluent in the language of mathematics to be able to create and manipulate that model. Reality is complicated, so inevitably simplifying assumptions about reality have to be made in order to enable the student to use the mathematical skills they possess. Using simplifying assumptions means simplifying reality and consequently limiting the depth of understanding possible about that reality. Furthermore, since classical mechanics is a very familiar reality, the simplification is obviously unrealistic and undermines the perceived relevance of applied mathematics as a subject. Mechanics, being presented as a branch of applied mathematics, may also be seen as irrelevant. Since mathematics is an esoteric language, mechanics may also be perceived as too difficult to learn. Considering that most people can learn intuitively how to kick a ball or ride a bicycle, this would seem to be a false perception. In short, the reliance on mathematics as a medium for conceptual learning of classical mechanics is itself both a limiting factor and a de-motivating factor on that learning.

Proposed scheme of graphic communication

By way of an example of conceptual explanation using the symbol system, consider a projectile moving through the air under the influence of gravity (Figure 7a).

97

Vivian

Since text represents speech, it is a one-dimensional or linear symbol system. Words are strung together to create sentences and sentences are strung together to tell stories. With a two-dimensional symbol system, the sentences are replaced by two-dimensional arrays of symbols and the story is told using a storyboard as in figures 7, 8 and 9.

Figure 7: Storyboard analysis of projectile.

Figure 8: Storyboard analysis of projectile (continued).

98

Conceptual Understanding of Classical Mechanics

Actually, there are 17 such 'sentences' in this story, which have been collated into 5 images purely for the printing convenience of this paper. Although these images have been created using digital software, the symbols are deliberately imprecise to connote that they could be drawn by hand, and the text boxes are used to connote that they could be replaced by speech. A tutor could take students through this story using a whiteboard or pencil and paper. Figure 9: Storyboard analysis of projectile (continued).

This storyboard has considered the problem of the projectile including the fact that the air creates resistance. Further analysis would explain how that resistance changes with speed, thus affecting the overall shape of the trajectory. In the analysis of projectiles, it is normal for textbooks to ignore the effect of air resistance making the mathematics simpler but in so doing making the scenario unrealistic. Figure 10: A commonplace projectile

99

Vivian

A more complex, yet common, scenario of spinning a ball to create a curved trajectory (figure 10) is even harder to analyse mathematically but it too can be explained using the symbol system.

Proposed pedagogic framework

The scheme described in the previous section is designed to replace mathematical analysis in an otherwise traditional framework for teaching engineering science. The framework is being implemented in the form of a Moodle course. Moodle is an opensource Virtual Learning Environment (VLE) based on social constructivist pedagogy (Moodle, 2008). At the moment the course is restricted to staff and students at Blackpool & The Fylde College but towards the end of the research it is planned to make it freely available on the world wide web, thus enabling access by any parties interested in this work. Figure 11: Proposed pedagogic framework

Figure 11 shows the basic structure of the course. Each Moodle course can be split into blocks of related resources and activities called topics. In this case, there are three types of topic:

100

Conceptual Understanding of Classical Mechanics

Language: Introduces the student to the writing system of classical mechanics its signs and grammar, to a restricted geometry based on vector graphics software and to a glossary of terms, supported by further semantic explanations. The glossary is automatically linked to the other blocks using hypertext. Exemplars: Many excellent conceptual explanations have been developed in the field of classical mechanics, some of which have fallen into disuse. These explanations can be translated into a more contemporary style using the writing system of classical mechanics and they can provide a basis for further conceptual development. Exercises: As with a normal engineering course, the teaching strategy uses exercises to motivate students to develop and check their understanding. As well as providing practice in graphic construction and communication, the exercises are designed to provide a context for deeper conceptual development. Participation in the exercises motivates the desire for greater conceptual understanding. The combination of the glossary, exemplar developments and exercises results in a lot of repetition of explanation. This repetition is deliberate. What does not work for a student in one context, may in another, and the plan is that with perseverance the student will be rewarded by passing through a 'conceptual gateway that leads to a transformed view' (Meyer & Land 2006: 19) of classical mechanics and of the world. In the Moodle VLE environment, the students can be allocated to groups and the on-line activities can be used in differing modes of group operation. No groups: All the students work together as a single cohort. Visible groups: Students work in groups but they can see the activity of other groups. Separate groups: Students work in groups and cannot see the activity of other groups. These modes are logistically significant as they enable disparate groups of students to share common resources. The group could be from a single school, FE or HE institution, or be comprised of individuals who share a common desire to work together to further develop their understanding of classical mechanics. Institutions can choose how they would utilize this course. On-line: The students access the resources and participate in the activities on-line. No formal class contact is at the institution. Classroom: The institution can download all the explanations and exercises (in Power Point format) and use them for classroom presentation. All activity is institution based and students do not have access to the Moodle course. Blended: The students have access to the Moodle course but their level of involvement is directed and supervised by the institution. 101

Vivian

The framework can be integrated into existing courses, which would have to specify the level of involvement expected of the student, or it can be used by students as an additional resource either before or during their study at university.

Conclusions
This research has identified that: There are two distinct forms of direct graphic communication Description and Depiction. There exists in classical mechanics a well developed but under-utilised semasiographic symbol system. The Free Body Diagram is a combination of a depiction system, i.e. diagram and a description system, i.e. the symbol system. There is a theoretical basis for the hypothesis that an enhanced form of the Free Body Diagram could be more effective than mathematics for conceptual communication of classical mechanics. A scheme has been designed and integrated into a pedagogic framework. This framework is now being offered for trial as a palliative solution to the 'maths' problem

Bibliography
Arthur, Kennedy. (2007) How dyslexia can affect an illustrator in a working environment [Unpublished dissertation]: Blackpool & the Fylde College Darling, Jos, Drew, Ben., Joiner, Richard, Lacovides, Loanna and Gavin, Carl. (2008) Game-Based Learning in Engineering Education. Proceedings of the International Conference on Innovation, Good Practice and Research in Engineering Education. EE2008 held 14-16 July 2008 at Loughborough University, England. Engelhardt, Jorg von. (2002) The Language of Graphics Amsterdam: ILLC-publications Ji, Tianjian and Bell, Adrian. (2008) Enhancing the Teaching and Learning of Structural Concepts through Experience of Seeing and Touching. Proceedings of the International Conference on Innovation, Good Practice and Research in Engineering Education. EE2008 held 14-16 July 2008 at Loughborough University, England. Horn, Robert, E. (1998) Visual Language Washington, USA: MacroVU Kent, Phillip and Noss, Richard. (2003) Mathematics in the university education of engineers London: Ove Arup Foundation

102

Conceptual Understanding of Classical Mechanics

Lee, Stephen., Harrison, Martin, C. and Robinson, Carol, L. (2006) Engineering students' knowledge of mechanics upon arrival: expectation and reality. Engineering Education 1, (1) 32-38 Lohse, Jerry, Rueter, Henry, Biolsi, Kevin. and Walker, Neff. (1990) Classifying Visual Knowledge Representations: A foundation for visual research. University of Michigan: Cognitive Science and Machine Intelligence Laboratory Meyer, Jan and Land, Ray. (Eds) (2006) Overcoming barriers to student understanding : threshold concepts and troublesome knowledge London: Routledge Moodle (2008) Philosophy [online] available at http://docs.moodle.org/en/Philosophy (Accessed 21 Jan 2008) Norman, Jesse. (2000) Differentiating Diagrams In Anderson, Michael, Cheng, Peter, and Haarslev, Volker. (Eds) Diagrams 2000: Proceedings of first international conference on Theory and Application of Diagrams Berlin Heidelberg: Springer-Verlag PRHE (2008) Curriculum change for Learning. 2nd International Pedagogical Research in Higher Education Conference. PRHE2008 held 16th - 17th June 2008 at Liverpool Hope University: Liverpool, England. Richards, Clive (1984) Diagrammatics [Unpublished PhD Thesis] London: Royal College of Arts Sampson, Geoffrey (1985) Writing Systems London: Hutchinson & Co Shimojima, Atsushi. (1999) The Graphic-Linguistic Distinction Artificial Intelligence Twyman, Michael. (1979) A Schema for the Study of Graphic Language In Kolers, Paul, Wrolstad A. and Merald E. (Eds) Processing Visual Language (Volume 1) New York: Plenum Press

103

PRIME Volume 3(2)

Encouraging Reason: A pragmatic approach to dialogic teaching in the primary school.

John Smith Manchester Metropolitan University Institute of Education

Summary
In this paper a pilot project to develop dialogic teaching within a primary school is described the background to the project and some of its implications are also described. This project has involved both school and Higher Education Institution (HEI) colleagues and has strengthened the links between the two as they have worked together, in a pragmatic, problem-solving manner, to implement pedagogical and curriculum change. PRHE Conference themes developing curriculum for the 21st century and implementing curriculum change in a traditional context.

The uneven development of oracy in the primary classroom

How does the busy teacher help children to develop the closely connected skills of talking and thinking? Common sense might suggest that they will simply develop as the child matures and as by-products of intellectual energy expended elsewhere. This contention may well be true up to a point but I would argue that the development of these skills can be greatly assisted by a carefully chosen range of strategies. It is important to contextualise current developments in classroom practice by considering relevant initiatives from recent decades. The significance of what has increasingly been referred to as oracy (as distinct from literacy) has been appreciated for many years but its development in the primary classroom has been uneven and it has often been less carefully planned for than literacy. By the 1960s, educational sociologists such as Bernstein (eg Bernstein 1973) and Barnes (Barnes, et al 1969) had begun to identify differences in language use which appeared to be related to socio-economic background and to be educationally significant. (A useful and concise summary of these developments is given in Alexander 2006.) The work of Basil Bernstein in the 1950s and 1960s is particularly relevant to the current of thought underpinning the present paper. Bernstein suggested that the spoken language which children from different socio-economic groups brought into school was educationally advantageous in the case of the middle class and disadvantageous in the case of the working class. (An exploration of the suitability of such labels for social class in contemporary society would be extremely useful but is beyond the scope of the present paper.) Bernstein labelled the language form which allowed middle class children greater educational access an elaborated code and he contrasted this with the restricted code which was the typical form of language used by children from the working class. Bernsteins early work was concerned, in part at least, with identifying the key features of these linguistic variants and for a time notably around the time that the Plowden Report (DES, 1967) was published - these ideas were widely regarded as being not only important but as relatively unproblematic. Promoted by HEIs along with other compatible theories, they began to influence attitudes and approaches to classroom uses of spoken language. 105

Smith

This development was more or less stopped in its tracks by a growing unease about the implications of Bernsteins theories, an unease voiced most famously by the American linguist William Labov (1969) who claimed that the theories advanced by Bernstein and others took insufficient account of the quality of thinking which underlay the surface appearance of non-standard English use. The linguistic deficit model, as it came to be known, was seen by many to have been discredited, despite claims by Bernstein that his theory was actually concerned with linguistic difference, rather than deficit. In the 1970s, two further developments had significant influence, for a time at least, upon primary practice. The work of Joan Tough for the Schools Council (see, for example, Tough, 1977) re-opened the quest to develop more advantageous language varieties among children and promoted interventionist strategies to boost the language skills of those children needing the greatest assistance. The notion of linguistic disadvantage seemed to have become a less contentious issue than it had been when Bernsteins ideas were debated. The Bullock Report (DES 1975) located the development of spoken English at the core of English teaching, on its own and across the curriculum, and it too encouraged a range of imaginative classroom initiatives. The National Oracy Project, which ran from 1987 to 1993, was a further spur to the development of spoken English but, as the National Literacy Strategy (DfEE 1998) and other centralised approaches to the primary curriculum took hold, the pursuit of oracy appeared once again to have become subordinated to the pursuit of literacy. In this decade there appears to have been a significant revival of interest in oracy. There is growing concern about the difficulties faced by children who enter school with poor language skills and a variety of intervention initiatives have arisen in an attempt to remediate these difficulties. (See, for example, the website of Stoke Speaks Out, a project based in an area facing acute problems of this kind.) Another initiative, highly significant and especially relevant to the present study, is that of dialogic teaching, associated particularly with the work of Robin Alexander (eg Alexander, 2006) which will be explored a little more in the following section.

Dialogic teaching and its links to thinking skills

Dialogic teaching is a label for a model which is being developed as an alternative to the dominant teaching model in UK primary schools. In the latter, childrens experience of classroom talk is often limited both in quantity and in scope. Alexander (2006) makes a powerful case for a different approach to classroom talk, one which would allow for richer dialogue hence dialogic teaching - between teacher and pupil and between pupil and pupil. Essential features of dialogic teaching (from Alexander, 2006) are that it is: Collective Reciprocal Supportive Cumulative Purposeful The term dialogic teaching does not describe one particular approach. Rather it is a set of criteria essentially those identified above which a variety of approaches can be measured against. Dialogic teaching demands a repertoire of teaching skills and 106

Encouraging Reason

techniques, all underpinned by clear educational principles, andthis repertoire probably includes such traditional and in some quarters despised procedures as rote, recitation and exposition. (Alexander 2006, p 36.) One approach which pre-dates the notion of dialogic teaching but which provides an extremely close match to its criteria is that of Philosophy for Children (P4C), an approach to the development of thinking skills. Originating in the USA and arising from the work of Matthew Lipman (eg Lipman, 1991), P4C has been developed internationally and is growing in influence in this country (see Fisher, 2003, Haynes, 2008 and the website of the Sapere organisation.) P4C endeavours to establish communities of enquiry within primary classrooms in which the P4C trained facilitator (typically the class teacher) provides an initial stimulus and children then consider questions which they identify in a deep and systematic way. Great importance is attached to the notion of reason, in its dual senses of being able to reason and being reasonable. The similarities between Alexanders criteria for dialogic teaching and the P4C approach become evident when Lipmans community behaviours (Lipman, 1991, p 52) are considered. These include the suggestions that: Members question one another Members build on one anothers ideas Members deliberate among themselves Members cooperate in the development of rational problem-solving techniques I had had a growing interest in Alexanders notion of dialogic teaching and I had been trained in P4C when an interesting opportunity arose to put these into practice in a school context.

Background to the school-based project

The school in which this project has been located is a very large South Manchester primary school, close to the university campus where I work. It is a high-performing school and its intake contains children with a wide variety of first languages and ethnic backgrounds. Its proximity to the university campus and a number of other fortuitous circumstances, including a head teacher and staff who welcome collaborative professional development work, have meant that we have worked on a range of projects over a number of years, particularly in relation to primary mathematics. (See Smith and Walsh, 2008 for further details of some of this work). Because of the close and extremely beneficial working relationships which have evolved, the Head Teacher and I negotiated a project in which I would lead sessions of P4C within the school in order to further the dialogic teaching which had already developed there. An important background factor has been the schools commitment to Kagan structures. This is an approach to cooperative learning pioneered by the US educationalist Spencer Kagan (see, for example, Kagan, 2000) and involves teachers planning very carefully the groupings, routines and other procedures affecting the classroom in order to maximise, and to create more equal access (as far as possible) to the talk and thinking which take place. Like P4C, the Kagan approach contains a strong ethical core and many of its practices appeal to childrens sense of reason in both of the 107

Smith

senses described earlier. It is therefore a compatible approach to P4C and created an extremely useful background to my own work in school. The project had three stages: Working with a group of Year 6 children on sessions of P4C. These sessions took place weekly over a half term and involved the same 12 children each time. Evaluation with children and staff with a view to further development in school in school year 2008-9. Evaluation of the project to re-assess relationships between school and HEI and trying to establish wider implications for Initial Teacher Training.

The first of these stages was completed in Summer Term 2008, the second was begun then and will be completed in Autumn Term 2008 and the third is a wider, ongoing objective.

Outcomes of the school-based project

The P4C project appeared to be very successful in terms of childrens enthusiasm. The best illustrations that I can give of the effectiveness of the project in developing childrens reasoning skills through spoken language are some extracts from one particular enquiry which I video-taped for subsequent analysis. There were two stimuli for this particular enquiry. The first was a letter from the War Office to my great greatgrandmother informing her that her son, my great uncle, was missing, presumed dead at the Battle of Ypres in 1917 and the second was an extract from the childrens novel Private Peaceful by Michael Morpurgo, which I read to them. The eponymous hero of that novel deserted at the same battle as my great uncle had died and the children devised a series of questions suggested by these stimuli. They then selected the question that they wished to discuss which was: Is it better to save your own life in war or is your countrys freedom and pride more important and why? Among the responses which children gave to this question and, importantly, to the contributions of their peers, were the following: Child A: I think its sort of a two-way questionIf you desert your country youll be known as a deserter but youll also be saving yourselfmaybe youre the last person from your generationIf youre fighting for your country like B said, you may be dying but you may be saving, like, a hundred other people Child C: Either way, if you stay or go youre probably going to get killed so if you stay youre going to get killedof your own accordbut if you desert youll be caught eventually so theres just no point. Either way youre just going to die.

108

Encouraging Reason

Child D: Im just going to try and build on F and Ds pointsOnce you go into war theres no way out because either way youre going to be scarred for life. I mean, if you stay in the battleitll scar you emotionally and if your friends get killed youll be mentally messed up inside your mind. Whereas if you were on the run, desert your army, itll still scar you for life because youre gonna feel that guilt inside you at leaving your friends to die basically. The clarity and force of these arguments are compelling and the childrens ability to take account of the views expressed by other speakers is very impressive. Additionally, in this discussion a real sense emerges of a compassionate approach to the issues under consideration. The children thought readily of the effects of war upon the participants and those close to them. They seemed very aware of the shock-waves which distant events could send out to those who were emotionally tied to those directly affected. There was, in short, real evidence of emotional intelligence (see Goleman 1996) in their dialogue. I felt that the enquiries offered clear evidence too, of Alexanders five criteria for dialogic teaching, the talk within them being collective, reciprocal, supportive, cumulative and purposeful. After the sessions had ended I interviewed the children, the class teacher and the Head Teacher, mostly on video once again to aid analysis. One of the children noted the importance of the rules as a regulatory framework to guide their discussion. Child A: If (the rule) hadnt said I will comment on the question not on the person, everyone would have commented on the person (laughing) not the question. Another child noted that: Child C: It was good howwe tried not to make assumptions about things, we all tried to be fair on each other. The Head Teacher of the school made the following comments in his evaluation of this stage of the project: Philosophy for Children is not quite the same as the normal classroom routine. The children have many more opportunities to think, many more opportunities to discuss and to engage in a dialogue and in many respects they have more opportunities to set the agenda themselves for their own learning. Another important point to note here is the extent to which these children engaged in metacognition. This ability to reflect upon thinking thinking about thinking as it is sometimes called is a high-order skill which the children demonstrated themselves capable of during the enquiries and during the evaluations. The capacity to engage in metacognition has long been recognised as a vital ingredient in the development of thinking skills and has, for example, been a key element in the school-based research of Professor Carol McGuinness at Queens University, Belfast (cited in James and Pollard, 2008). 109

Smith

The decision to extend my own project into school year 2008-9 has already been taken. It is important now to consider the way that national developments in the primary curriculum in this country seem to coincide with this development.

Current developments in policy

Somewhat confusingly, there are currently two major national reviews of the primary curriculum taking place in this country and, although neither of the final reports of these reviews have been published, there are some signs that dialogic teaching (whether or not it bears that name) is likely to be recommended more widely by each body. The Primary Review based at the University of Cambridge is directed by Robin Alexander, the author of the key text on dialogic teaching cited in this paper. Howe and Mercers (2007) Primary Review Research Survey gives a sympathetic analysis of many of the facets of dialogic teaching (which in turn subsumes some of Mercers earlier work, eg Mercer, 2000). Conroy et als (2008) study for the same review gives extensive and encouraging space to a consideration of both dialogic teaching and P4C. The other review labelled an Independent Review of the Primary Curriculum - is the one led by Jim Rose and commissioned by the present government (in contrast to the review considered in the previous paragraph) with a view to making recommendations for curriculum changes in 2011. This second review is collecting evidence at the time of writing but papers submitted to it seem to point towards a greater emphasis on thinking skills in the primary school. Although this is not the same as dialogic teaching, there is considerable overlap as I have suggested, and P4C, the approach discussed above, clearly falls within this category. One evidence dossier submitted to this more recent review states that: The primary review should consider looking at the skills frameworks which have been successfully implemented in some of the countries indicated, to ensure that thinking, learning and personal skills are sufficiently emphasised and taught across the curriculum. (QCA, 2008) It seems likely therefore, that the kind of work discussed in this paper will become more widespread over the coming years.

Conclusion: Challenges for school-HEI partnerships

The dossier cited in the preceding paragraph also states that If skills are going to be more appropriately and coherently expressed in a new primary curriculum this will have pedagogic implications which need to be considered. (QCA, 2008) Considering this issue requires a re-examination of the relationships between HEIs and schools and the ways in which the former can assist the latter within schools partnership work to support initial teacher training and continuing professional development. 110

Encouraging Reason

It seems to me that HEIs have a vital role to play in terms of ensuring academic rigour as new approaches develop. There is a danger that busy teachers will otherwise be pushed into a position where they are working with shrink-wrapped ideas, to use John Whites evocative term (White, 2005). Richard Pring has suggested that [u]niversities ought to be places of research, scholarship and critical though informed deliberation. Such deliberation need not be immediately practical, but indirectly it must be so in that any practice participates in a world of ideas and is affected by the shifts and changes within that world of ideas. (Pring, 1996, p 19) Furlong et al. make a similar point when they argue that the key strength of the HEI partners is theorising the epistemological and pedagogical underpinnings of training, so in their absence (the) complexity and contestability of professional knowledge is no longer seen to be at the heart of what partnership is about; professional knowledge becomes simplified it is essentially about contemporary practice in school. (McNamara et al 2008, p 9) There is therefore, a danger that HEIs will become too remote from classroom realities and school will appear to teaching trainees as the place that the craft of teaching is really learnt. It seems to me that we might characterise this perceived divide between the domains of school and HEI at their extremes as the domain of unquestioned answers (school) and the domain of unanswered questions (HEI). Clearly this is to caricature the situation greatly in very many situations there are extremely beneficial relationships operating across schools and HEIs - but we must remain vigilant about the possibility of such a divide opening up and we must be careful to provide bridges, in an era of constantly shifting relations, between school and HEI. The Head Teacher of the school in which my study was based made an interesting suggestion in our discussion: What I think we need to think about in schools is more creative ways of setting safe environments for students to experiment in. If they do make mistakes all teachers make mistakes no ones going to come to any harm and in fact people can benefit because we all learn from making mistakes. Small-scale interactive projects like the one examined above, which draw upon the expertise of school and HEI staff and are designed to maximise benefits for children, trainee teachers and experienced teachers, seem to me to have the greatest likelihood of success, where they can be established. This can only happen if we ensure that pragmatic, well-focussed collaboration between schools and HEIs takes place. The ALACT model described by the Dutch teacher educator Korthagen (2001) suggests one way forward and resonates with my own experiences. The ALACT model proposes a cycle based upon that acronym in which the stages represent its letters: 111

Smith

Action Looking back on the action Awareness of essential aspects Creating alternative models of action Trial This interplay between trial, reflection, refinement and re-trial seems an extremely useful model and has important implications for HEI pedagogy over the coming years. My project has therefore explored the parallel strands of school curriculum development and the negotiation of pedagogy between school and HEI. At its heart lies work coalescing around the notion of dialogic teaching, which, as Alexander suggests; ..seems to find a convincing place in the 21st century nexus of citizenship, personalization and lifelong learning. Each of these is or, once detached from its surrounding rhetoric, ought to be about empowerment of the individual: as thinker, as learner and as citizen. (Alexander, 2006, p 36)

Acknowledgements
The author is extremely grateful for the support received throughout this project from staff and children at Beaver Road Primary School in Didsbury, Manchester and to Gordon James at the MMU Institute of Education for his invaluable technical assistance.

References
Alexander, R. J. (2006) Towards Dialogic Teaching (3rd Edition), Dialogos. Barnes, D., Britton, J., Rosen, H., (1969) Language, the learner and the school, Harmondsworth: Penguin. Bernstein, B. (1973) Class, codes and control. Vol 1, Routledge and Kegan Paul. Conroy, J., Hulme, M. and Menter, I. (2008) Primary Curriculum Futures (Primary Review Research Survey 3/3), Cambridge: University of Cambridge Faculty of Education. Online. Available: http://www.primaryreview.org.uk/Downloads/Int_Reps/6.Curriculumassessment/Primary Review_RS_3-3 report_Primary_Curriculum_Futures_080208.pdf (accessed 29 July 2008). DES (1967) Children and their primary schools, London: HMSO. DES (1975) A language for life, London: HMSO. DfEE (1998) The National Literacy Strategy, London: DfEE. 112

Encouraging Reason

Fisher, R. (2003) Teaching Thinking (2nd Edition), London: Continuum. Goleman, D. (1996) Emotional intelligence: why it can matter more than IQ, Bloomsbury Publishing plc. Haynes, J. (2008) Children as Philosophers (2nd Edition) Abingdon: Routledge. Howe, C. and Mercer, N. (2007) Childrens Social Development, Peer Interaction and Classroom Learning (Primary Review Research Survey 2/1b), Cambridge: University of Cambridge Faculty of Education. Online.Available: http://www.primaryreview.org.uk/Downloads/Int_Reps/4.Children_developmentlearning/Primary_Review_2-1b_report_Social_development_learning_071214.pdf (accessed 29 July 2008).

James, M. and Pollard, A. (2008) Learning and Teaching in Primary Schools: insights from TLRP (Primary Review Research Survey 2/4), Cambridge: University of Cambridge Faculty of Education. Online. Available: http://www.primaryreview.org.uk/Downloads/Int_Reps/9.Teachinglearning/RS_2-4_report_160508_Learning_teaching_from_TLRP.pdf (accessed 29 July 2008). Kagan, S. (2001) Cooperative Learning, Kagan Publishing. Korthagen, F. A. J. (2001) Linking practice and theory. The pedagogy of realistic teacher education, New Jersey, Lawrence Erlbaum Associates. Labov, W. (1969) The logic of non-standard English, Georgetown Monographs on Language and Linguistics, Vol 22: pp 1-31. Lipman, M. (1991) Thinking in Education, New York: Cambridge University Press. McNamara, O., Brundrett, M. and Webb, R. (2008) Primary Teachers: initial teacher education, continuing professional development and school leadership development (Primary Review Research Survey 6/3), Cambridge: University of Cambridge Faculty of Education. Online. Available: http://www.primaryreview.org.uk/Downloads/Int_Reps/8.Settingsprofessionals/RS_6-3_report_Teacher_training_development_080418.pdf (accessed 29 July 2008). Mercer, N. (2000) Words and minds: how we use language to think together, London: Routledge. Pring, R. (1996) Just desert, in J. Furlong. and R. Smith 1996. The role of higher education in teacher training, London, Kogan Page, chapter 1, pp. 8-22. QCA (2008) Analysis of evidence from an international study on the teaching of skills Primary Evidence Dossier, Section 7. 113

Smith

Online. Available: http://www.qca.org.uk/libraryAssets/media/FINAL_Primary_Evidence_Dossier_section_ 7_International_review_of_the_teaching_of_skills.pdf (accessed 29 July 2008). Smith, J. and Walsh, K. (2008) Learning to teach mathematics in a whole school context, Primary Mathematics, 12 (2): 20-22. Tough, J. (1977) Talking and learning: a guide to fostering communication in nursery and infant schools, London: Ward Lock. White, J. (2005) Howard Gardner: The myth of multiple intelligences, Viewpoint, No. 16. Institute of Education, University of London.

Other websites referred to:

Sapere, the organising body of P4C in this country. Online, Available: http://www.sapere.net (accessed 29 July 2008). Stoke speaks out Online, Available: http://www.stokespeaksout.org (accessed 29 July 2008).

114

PRIME Volume 3(2)

Variation in Student Engagement: A Design Model

Ian Solomonides and Anna Reid Macquarie University, Learning and Teaching Centre

Summary
Empirical research in Design education has uncovered significant differences between how students and tutors conceptualise engagement (Solomonides and Martin, 2008). This may have implications for the increased interest and focus of measuring student engagement through instruments such as the NSSE the National Survey of Student Engagement (Zhao and Kuh, 2004) and in particular, the newly developed AUSSE the Australasian University Survey of Student Engagement (Coates, 2006). Subsequent work by the authors involving interviews with students in Australia and the UK has enabled the development of an illustrative model that may help to explain variation in the ways that students engage with their studies (Reid and Solomonides, 2007). At the centre of this model is a dimension described as Sense of Being that mediates the way in which students focus their attention on other dimensions of their studies that lead towards professional work. Ideally, as students progress through their studies they take on attributes and ideas that relate to their future profession, and this leads to a broader engagement with their studies. In this paper we explore the pedagogical implications of this model as it has the potential to help us understand the ways in which formal studies incline students towards different levels of engagement with their studies, which in turn are mediated by their expectation and experience of the curriculum.

General Background
Don Markwell, the DVC of Education, University of Western Australia, recently described a number of challenges related to the promotion of student engagement (Markwell, 2007). In doing so he drew on the work of Derek Bok (2006) and others in discussing student engagement in the context of current higher education practice. According to authors like Markwell, student engagement can be thought of as a description of the extent or quality with which students are committed and actively involved in their learning. It is, however, more than participation in classes or completion of work outside the classroom with most eureka and epiphanies probably taking place outside formal situations. Engagement can also refer to a sense of belonging fostered by such things as extra-curricular activities, and the blurring of the boundary between formal and informal student life. As such, universities might seek ways in which a community of learners can be established around both co-curricular and extra-curricula activities. This might be as simple as enabling students to work in groups or to feel part of an identifiable cohort; students who struggle often work alone or do not have a sense of belonging. Unfortunately it is sometimes difficult to inculcate a sense of belonging and inclusion. Residential education might be very beneficial but it is far from normal practice. In the absence of structures that encourage student unionism there is less incentive to be involved socially with university. Likewise, staff overburdened by routine work cannot devote time they may otherwise do to supporting engagement. Student engagement will involve and require staff engagement as well as interaction, including student to staff and peer to peer. Moreover, the quality of interaction is important and may even involve the mentoring of students by staff. Student engagement may vary over time and relative to 115

Solomonides and Reid

experience, exposure and the students position in the lifetime of their study. It may be developmental. It might be encouraged by negotiation and choice and involvement in significant pieces of work. With the tools to hand it might be important to attempt to focus, on what measures of student engagement will be most accurate and most useful to us and to act on those measures (Markwell, 2007, p.9). In this paper we focus on illumination, rather than measurement, based on discussions with students from Design disciplines about the nature of their engagement with study. We feel that there is a need to continue research work into what engagement means to students and staff if we are to understand the outcomes of measurement tools such as the AUSSE tools that presently tend to focus primarily on the cognitive and conative (i.e. effort) aspects of study. The concept of engagement is viewed in various ways in the literature and in practice depending on the philosophical and pragmatic stances taken. Broadly, these stances may be thought of as focusing either on student behaviour, including effort, time on task, and use of resources (Kuh 2006, Coates, 2006); or on socio-cultural factors, including a perceived sense of belonging to, or lack of alienation from the group (Tinto, 1993, Astin, 1999, Kember et al, 2001, Mann, 2001). The university learning community is dynamic and students transit through a number of cognitive and emotional borders: between school, the University and work, as well as through various complexities and fields of knowledge (Reid and Solomonides, 2007, Wood and Solomonides 2008). Engagement is significantly affected by the experience students have and outcomes of these transitions. (DEST, 1999, Krause et al, 2005, Kift, 2004, Rhodes and Nevill, 2004). Bryson and Hand (2008), evoking the work of Fromm (1978) and Perry (1970) call for more of an emphasis on the becoming aspects of learning, suggesting that: Taking this perspective means that we must be cautious about focusing too narrowly on one facet of learning such as deep v. surface, or learning styles or orientations, or motivation however insightful they appear, because they are insufficient to describe holistically the full individual experience of learning. We propose that engagement might be such an holistic concept. (Bryson and Hand, 2008, p.8) Following this and similar to Barnetts (2007) proposals relative to the will to learn he argues for a pedagogy in higher education that uses and applies a more affective language and approach to teaching in sustaining and developing a students will to learn and perseverance we argue that there is a need to describe engagement, evoking as we propose in this paper, a Sense of Being.

A Model of Student Engagement from the field of Design

Much higher education research in the late 20th and early 21st century focused on the creation of student-centred learning environments (Ramsden, 2003, Biggs and Tang, 2007). This focus included the development of materials, assessment tasks and group working processes that hopefully provided relevance to students. In this case relevance was seen to foster attention in particular to the subject material and generic skills. Following previous work (Solomonides and Martin 2008), the current authors conducted a meta-analysis of data drawn from discussions with student of various Design disciplines using a phenomenographic method. This resulted in the development of a model of student engagement from the point of view of Design students and is presented in figure 116

Variation in Student Engagement

1. Considering the centre or hub of the model in figure 1, here students suggested that their engagement with learning in a particular discipline was mediated by their Sense of Being. Unlike other models of engagement that tend to focus on study activities and effort, the Sense of Being describes a more affective internal relation to students learning. This view could be described as an ontological (rather than epistemological) expression. Figure 1: Elements of Student Engagement in the field of Design
Sense of Artistry Utility Problem solvers Doing

Sense of Transformation Learning Understanding Thinking

Sense of Being Confidence Happiness Imaginative Self Knowledge

Sense of Being a Designer Professionalism Community

Sense of Being within a Specific Context Becoming Belonging Involved

The students involved in the Design study recognised that their learning was essentially about themselves how they saw themselves within certain situations and environments. In turn, each of the other situations and environments contributed to the further development of this Sense of Being emphasising confidence, happiness, imagination and self-knowledge. Notably, the language that students used to describe engagement was primarily positive as they sought to describe their personal relationships and learning approaches. One of our participants articulates it this way: DEP S12: Basically it is not about money, it is not about what jobs you can get; it is about preferably doing what you want and being who you are. Another student described the positive affective dimension this way: ES 10: Being engaged with your studies means the same as being engaged to a person - the reason you get engaged (to be wed) is because of the love and passion you have for one another. If one is engaged with 117

Solomonides and Reid

his or her studies there is a love for the subject and a great passion to learn and become more and more involved with the studies as they progress and develop. In our study of Design Students, Sense of Being is central to their experience of Artistry, Designer, Transformation and Context. Two aspects - Artistry and Designer are specifically linked to the students learning and professional domains. We postulate that there could be an equivalent to these in different areas. For instance, in the domain of mathematics it could be Problem Solving and Mathematician. What is critical about these two dimensions is that one Artistry (problem solving, etc) focuses on the students understanding or experience of the core activities of that field, where the other, Designer (or Mathematician, or Architect, etc) focuses on the students becoming part of that particular community of practice (in much the same way as Wenger, 1999 describes it). The Context dimension is rather more tied to the way that the students see themselves as part of a learning situation. Finally, the Sense of Transformation represents the results of the interactions between the dimensions, an essential change in a way of thinking about themselves. By looking at the characteristics of each of the dimensions separately, we can start to infer what these may mean in relation to each other, and also within different disciplinary areas. Each of the wheel dimensions (see Figure 1) relates to a different aspect of the experience of being a design student. When the students responses were directed towards the activity of being a Designer they described a Sense of Artistry which focused on the practical utility of their work, their ability to solve problems and the actual making of an object. In a broader context than Design we might think of this being discipline knowledge and ways of working with that knowledge. Student ES11s quote below shows how artistry involves the craft of design where the utility of the made object for others use is a key component. ES 11: Engaging with your studies means to be organised with how you go about your work and to look at it from all angles, not to limit your ideas and to try and make sure you dont create something because you think its good but to make sure it appeals to a wider audience. ES 26 describes this sense of artistry as the skill of problem solving against a particular brief. The language used by the student demonstrates a strong alignment between personal goals (that is, the sense of self or being) and the making situation. The language demonstrates the strong affective components of the Sense of Being dimension. ES 26: Being able to learn the processes of design and use my practical skills and rationalism to provide solutions for a living is a dream come true. Once the brief is set I'm engaged and from then on I love having the problems to solve. Being engaged with a course that may lead to a future in the field of design is something I've dreamt of for many years. The students descriptions of themselves in differing Design related contexts, builds a picture suggesting that engagement is strongly linked with their personal identity. A slightly different way of students thinking about themselves emerges as the focus is directed towards their work within the professional field. As we suggested before, 118

Variation in Student Engagement

Wengers (1999) theory of community of practice, which includes larger or smaller elements of engagement with the community, come to a focus in this domain. Here the Sense of Being a Designer is a dimension that involves being a professional with the design community. In this category students take on the attributes of the design professional and consider themselves as initiates within that specific community: DEP 14: I think experience in design is important like for myself going to work experience, I went to [three design studios] and they have so many different aspects of creating a design, is just about experience and what you learn from it, learning from other professionals in the industry. DEP 14 makes a note of the importance of learning from other professionals. As part of this students course, work placement was integral with work-based elements subject to reflection and activity in the return to a more formal learning situation. We wonder if this real work activity was essential for students formation of their professional selves, or if it is possible for them to develop such orientations in other ways. ES 15 provides a different sort of view about professional work where the enjoyment aspects of work are emphasised. The quote below, as in many other cases in our participant group, also demonstrates an ontological response. ES 15: For me being engaged in studies or being engaged in studies is to work for fun, to be disengaged from the competition and just enjoy your work. Its rare that the success of your project escapes the rat race and becomes something actually meaningful to your life, but when it does it becomes less about stress and deadlines but more about the work and in turn the deadlines are met and the stress is minimal. Participants indicated that they were able to change their way of thinking and working when the context changed. They described a Sense of Being within a Specific Context that afforded them the ability to engage creatively with the activity to hand. In this sense, the students recognised that part of their experience of the context was how they belonged to it and the nature of their involvement. In a more formal sense, the specific contexts that participants referred to were nearly always formal learning situations. In the next two quotations that follow we see samples of belonging to a learning context and being part of the general community where their design work is appreciated: DEP S17: I think there has to be a breaking of the boundary between teacher and student; it has to be more of one level relationship rather than dogmatic teaching. I think there has to be a more a passing on of knowledge rather than enforcing that this is how you do it, this is how you should think. There should be a generation of ideas, a generation of a way of thinking rather than saying these are my philosophies and this how you shall think, this how you should put into your design practice. ES 17: To be dynamic, to act and react to, to be enthusiastic about your work and to be aware of what your work means to yourself and others. Sense of Transformation is a dimension that relates to the ways students Sense of Being is changed through learning. Here, students indicate that they are personally transformed 119

Solomonides and Reid

by their experience of becoming designers, that they develop an appreciation of the work and life of a designer over time and proximity to designers, teacher and peers, and that their modes of thinking develop. DEP 14: Personally I think spending four years here you can see the good from the bad. You still learn to experience. I think experience in design is important like for myself going to work. DEP S14 recognises that the formal elements of learning can generate a professional viewpoint (as we may have hoped), but that it is a reflected view. The Sense of Transformation dimension is about changes in ones core sense of being that are generated through reflection on the overall experience. This would lead reciprocally to engagement with learning. ES 14 and DEP 23 put it these ways: ES 14: I would say that if I feel I am engaged with my studies I am fully involved in a project, I feel happy and confident about what I am doing and most importantly I am enjoying it. It is the kind of project where I never fully stop thinking about it - everywhere I go I see things that could be relevant or useful. DEP 23: Personally I think I know I'm engaged when I spend a large proportion of my free time thinking about a topic, before I fall asleep, when I'm watching the TV, that sort of thing. I also find I think about the topic in context to other things, things I see in the news etc and try to develop personal opinions about it. In passing, we were struck by DEP 23 and a recent quote from a teacher being interviewed about engagement: HT 1 (Humanities): So its part of my being. Maybe the mechanisms of it I can switch on and off But when Im not at work, Im still thinking about of these things. Im watching TV. Im watching and Im thinking in the same terms. In these quotes from ES 14 and DEP 23 we can again observe a certain passion for the discipline and learning within it. The student stresses the vital intersections between all sorts of different activities. Likewise, ES 16 suggests that she becomes engrossed. ES 16: For me personally to be engaged with my studies is to connect on a personal level with the work I am creating. It is to be thoroughly engrossed with what I am doing, with an understanding on my personal level. The desire to learn prevails from the interest already exists, and therefore a snowball effect is created - the more engaged I am with my studies the more I want to learn. Participant ES 22 provides us with evidence for the way in which each of the categories may be related to the hub category Sense of Being: 120

Variation in Student Engagement

ES 22: Being engaged with my study to me is imperative if you wish to be successful at it. Engagement conjures up thoughts of marriage, a marriage between your own ideas and those of the discipline youre involved in. I think this is relevant as I personally believe that if you are truly engaged in something then it will naturally and subconsciously become a part of your life. As far as design is concerned I feel that it is a lifestyle choice a way of thinking and I find it very easy to consider problems and approach them as if it is a design brief which demands a solution.

Critiquing the Model

DallAlba and Barnacle (2007) describe how, in many learning and teaching approaches in higher education, ontology has tended to be subordinated to epistemological concerns (p.679). Barnett (2004) makes similar claims when he suggests that learning for, an unknown future has to be conceptualised in terms of, human qualities and dispositions over knowledge or skills (p.247). Both of these arguments place an emphasis then on the development of the self in an ontological sense. This is more than the development of the intellect and the acquisition of knowledge and skills; the question for students is not only what they know, but also who they are becoming (DallAlba and Barnacle, 2007 p.683). Learning is embodied in a Sense of Being. Barnett (2004) suggests that some teachers intuitively realise that the language of skills, knowledge and accomplishment is insufficient in fully describing the pedagogical development of students, and indeed, if students are to be equipped for an unknown future then they need to deal with uncertainty and that this cannot be developed within a system that focuses on epistemology over ontology. The problem (as articulated by DallAlba and Barnacle, 2005, 2007 and Barnett, 2004) is that in the experience of the student so much learning is fragmented and decontextualised. Reid et al. (2005) used a phenomenographic methodology in showing that students who had a better understanding of the context of their discipline were better able to conceptualise their future and their utility. Those that did not were left with a sense of confusion as described by these students: STAT 08: Dont know, dont know, dont know, no, Im not really sure. Who knows, I certainly dont, I dont know what I want to do. MAT 21: Thats funny because a lot of people say to me, Oh well, you are doing a maths degree, you going to be a mathematician or something? and Ill say, I dont know, what does a mathematician do? When I hear the word mathematician I think of, you know, Pythagoras, you know someone who is sitting in a closed room proving theories and discovering things. The model in this paper was derived from a study of Design students. The description of each of the models dimensional elements lead toward the centrality of the notion Sense of Being. The outer wheel dimensions of the model illuminate different aspects of the student Designers world. The wheel dimensions are all interpreted through the Sense of Being and can also be understood in relation to each other. We suggested that if the 121

Solomonides and Reid

context were other than Design, the dimensions may be described slightly differently. For instance, our group of students were involved directly in the context of formal learning. We wonder how much of this model would be relevant in professional work communities where production is perceived as having greater importance than individual learning. It seems that our model presents a very affirming positive view of learners. In this case, perhaps the domain of Design education has a particularly positive environment (despite our participant group coming from different institutions and different countries). Mason and Johnston-Wilder (2004) suggest that not all activity produces learning, which leads to the question of how our participant group understood the nature of learning. The wheel dimension, Sense of Transformation, provides a bit of an answer to the issue as it is a component that recognises the role of reflection as a means of both integrating the experiences and knowledge represented in the other arms of the model, and that creates a process for the outcomes of learning to be integrated with the participants Sense of Being. Our participant group were not privy to common higher education parlance. Hence, they did not articulate components of their learning or design experience that could be considered generic skills or gradate capabilities etc. Te Wiata (2006) suggest that this may be because student can sometimes be unaware that they are developing generic things as they are not the essential focus of their activity and that such recognition (and language) is often developed extra curricula. Instead, the students in our study focused on how they were developing and changing as a person in the context of their formal learning in a specific discipline. The qualities of the Sense of Being, confidence, happiness, imagination and self knowledge suggest that learners can be innately engaged with their formal studies in a way that is rarely recognised by teachers. We would suggest that many formal pedagogical structures can, in some ways mitigate the affective approach these participants have articulated. Our model also falls short as it fails to show the strong relationship between Sense of Being and Sense of Transformation. On reflection, perhaps these dimensions are more entwined, where the Sense of Transformation is a means through which the participants expand and enhance their Sense of Being. Our model also falls short as it discipline specific. Perhaps it is possible to consider the model in a more generalised way that would enable pedagogical practitioners in different areas make sense of it for their own contexts. We would suggest then, amendments to our model that recognise the more entwined ontological components in relation to the discipline specific as well as the possibility of unknown or un-thought of aspects. In figure 2 we present the amended model in which we have aligned Sense of Transformation with Sense of Being to indicate that it is the transformation of experiences that affect the students Sense of Being. Hence it is the relationship the student has with their study that affects Sense of Being, or more accurately in Barnetts (2007) terms: Such an invocation of a relational account of the student in her educational setting has merit on its side, but it is also misleading. It sets off the student from her settings We do not properly understand the student as separate from her educational settings, even if related to her educational settings. Rather, we understand the student more properly as being in her educational settings. The question is: what is the nature of that being? 122

Variation in Student Engagement

The nature of, in our terms, Sense of Being as described in the model in figure 1 involves a complexity of interaction between the Senses of Transformation, Being a Designer, Artistry and Context. But these are born out of discussions with students of Design. If we attempt to think about how these concepts might be described in more general ways and in terms applicable to higher education more broadly then perhaps we can reconceptualise Being a Designer with a Sense of Being a Professional; Sense of Artistry becomes a Sense of Discipline Knowledge; and Sense of Being within a Specific Context might be considered as a Sense of Engagement. As we have already argued, Sense of Transformation and Sense of Being remain the inherently more general dimensions and concepts used to describe the nature of being and as such deserve to be at the centre of the model shown in figure 2. Figure 2: A Relational Model of Student Engagement
Sense of Being a Professional
Reid and Davies 2002 Reid and Petocz 2004

Sense of Discipline Knowledge

Dahlgren et al 2005 Abrandt Dahlgren et al 2007

Sense of Being Confidence Happiness Imaginative Self Knowledge

Barnett 2004, 2005, 2007

Sense of Transformation Learning Understanding Thinking

DallAlba and Barnacle 2007

Sense of Engagement

Bryson and Hand 2008 Coates 2006

In figure 2, we show the core referential and structural aspects of Sense of Being and Sense of Transformation together with the relational dimensions mentioned above. Here then, engagement is shown in a relational context with other things that support engagement, at least in the study of Design. We also leave spaces for yet undescribed concepts and invite researchers and practitioners of other disciplines to consider what if anything, these might be. The model is based on our earlier meta-analysis of student descriptions of engagement in Design (Reid and Solomonides 2007) and a subsequent alignment with previously published works. This has led toward the broader ontological perspective represented in the model in figure 2. Indicative research publications are shown in support of each of the relational dimensions. 123

Solomonides and Reid

Conclusion
When we return to the issue of student engagement and the starting point of this paper we believe the evidence presented here and elsewhere (c.f. Reid and Solomonides, 2007, Bryson and Hand, 2008) is aligned with Barnetts assertions that, pedagogical being is constructed around senses and feelings (2007, p.30), and that, ontology trumps epistemology (2005, p.795). This has implications not only for the measurement of engagement but also for the desire to achieve, an ontological turn for higher education (DallAlba and Barnacle, 2007) and suggests that higher education practitioners need to develop their curricula, skills and pedagogies that engenders the Sense of Being and Sense of Transformation as at the centre of the student experience. How possible this is in the age of decontextualisation, pseudo-scientific performance measures, and technorationalism, or how this extends to other disciplines and constructs of knowledge, remains a concern but we are encouraged by utterances from teachers such as this: DT 14 (Art and Design): maybe pandering to preferred learning types is wrong this is pivotal for me because I think of learning as change, and I am very interested, at a more advanced level, in how we can get learners to learn how to learn, in particular on the course I teach on, the answer is unknown beyond a generic, so I try to get students to understand what Ophelia states in Hamlet, we know what we are, but know not what we may be.

124

Variation in Student Engagement

References
Abrandt Dahlgren, M. Petocz, P. Reid, A. and Dahlgren, L.O., (2007). Preparation for professional work? A meta-analysis of two international research projects on the transition from higher education to work life. 5th International Work and Learning Conference, Cape Town. December. Online. Available HTTP: http://rwl5.uwc.ac.za/ (accessed 12 May 2008) Astin, AW., (1999) Student Involvement: A Developmental Theory for Higher Education Journal of College Student Development, September/October 1999 (originally published 1984), Vol. 40, No. 5, pp.518-529, Online. Available HTTP: http://www.middlesex.mass.edu/TutoringServices/AstinInvolvement.pdf (accessed May 30 2008) Barnett, R., (2004) 'Learning for an unknown future', Higher Education Research & Development, 23:3, 247-260 Barnett, R., (2005) Recapturing the Universal in the University Educational Philosophy and Theory Vol. 37, No. 6 pp.785-797 Barnett, R., (2007) A Will to Learn: Being a student in an age of uncertainty, Open University Press and Society for Research in Higher Education: Maidenhead Biggs, J. and Tang. C., (2007) Teaching for Quality Learning at University, Open University Press Bok, D., (2006) Our Underachieving Colleges: A Candid Look at How Much Students Learn and Why They Should Be Learning More. Princeton University Press. Bryson, C. and Hand, L., (2008) Student Engagement- SEDA Special 22, Staff and Educational Development Association: London Coates, H., (2006) Student Engagement in Campus-based and Online Education: University connections Routledge, Abingdon Chickering, AW. & Gamson, ZF., (1987) Seven principles for good practice in undergraduate education American Association of Higher Education Bulletin 1987, Vol. 39, No. 7, pp3-7. Online. Available HTTP: http://www.uis.edu/liberalstudies/students/documents/sevenprinciples.pdf (accessed 26 April 2008) Dahlgren, L. O. et al., (2005) Students as Journeymen between cultures of education and working life. Online. Available HTTP: http://www.hewl.net/HPSE_CT-20010068__final_ju.pdf (accessed 29 May 2008) Dall'Alba, G. and Barnacle, R., (2007) 'An ontological turn for higher education', Studies in Higher Education, 32:6, 679-691

125

Solomonides and Reid

DEST., (1999) Transition from Secondary to Tertiary: A Performance Study. Department of Education, Training and Youth Affairs Online. Available HTTP: http://www.dest.gov.au/archive/highered/hes/hes36.htm (accessed 17 May 2008) Fromm, E., (1976) To have or to be? Harper and Row: New York Kift, S., (2004) Organising First Year Engagement around Learning: Formal and Informal Curriculum Intervention, Australian Disability Clearinghouse in Education and Training, Online. Available HTTP: http://www.adcet.edu.au/uploads/documents/Sally%20Kift_paper.doc (accessed 21 May 2008) Kuh, GD., (2006) Making Students Matter in J.C. Burke (Ed.), Fixing the fragmented university: Decentralization with Direction. pp.235-264 Bolton, MA: Jossey-Bass. Online. Available HTTP: http://cpr.iub.edu/uploads/Kuh%20Chapter%2010%20for%20Burke%202006.pdf (accessed 12 May 2008) Kember, D. Lee, K. & Li, N., (2001) Cultivating a sense of belonging in part-time students International Journal of Lifelong Learning Vol. 20, No. 4 pp.326341. Online. Available HTTP: http://taylorandfrancis.metapress.com/index/PW0T4412UYK94WC7.pdf (accessed 29 May 2008) Krause K-L. Hartley R. James R. and McInnis C., (2005) The First Year Experience in Australian Universities: Findings from a decade of national studies Department of Education Science and Training and Centre for the Study of Higher Education, University of Melbourne. Online. Available HTTP: http://www.dest.gov.au/NR/rdonlyres/1B0F1A03-E7BC4BE4-B45C-735F95BC67CB/5885/FYEFinalReportforWebsiteMay06.pdf (accessed 29 May 2008) Markwell, D., (2007) The Challenge of Student Engagement, Key-note address Teaching and Learning Forum 2007, University of Western Australia, 30-31 January 2007. Online. Available HTTP: http://www.catl.uwa.edu.au/__data/page/113019/Page_6-15_from_CATLyst.pdf (accessed 29 May 2008) Mann, S., (2001) Alternative Perspectives on the Student Experience: alienation and engagement Studies in Higher Education Vol. 26, No. 1 pp7-19. Online. Available http://taylorandfrancis.metapress.com/index/BWD5LDH5KFG6PJ45.pdf HTTP: (accessed 14 May 2008) Mason, J. and Johnston-Wilder, S. (2004). Fundamental Constructs in Mathematics Education. Routledge Falmer: New York. Perry, W., (1970) Forms of Intellectual and Ethical Development in the College Years: A scheme, Holt Rinehart and Winston: New York 126

Variation in Student Engagement

Ramsden, P., (2003) Learning to Teach in Higher Education, Routledge Reid, A. and Solomonides, I., (2007). Design students experience of engagement and creativity. Art, Design and Communication in Higher Education. 6(1) 27-39. Reid, A. Wood, L.N. Petocz, P. & Smith, G.H., (2005). Intention, approach and outcome: University mathematics students conceptions of learning mathematics. International Journal of Science and Mathematics Education, 3, 567-586. Reid, A. and Davies, A., (2002) Teachers' and Students Conceptions of the Professional World, Improving Student Learning Theory and Practice - 10 years on, Oxford Centre for Staff and Learning Development Reid, A. and Petocz, P., (2004). The Professional Entity: researching the relationship between students conceptions of learning and their future profession. In C. Rust (ed.), Improving Student Learning: Theory, research and scholarship, Oxford Centre for Staff and Learning Development, 145157. Rhodes, C. & Nevill, A., (2004) Academic and Social Integration in Higher Education: a survey of satisfaction and dissatisfaction within a first-year education studies cohort at a new university Journal of Further and Higher Education Vol. 28, No. 2, May 2004 pp.179-193. Online. Available HTTP: http://taylorandfrancis.metapress.com/index/T7Y71PFT3XTLXRYJ.pdf (accessed 14 May 2008) Solomonides, I. and Martin, P., (2008) All this Talk of Engagement is Making me Itch in Bryson, C and Hand, L., (2008) Student Engagement- SEDA Special 22, Staff and Educational Development Association: London Te Wiata, I., (2006) Generic attributes and the first job: graduates perceptions and experiences. In Hager, P. and Holland, S., (eds.) Graduate Attributes, Learning and Employability. The Netherlands: Springer. Tinto, V., (1993) Leaving College: Rethinking the Causes and Cures of Student Attrition, University of Chicago Press, Chicago. Wenger, E., (1999), Communities of Practice: Learning, Meaning, and Identity, Cambridge University Press: Cambridge Wood, L. and Solomonides, I., (in press - 2008) Different Disciplines, Different Transitions Mathematics and Education Research Journal Zhao, C. and Kuh, G., (2004) Adding Value: Learning Communities and Student Engagement. Research in Higher Education. 45(2):115-138.

127

PRIME Volume 3(2)

Action research cycles on embedding academic skills: how current pedagogical research can steer curriculum
Peter Redding University of Wales Institute, Cardiff School of Management

Summary
Action research provides a useful framework for analysing topics and, perhaps more importantly, bringing about change within an organisation or personal practices. Proponents advocate a methodical approach to analysing information and acting upon it, often in a series of cycles of continuous improvement. The subject under scrutiny is one facultys attempt to enhance the academic skills of their undergraduates. The value of this is well established in pedagogic literature. However, an effective method of delivery within the context of real-world constraints is perhaps less well understood. There continues to be debates over whether to incorporate academic skills within or external to the curriculum. The faculty is completing its third year of experimenting with approaches, with each cycle generating strong themes that have been linked to current pedagogical research, which have then informed the modifications. Early cycles produced reflections on basic content and delivery, with practices grounded in pedagogic theory. Further cycles produced concerns over student instrumentality. Practical responses to such instrumentality show the compromises that are often made when educational aspiration meets student disengagement. Looking toward external practice, a qualitative analysis of a popular discussion board used by learning support staff across the UK provided interesting insights to recurrent themes including effective pedagogic approaches, organisational/management/resource issues, and even emotional response to perceived roles within the educational process. More recent cycles and reflection have expanded the list of critical issues. While pedagogic theory helps us understand effective methods for learning and teaching, management theory helps understand how organisations communicate and manage their resources to achieve common goals.

The Value of Action Research

Action research is an ethos and a methodology that distinguishes itself from the more traditional approaches of Positivists and Technical Rationalists. Whereas a conventional scientific methodology often requires a detached, objective application of procedures and emphasises repeatability, action research fully acknowledges the subjective and messy nature of real-world problem solving. It is no surprise that it was first used within the field of management, where the subject (organisations and people) lend themselves to the more qualitative approaches found in social sciences. Proponents such as Lewin advocate a methodology which fully accepts that the researcher is part of the phenomenon being studied (Saunders et al. 2003; Schon 1991). It acknowledges that we learn by doing an educational tenet that can be traced back through Piaget and Dewey (Anderson 2005; Field 2007; Geen 2001). Schn (1991) refers to practitioners going through a process of knowledge-in-action that allows us to observe from within. 129

Redding

Through methodical reflection we can start to build theory that can be applied to problems being researched. At its simplest, the process of action research can be described by the following steps: formulating a problem, reconnaissance of information, formulating action, implementing, monitoring, evaluating, and repeating the cycles (Saunders et al. 2003). This process is similar to Demings Plan, Do, Study, Act a mantra which every management student knows as a formula for implementing quality systems within an organisation. This then highlights the true value of action research: an emphasis on bringing about change within an organisation. Historically, action research applied to management issues. However it is increasingly finding a home in educational and pedagogical research. It therefore provides a useful and appropriate framework for analysing and modifying curriculum design in higher education.

Background to Study Skills

Study skills are part of a wider agenda of skills. The recent publication of the Leitch Report (HM Treasury 2006) re-emphasises the governments interest in the role of higher education (HE) in developing skills in addition to simply subject-specific knowledge, thereby enhancing employability. For years, pedagogic researchers and governmental organisations have been defining and refining skills. By the end of the 20th century, pedagogic discussions were awash with a bemusing range of permutations on a central theme, using interchangeable terms such as key, common, transferable, academic, learning and other skills. The National Advisory Board for Public Sector Higher Education (NAB/UGC 1986) used the term transferable skills to refer to those skills that needed to transfer with the student from the world of academia to the professional world. Bridges and Hurley (1993 and 1994 as cited in Cottrell 2001) respectively referred to meta-skills and core skills to describe those skills that could be adapted across different contexts. These concepts influenced the governmental educational bodies who in turn began to codify the terminology. In the UK, the Qualifications and Curriculum Agencies (QCA) is the national body responsible for developing the national curriculum and accrediting qualifications. Since the 1980s they have opted for the term key skills, which have now settled into the following broad categories: Application of number Communication Improving own learning and performance Information and communication technology Problem solving Working with others. (Qualifications and Curriculum Authority 2004) Practitioners within HE are not (as) bound by national curricula and therefore have latitude in interpreting how to implement the range of skills. The Quality Assurance Agency (QAA) does have certain expectations of embedding key skills into curricular 130

Action Research Cycles

design as part of the validation and review process (Quality Assurance Agency 2006). However it is debatable just how explicitly these skills are to be implemented in the curriculum. Nevertheless, the term study skills, per se, is not as defined and would appear to be more of a term of practice rather than one of standardisation and regulation. Within the context of this research, it has been interpreted locally to be: those skills needed by first-year undergraduate students for success at university. The evolved approaches to interpretation and implementation are discussed below.

Setting for Action Research

The subject of scrutiny is a first-year module historically named Professional Development 1, Research Methods 1 or Research Skills 1, depending on the programmes in which it originally appeared. It is a 10-credit module, giving some indication of the resources and time allocated to it. It serves a School of Management within a post-92 university, with a yearly undergraduate intake of around 500. The module operates within a matrix where some modules are shared across a range of programmes covering business, management, computing, tourism, hospitality, events, etc. Recent reorganisations have resulted in rationalisation of some first-year modules with a move toward generic modules. In developing the curriculum, there had been considerable debate over whether to develop study skills within a standalone module or throughout the curriculum a decision which had to balance available space within a curriculum, the prevailing pedagogic preferences for embedding skills, and, to a considerable extent, staff engagement with underlying precepts. Acknowledging an element of political expediency, the compromise resulted in a stand-alone, generic module that allowed for the possibility of subject-specific tailoring. The module addressed the following study skills: goal setting, time management, accessing academically legitimate sources, referencing, plagiarism, communication (written and verbal), reflection (rudimentary). Explicit action research was carried out over the last 3 years. These action research cycles have resulted in a steady evolution of the module. The more recent cycles have seen fewer changes in content, and more changes in delivery patterns and learning and teaching (L&T) strategies. The analyses have produced several major themes for exploration. The discussion which follows is admittedly revisionist at times, as some of the themes did not fall neatly into chronological order. Nevertheless, the analysis is presented using the following template, which corresponds to the iterative methodology described above. The Issue: in classical action research terminology, this is often the problem to be solved, or the criteria for change intervention. The Research: this includes findings from literature reviews, analysis of primary or secondary data, etc. The Debate: while this does not correspond to stages described in most approaches to action research, it was found to be a useful categorisation because it highlights the practical aspects of coming to a decision within an organisation 131

Redding

which aspires to collegial and collective decision-making, not to mention healthy debates. The Decision: this describes the practical steps to bring about change. The following discusses each of the cycles and their resultant themes.

Action Research Cycle 1

The Issue: Content Refinement Having completed a year of delivery, there were some questions over whether the contents of the module were appropriate, and whether the scope of the module was realistic and/or ambitious. And while the tutors had a certain amount of knowledge-in-action surrounding study skills, there was room for grounding the practice with theory. The Research: Study skills are inextricably bound to the theory and practice of education, and are well represented in academic literature. The study of education has roots in philosophy, with pundits tracing theories back through Plato, who framed many of the basic questions of epistemology, (Phillips 2008; University of London 2005) and Socrates who advocated formulaic processes for deriving knowledge (Geen 2001; Metaphysics Research Lab 2005). The field of psychology and, more specifically, cognitive development start to address the issues of how and when children learn. And finally, the field of pedagogy draws on many fields to focus on the science of teaching. Three of the more influential theorists of the 20th Century were Piaget, Vygotsky and Dewey. Piaget put forth theories of developmental stages that a child would go through while acquiring knowledge (McNally 1974; Smith et al. 1997). Perhaps more relevant to HE study are his later theories of constructivism and schemata which describe processes where learners acquire knowledge by applying it to existing understandings and mental constructs. Piaget also offered theories of equilibration, where students go through periods of cognitive disorientation before understanding new concepts. This phenomenon will surely ring true for anyone witnessing students struggling with metacognitive concepts of critical analysis and reflection found in HE. Vygotsky built upon theories of Piaget, yet emphasised the importance of social interaction in acquiring knowledge. His theories highlighted the need for a structured approach in helping students achieve the higher cognitive skills, with implications for curricular design and concentration on specific skills (Cottrell 2001). Dewey also contributed to the philosophical debates of epistemology, advocating the more heuristic approaches which involve discovery methods or problem-solving methods (Field 2007; Geen 2001). Throughout the 20th Century, these theories influenced primary and secondary education, often through the mechanisms of national curriculum. However, it wasnt until late in the century that the concepts of pedagogy made significant headway in HE. Academics began to advocate a move away from universities as centres for transmission of knowledge through standard didactic practices, and toward a managed process of learning (Ashcroft and Foreman-Peck 1994; Cottrell 2001; Gibbs and Jenkins 1992; Ramsden 1992). Within this managed process, there was emphasis on the difference between surface learning and deep learning, the importance of contextualised learning, and an 132

Action Research Cycles

acknowledgement of the meta-cognitive aspects of learning to learn. And all of this was framed in a student-centred approach. The tools for achieving this new ethos in HE include an array of assessment techniques which move away from reliance on summative and toward more diagnostic and formative methods. The very nature of these discussions often requires proponents to speak in generic terms, i.e. not within any particular subject. In other words, the techniques and underlying theories for effective pedagogy in, say, engineering should also apply in psychology. The language of generic pedagogy often revolves around skills. It could be argued therefore that skills become the new currency in helping HE implement the new approaches and designing a curriculum. Turning to the more pragmatic issue of how and which study skills to deliver, there is no shortage of advice. Targeted funding of L&T in HE in 90s led to the establishment of bodies with the remit to disseminate best practice in the UK, most notably the Institute for Learning and Teaching (predecessor to the current HE Academy) and their related Subject Centres. More recently, significant funding has been given (England only) to set up Centres for Excellence in Teaching and Learning (CETLS). These organisations regularly publish guidance and case studies on a wide range of pedagogic issues, including study skills (HE Academy 2008). The case studies provide evidence of not only the widespread efforts to bolster study skills, but also the range of techniques and resources used. The Debate: Based on the research (and knowledge-in-action) the module team was satisfied that the contents of the study skills module were appropriate and in line with national and international initiatives. The modular structure was fixed, and the issue of free-standing versus embedding was off limits to debate. With such an ambitious and crucial remit of improving study skills, the debate soon became about choosing and focussing on the most important learning outcomes and re-visiting some of the more complex concepts. Rather than characterising this as a form of dumbing down, emphasis was placed on stripping down. Ideally, a module within a curriculum should offer a process where deep learning occurs and students undergo personal development. In practice however, modules easily become units of transmitted knowledge. Stripping down the contents to the bare essentials and a series of disparate topics creates a tension in pedagogic aspirations. Nevertheless, some consolation is offered by Wingate (2006) who makes an interesting semantic distinction between study skills and study techniques. Skills, she argues are best acquired through a process of deep learning within subject-specific content, whereas techniques include those little tricks such as formulas for structuring essays or learning the conventions of referencing. The idea is reinforced by Hattie (1996) who describes some of the necessary approaches to study skills as simple mnemonic performance. Therefore, creating a module that concentrates on techniques might be justifiable if acknowledging that true skills development will occur within other parts of the curriculum. The process of paring down the modular content raised another interesting debate over the place of pedagogical theory within the curriculum. Much has been said recently about the way in which research can inform teaching (Jenkins 2004). It is only natural 133

Redding

(and desirable) that those who research a subject inject their enthusiasm for research and their knowledge into the curriculum. Again, it is desirable that those who teach study skills should have a grounded understanding of pedagogy. It does raise the questions however of whether study skills modules should teach the students theories behind learning. While a strong understanding of assessment techniques and learning styles is undeniably valuable in creating a module, it is questionable whether the topics need be made explicit to students. Students of education will have need of such concepts in their careers, however other students may wrestle with the concepts and vocabulary at such an early stage of their academic journey. The Decision: Therefore, in refining the contents of the study skills module, the team undertook an exercise which streamlined the learning outcomes, overall and for each session. At times it was a painful triage, especially when it came to those topics which directly related to the science of pedagogy. For example exercises in identifying preferred learning styles, e.g. Honey and Mumford (1992), were activities that offered students the opportunity to reflect on their own learning and could arguably start them on their way toward meta-cognition. However, the time devoted to such reflective activities needed to be directly compared to the time needed for drilling home the fundamentals of accessing academically legitimate sources and defining plagiarism. In the end, less emphasis was placed on any explicit mention of pedagogic techniques, e.g. students were not to be exposed to terms such as formative or cognitive development or deep learning. The design of the learning journey became more implicit to the students. The streamlined contents were also re-formatted so that there was a consistent pattern of delivery that included concise learning outcomes, in-class activities in small groups and out of class activities and formative assessment on a virtual learning environment (VLE).

Action Research Cycle 2a

The Issue: Engagement/Instrumentality/Utilitarianism Being relatively satisfied with the module contents and delivery methods, the lecturing team remained concerned over some students level of participation with the module. Across academia, there is widespread talk of students not taking their studies seriously, not seeing the relevance of some content, not putting in the time, or not attending. There is often the perception that some students will do the minimum amount of work to receive credit. This phenomenon is often referred to as instrumentality or utilitarianism. The Research: Philosophical and Pedagogical Aspects of Instrumentality Much has been made of the changing nature of students, within the context of the widening participation agenda or the rapidly changing nature of a globalised society (DfES 2003; Gibbs and Jenkins 1992; House of Commons 2001; Yorke and Longden 2004). However few people can say with any certainty why many students appear to be taking it less seriously. Disengagement and instrumentality soon become the subject of philosophical debates. At its simplest, instrumentalism refers to the ends justifying the means. A student may well see the university experience as nothing more than the means, i.e. a serious of tasks that must be endured, in order to achieve the ends of the larger prize of employment. A students instrumental approach often places them at odds to the expressed desires of 134

Action Research Cycles

those who design and manage education where the ultimate end would be a life-long commitment to learning and thinking. Instrumentality is a concept that helps explain behaviour of individuals as well as institutions. Philosophers such as Immanuel Kant and David Hume invoked it in the 18th Century when exploring ethics, morality and value judgements (Denis 2008). As with utilitarianism and pragmatism, the morality of choosing a particular path is seen within the context of the morality of the end goal. It is ironic however that the term instrumental now has pejorative overtones when applied to education. Paradoxically, John Dewey used it extensively in the mid 20th Century when justifying his influential approaches to educational reform (Anderson 2005; Field 2007). Of course this can be explained by the different ends that were being debated. For Dewey, the end was a meaningful education, i.e. an educational system and curricula which take into account our understanding of cognitive development and societys preferences for morality. For many students, the end may be the less philosophical goal of earning a living. It is questionable of course whether students will be engaging with the more conceptual aspects of amorality and pragmatism when making the decisions of how much time and effort to devote to their studies. Whatever the root cause of instrumental and utilitarian tendencies, they run counter to our professed goal of independent learning. These philosophical debates may or may not bring us closer to the real-world problems of students disengagement. More practical (or pragmatic) solutions lie in pedagogic discussions. Engagement can be directly related to L&T strategies, and educational policies. Some point the finger to the approaches adopted in primary and secondary education, and the institutional systems behind the policies, where education is characterised by spoon-feeding and an over-emphasis on preparing for examinations and achieving higher ratings in league tables (National Audit Office 2002; Wingate 2007). Such characterisations may seem a bit divisive, running the risk of school teachers being understandably defensive, given that increasingly there have been attempts to counter instrumentality at earlier stages (Jeffrey 2003). Within HE, the debate of instrumentality is directly related to the debates over L&T strategies, and specifically those involving a move away from the more traditional didactic approaches (as discussed above). It is not that alternatives to the sage on a stage are more entertaining, but rather these alternative approaches address the issues of surface and deep learning, contextualised study, learning to learn, etc. Getting students engaged with a meaningful process shifts the focus away from seeing education as a means, and refocuses on education as an ends itself. Pedagogic researchers talk about spending more time within the curriculum to explicitly align student expectations (Lowe and Cook 2003; OECD 2002; Sander et al. 2000). They are, in effect, talking about aligning the ends, in an instrumental sense. The Debate: Neither the philosophical musings on human nature, nor the blaming of governmental and institutional policies will bring about the necessary changes at the modular (or perhaps even the curricular) level in their first semester. The central debate then becomes whether to utilise the students utilitarian tendencies, where learning is reduced to a rather Pavlovian model of learn this receive this credit or whether to concentrate efforts on the very difficult weaning process of producing autonomous 135

Redding

learners. Alignment of expectations implies a certain level of meta-cognitive skills and a shift away from absolutist approaches to learning a tall feat according to the psychologists and educationalists mentioned earlier. Again modular design and content need to be seen within the context and constraints of a first-semester 10-credit module. The Decision: In the spirit of amoral pragmatism, the module leaders decided to take advantage of students instrumental approach. The assessment of the module was adjusted so that 25% of the mark was based on participation, as defined by attendance and completion of formative assessment in the VLE. The move was admittedly at odds with overall goals of independent learning. It was however an acknowledgement that the majority of the students had just come from a learning environment that was highly structured and tutor-led, and that in their first semester, students would be subject to numerous changes. The content was further refined to include more on-line formative assessment, with the opportunity to retake any of the tasks throughout the year. The decision was also taken to front-end the study skills module by offering it in an intensive academic induction week. In order to mitigate against further perceived separation of skills from the core curriculum, the module would be delivered by programme-specific staff who had undergone staff development on its contents with specific opportunities for tailoring for their subject. By using the same staff who would be teaching the students throughout the year, there would be an increased chance of vicariously embedding the skills throughout the curriculum.

Action Research Cycle 2b

The Issue: The Community of Learning Developers While the pedagogic discourse gives us a grounding for embedding or enhancing study skills, and guidance gives us practical tools to use, there remain numerous questions over the practical aspects of managing the process within universities which are notoriously cash-strapped and difficult to manage. Pedagogic conferences and workshops provide plenty of anecdotal evidence that most universities are experiencing similar issues when it comes to study skills and support. While learning development is a central responsibility of all lecturers, there appears to be an emerging community of those whose jobs are specifically focussed on developing the more generic aspects of a curriculum. An analysis of this community can provide insight to help the reflective practitioner. The Research: One example of this emerging trend is the Learning Development in Higher Education Network (LDHEN) and their subsequent Association of Learning Development in Higher Education (ALDinHE). The organisation holds an annual symposium and hosts an on-line discussion forum, providing a database of evidence for the many issues that confront the community (ALDinHE 2008). The discussion list spans five years, over 500 subject headings, and over 1000 postings. In addition, the organisation periodically compiles data on the profile of its members, indicating the mechanisms and structures within the universities for supporting student learning. They are also in the process of producing an in-depth qualitative analysis of the postings (Cash and Hilsdon 2008). The analysis discussed below is perhaps less sophisticated in its qualitative methodology than that proposed by ALDinHE, yet correspondences with its authors indicate similar approaches and findings. 136

Action Research Cycles

In order to identify the major issues confronting the community of learning developers, the threads with the most postings were identified, from the inception of the listing until December 2007. This crudely measured the overall interest in a subject. It is recognised that threads often veer from the original topic, and that the overall population of potential contributors was expanding through the years. Nevertheless such a semi-quantitative approach allows themes to emerge. The list of the more popular threads was originally segregated into 10 themes. Those issues concerning the running of the organisation, or general announcements were discounted. The remaining themes were then placed into the following four broad categories. The How-To Issues As would be expected in an emerging field, participants spend a good deal of time sharing tools and requesting advice. Recurrent topics included educational resources that could be used to improve writing, referencing, not plagiarizing, supporting dyslexia, diagnostic and formative assessment, motivation and engagement, etc. As discussed above using Wingates terminology, many of the resources would be aiming at study techniques. Many of the contributors to the discussions operated from a central support unit, rather than subject-specific departments. While a specific correlation between the location of the staff and the types of techniques discussed was not attempted, the discussions highlighted a significant amount of work being done to address skills outside the curriculum. As for delivery of these study skills and techniques, there was much emphasis on VLE and other mechanised approaches. This of course highlights limited financial resources in the face of large numbers, and the desire of organisations to do things economically. Conceptual Issues There was much discussion over concepts of education and learning, many highlighting semantic distinctions between teaching and developing learning. In many ways, these discussions act as a mechanism for values clarification within the new community. They reflect the evolving interpretations of HE and, perhaps more importantly, the roles of lecturers and others within the process. Organisational Issues Members showed a good deal of curiosity over how their respective institutions organised and managed their learning development activities. Some discussions compared institutional policies and rules, others compared where the activities were located within the organisational structure, whether the support was central or local within departments. One particularly interesting thread discussed the profile of a learning developer and how an individual evolves in their career. This highlights that L&T is increasingly recognised within the management structure of universities, and the career ladders. Whereas research has been the traditional path to advancement, universities are at various stages of recognising the contribution of individuals dedicated to L&T (DfES 2003).

137

Redding

Emotional Issues Related to the last point on career advancement, there were some recurrent themes that reflected an amount of frustration or marginalisation. It would appear that many participants felt that learning development in general, and centralised support in particular, are perceived as marginal activities within the organisations. Worse still, there are similar feelings of offering a service that could be perceived as a deficit model, where underperforming students are passed on to study with others needing remedial help. These views would seem to indicate that many of the emerging concepts of pedagogy and the value of skills enhancement as discussed above are not shared within organisations, and that some lecturing staff still cling to traditional approaches. Not surprisingly there emerges a parallel between L&T support residing outside of traditional management structures and the enhancement of skills outside the traditional curricular structures. The Debate and The Decision: The exercise in reviewing external practices provided valuable benchmarks for comparison. While this particular action research cycle did not results in any significant changes to internal practice, it did instil confidence that current efforts were on the right track, and that many institutions were feeling the same growing pains.

Action Research Cycle 3

The Issue: The Silo Effect As evidenced in the discussion above, there is an increased interest in universities providing additional learning support, often by central departments or library divisions. It too was the case in the university under scrutiny. Coinciding with the development of the study skills module, the university was funding a series of posts to develop a strategy for and implement additional learning development. These efforts were in addition to other departments with similar remits, e.g. an L&T development unit, various libraries with their advisors and other staff in drop-in help desks. Organisationally, there was a significant amount of overlap in responsibilities. The term the silo effect adequately describes the situation where, within a large organisation, there is less-than adequate communication and coordination between functional units. The Research: Organisational Communication The phenomenon of right hand not knowing what the left hand is doing is well reflected in literature for business studies or organisational behaviour, whether described as compartmentalisation or ineffective internal communication. Pettinger (1996) recounts the traditional approaches to hierarchical management structures, whether pyramid or flat, and the corresponding challenges to communication between units. Hardy (1993) even identifies poor communication, laterally and vertically, as major sources of conflict and barriers to efficiency. Both point out the almost inevitable tendency for compartmentalisation where units turn inward and concentrate on pursuing their own objectives, influenced greatly by internal targets and opportunities for individual career advancement.

138

Action Research Cycles

Academics are never short of voicing disapproval of their own management and management style, and much as been said recently about managerialism in HE (Deem et al. 2007). However, a more fundamental debate is taking place surrounding century-old approaches to corporate organisation. Many point to the limited life expectancy of traditional corporate structures which can be traced directly back to the theories of Adam Smith who championed the division of labour and increasing specialisation (Hammer and Champy 2001; Handy 1993). Great strides were made in the industrial revolution by compartmentalising the workforce so that they could become increasingly efficient in fewer areas of the manufacturing process. These theories were put into place with much success by Henry Ford and his mass production. The ideas were given even more credence when Taylor and others applied scientific principles to maximising efficiency (Cole 2004). However, with the increases in competition, the dominance of the consumer, and the rapidity of changes brought on by globalisation and technology, the limitations of traditional corporate structures are becoming apparent. Modern corporations need to respond quickly to change in order to survive. Traditional structures are seen to mediate against this. Hammer (1997) puts forth the ideas of reengineering the corporation, where the emphasis is not on specialisation, but on the overall processes of achieving a satisfactory product or service. The ideas behind reengineering represent a continuation of previous initiatives such as Total Quality Management (TQM) which claim that the solutions lie in more fundamental changes in the ethos of organisations, whether its placing the customer at the centre of all decisions (Aguayo 1990) or focussing on the role and location of all employees within the process (Hammer 1997). Either way, self-contained functional units tend to focus inwardly, and not on the bigger picture. Theories of reengineering have resonance with HEs attempts to improve learning. It could be argued that proponents of deep learning are advocating a move away from a strict reliance on specialisation, whether in the form of subject specialists or even external specialist of learning development. A sound curriculum would be delivered by a team with knowledge of their field, plus knowledge of the pedagogy. And from an operational standpoint, process orientation would suit an academic organisation with its historical departmental divisions between academic, administrative, library and support staff. If the acquisition of factual information and the cognitive development of a student is the overall process that is to be achieved, then it stands to reason that everyone in contact with the student needs a strong understanding of and focus on that process. Operating within strict compartmentalised units can easily result in a silo effect. Debate and Decisions: Accepting that a complete reengineering of HE is not going to take place any time soon, and that many staff are weary of changing management structures, the module team was confronted with how to realistically bring about change. The solution was to adopt a local communication strategy, by identifying all stakeholders in the process and approaching them directly. Something as simple as talking about the module with members of various departments has resulted in increased cooperation and even expressions of gratitude for having been recognised. Key contacts were library staff and first-year module leaders. The communication strategy is an acknowledgement that informal networking is at times more effective than the designed corporate channels of communication. It further emphasises that effective curricular design goes significantly beyond pedagogic theory and institutional procedures. 139

Redding

Conclusions/Major Lessons
The wider pedagogic community is recognising the importance of reinforcing study skills in HE. There are parallel efforts to bolster these and other skills within and outside the curriculum. Action research has identified a number of lessons and action points within one programme. Specifically, if study skills are external to subject specific curriculum, it is vitally important to simplify, mechanise, repeat, and reinforce all content. Additionally, it is important to put extra effort into communication, actively involving subject specialists and central support. When formulating assessment for first-year students, a pragmatic compromise is to acknowledge the students instrumentality, and use it to ensure engagement. And finally, try to embrace the ethos of continual improvement by tweaking the delivery, based on methodical research and knowledge-inaction.

References
Aguayo, R. (1990) Dr Deming: The Man Who Taught the Japanese About Quality, London: Mercury Books. ALDinHE (2008) 'Association for Learning Development in Higher Education'. Online. Available HTTP: <http://www.aldinhe.ac.uk/> (accessed 14 July 2008). Anderson, E. (2005) 'Dewey's Moral Philosophy'. Online. Available HTTP: <http://plato.stanford.edu/archives/spr2005/entries/dewey-moral/> (accessed 1 June 2007). Ashcroft, K., and Foreman-Peck, L. (1994) Managing Teaching and Learning in Further and Higher Education, London: The Falmer Press. Cash, C., and Hilsdon, J. (2008) 'DRAFT Exploring the Territories of Learning Development.' Presented at (This draft paper is to accompany a workshop to be held at the LDHEN Symposium, University of Bradford, March 2008) Bradford. Cole, G. A. (2004) Management Theory and Practice, London: Thomson. Cottrell, S. (2001) Teaching study skills and supporting learning, Basingstoke: Palgrave. Deem, R., Hillyard, S., and Reed, M. (2007) Knowledge, Higher Education, and the New Managerialism - The Changing Management of UK Universities, Oxford: Oxford University Press. Denis, L. (2008) 'Kant and Hume on Morality'. Online. Available HTTP: <http://plato.stanford.edu/archives/sum2008/entries/kant-hume-morality/> (accessed 14 July 2008). DfES (2003) 'The future of higher education'. Norwich: HMSO. Online. Available HTTP: <http://www.dfes.gov.uk/hegateway/uploads/White%20Pape.pdf> (accessed 10 March 2008). 140

Action Research Cycles

Field, R. (2007) 'John Dewey'. Online. <http://www.iep.utm.edu/d/dewey.htm> (accessed 1 July 2008).

Available

HTTP:

Geen, A. (2001) Effective Teaching for the 21st Century, Cardiff: UWIC Press. Gibbs, G., and Jenkins, A. (1992) Teaching Large Classes in Higher Education - How to Maintain Quality with Reduced Resources, London: Kogan Page Limited. Hammer, M. (1997) Beyond Reengineering, New York: HarperCollins. Hammer, M., and Champy, J. (2001) Reengineering the Corporation. A Manifesto for Business Revolution, London: Nicholas Brealey. Handy, C. (1993) Understanding Organisations, London: Penguin. Hattie, J., Biggs, J., and Purdie, N. (1996) 'Effects of learning skills interventions on student learning: A meta-analysis.' Review of Educational Research;, 66(2): 99 - 136. HE Academy (2008) 'The Higher Education Academy - Resources'. Online. Available HTTP: <http://www.heacademy.ac.uk/resources> (accessed 14 July 2008). HM Treasury (2006) 'Leitch Review of Skills - Prosperity for all in the global economy world class skills'. Norwich: HMSO. Online. Available HTTP: <http://www.dfes.gov.uk/furthereducation/uploads/documents/200612%20LeitchReview1.pdf> (accessed 1 July 2008). Honey, P., and Mumford, A. (1992) The Manual of Learning Styles Questionnaire, Maidenhead: Peter Honey Publications. House of Commons (2001) 'Higher Education: Student Retention'. Online. Available HTTP: <http://www.publications.parliament.uk/pa/cm200001/cmselect/cmeduemp/124/12402.ht m> (accessed 1 July 2008). Jeffrey, B. (2003) 'Countering Learner 'Instrumentalism' through Creative Mediation.' British Educational Research Journal, 29(4): 489 - 503. Jenkins, A. (2004) 'A guide to the research evidence on teaching research relations'. City: The Higher Education Academy: York. Lowe, H., and Cook, A. (2003) 'Mind the Gap: are students prepared for higher education.' Journal of Further and Higher Education, 27(1): 55 - 76. McNally, D. W. (1974) Piaget, education and teaching, Lewes New Educational Press Ltd. Metaphysics Research Lab (2005) 'Socrates'. Stanford: Online. Available HTTP: <http://plato.stanford.edu/entries/socrates/> (accessed 15 February 2008). 141

Redding

NAB/UGC. (1986) Transferable Personal Skills in Employment: The Contribution of Higher Education, London: National Advisory Board for Public Sector Higher Education/ University Grants Council. National Audit Office. (2002) 'Improving student achievement in English higher education. Report by the Comptroller and Auditor General HC 486'. London: The Stationery Office: London. OECD. (2002) Responding to Student Expectations, Paris: OECD - Organisation for Economic Co-operation and Development. Pettinger, R. (1996) Introduction to Organisational Behaviour, Basingstoke: Macmillan Business. Phillips, D. C. (2008) 'Philosophy of Education'. Online. Available HTTP: <http://plato.stanford.edu/archives/sum2008/entries/education-philosophy/> (accessed 4 July 2008). Qualifications and Curriculum Authority (2004) 'Key Skills: Standards and Guidance'. Online. Available HTTP: <http://www.qca.org.uk/qca_6455.aspx> (accessed 18 July 2008). Quality Assurance Agency (2006) 'Code of practice for the assurance of academic quality and standards in higher education. Section 7: Programme design, approval, monitoring and review'. Online. Available HTTP: <http://qaa.ac.uk/academicinfrastructure/codeOfPractice/section7/programmedesign.pdf> (accessed 18 July 2008). Ramsden, P. (1992) Learning to Teach in Higher Education, London: Routledge. Sander, P., Stevenson, K., King, M., and Coates, D. (2000) 'University students' expectations of teaching.' Studies in Higher Education, 25(3): 309 - 323. Saunders, M., Lewis, P., and Thornhill, A. (2003) Research Methods for Business Students, Harlow: Financial Times/Prentice Hall. Schon, D. A. (1991) The reflective practitioner : how professionals think in action, Aldershot: Ashgate. Smith, L., Dockrell, J., and Tomlinson, P. (1997) 'Piaget, Vygotsky and beyond : future issues for developmental psychology and education '. City: Routledge: London. University of London (2005) 'The London Philosophy Study Guide'. Online. Available HTTP: <http://www.ucl.ac.uk/philosophy/LPSG/Ep&Meth.htm> (accessed 10 February 2008). Wingate, U. (2006) 'Doing away with 'study skills'.' Teaching in Higher Education, 11(4): 457 - 469.

142

Action Research Cycles

Wingate, U. (2007) 'A Framework for Transition: Supporting 'Learning to Learn' in Higher Education.' Higher Education Quarterly, 61(3): 391405. Yorke, M., and Longden, B. (2004) Retention and Student Success in Higher Education, Maidenhead: Open University Press.

143

PRIME Volume 3(2)

Is There a Theory of SoTL?

Janet Parker The Open University, Institute of Educational Technology

Summary
There are now well established SoTL communities of practice and of discourse; there are international networks, dedicated journals, the newly internationalised Carnegie Academy for SoTL, but is there an established theory of SoTL? This was a question posed by Graham Gibbs at the International Society of Scholarship of Teaching and Learning conference (and the Carnegie-edited Special Issue of Arts and Humanities in HE 7 (3)). The question was posed and addressed, but not answered; partly because we are talking about different research bases, literatures and paradigms, partly because theory means something different in educational and disciplinary research and partly because scholarship is a term that is tricky and differently valued in the UK, Europe and the US. For a curriculum to be informed by a SoTL approach, clarity about theoretical bases is urgently needed. Should SoTL build on the decades of US SoTL, distinguishing SoTL from the other scholarships (of inquiry, integration and engagement)? Should SoTL rather look to pedagogic research, building on Australian and European research or should it look to discipline- and domain-specific epistemology and research traditions? Should scholars of Teaching and Learning be also, or rather, educational researchers? Should they we be experts in, in order to be practitioners of, SoTL? Conversely, should pedagogical researchers cut themselves away from the amateurism that they are sometimes accused of? This paper will try to clarify some of the issues.

Introduction
There are now well established SoTL communities of practice and of discourse. These include international networks, dedicated journals, the newly internationalised Carnegie Academy for SoTL and the newly constituted, truly international, International Society for Scholarship of Teaching and Learning (ISSOTL), now allying itself with European and Australasian research but how does that intersect with European, UK and Australian established pedagogical research traditions? Where does pedagogical research fit in higher education? Why is this important? It is partly for the identity, status and sense of purpose of individual pedagogical researchers: are we part of a field, a discipline, a practice or a movement? In the US SoTL started as a pressure group for change, as a lobby insisting on the importance of teaching. This is a very timely issue worldwide, and in the UK, a vital question. Right now in the UK, scholarship in general and SoTL in particular seem to offer a way out of what some academics might see as the disastrous consequences of the pernicious distinction between RAE active and, frankly, redundant research academics. Scholarship is likely to be a catch-all term, signifying a valuable contribution to teaching and the academic community which does not satisfy the RAE guidelines. This takes the UK back to the battle-lines that in the US, in the 80s, Boyer fought to negate. 145

Parker

Background to US SoTL
Boyers 4-fold scholarships of teaching, discovery or inquiry (meaning RAE-able disciplinary research), integration and application (engagement with and implementation in the community) were argued to be comparable in value and in potential for academic reward. In the many colleges that adopted his principles, promotion, tenure and reward criteria were rewritten to include teaching and less frequently other scholarships, in parallel to disciplinary research. The Carnegie Foundation for the Advancement of University Teaching from the 90s had a clear vision of what the scholarship of teaching and learning could be: the revitalisation of both classroom and the disciplinary research base through considered, evaluated and published action research. Carnegie Fellows looked into distinctive disciplinary processes (those that Shulman was to go on to develop as signature pedagogies i.e. the essential meaning-making processes that distinguish one discipline from another pedagogically and therefore, importantly, epistemologically). So, for example, History fellows researched discursive and recursive practices in History both as essential historical method and as keystone practices to be taught to undergraduates; American Studies looked to the effects of oral, digital and visual narrative-making in classroom and community; linguists researched intercultural communication in the classroom; a Nobel prize-winning Chemist explored his students writing of their shared discipline. From its roots in the Carnegie Foundation for the Advancement of University Teaching and at Indiana University, SoTL has grown quickly and influentially in the US and now beyond. The last few years have seen the newly internationalised Carnegie Academy for SoTL (CASTL) and the formation of the International Society for SoTL (ISSOTL).

SoTL so far, so good?

So far, so good. Or perhaps, not quite? The Washington ISSOTL conference in 06 raised some problems with this would-be all-conquering coalition of SoTL members. The historians split away, declaring the lack of relevance of the generic case studies presented, forming their own international SoTL network and newsletter (Pace 2006). An Australian researcher reported that: My experience of the first few days of the Washington conference left me convinced that SOTL American-style was more than just a social movement and was in fact some sort of cult. I witnessed a steady progression of American academics who had travelled to DC to share their, remarkable for them / unremarkable for the audience, road to Damascus experiences; how one day in the class room the scales had fallen from their eyes and they and their students had been saved by SOTL (Brawley 2009). So much and so much achieved, seemed to have been bypassed at this conference. The question of theory is contentious and difficult what theory? Or, rather, whose theory? That of educational psychology, sociology or philosophy?

146

Is there a Theory of SoTL?

The theory of SoTL is a current and prickly topic (the subject of a Special Issue of this contributors journal edited by Pat Hutchings and Mary Huber of the Carnegie Foundation for the Advancement of University Teaching (Arts and Humanities in HE vol. 7 (3), Nov 2008). Such a radical and problematic topic needs to be put to pedagogical researchers in HE. There are two elements to the question which I would like to explore that of scholarship and theory. Both are difficult and both were posed and addressed, but not answered, at ISSOTL; partly because we are talking about different research bases, literatures and paradigms: educational, discoursal, philosophic, political; partly because theory means something different in educational and disciplinary research; and partly because scholarship is a term that is tricky and differently valued in the UK, Europe and the US. The Carnegie rooted SoTL in the subject disciplines and so provided a single paradigm of theory, if varying by discipline. By tying SoTL to disciplinary rather than generic educational research, the Carnegie also tied SoTL to epistemology; the first question to ask about a History or Medical Sciences curriculum is not how but what and why it is to be delivered? The theoretical base of such research comes from the subject disciplines own methods and criteria and the results of observation, reflection, measured change and dissemination often mirror the practices of the discipline community. This base in the subject disciplines was in the UK taken into the Higher Education Academys Learning and Teaching Subject Centres Network. Growing out of and fostering networks of discipline academics interested in action research and pedagogical projects, Subject Centres were sited in disciplinary departments or professional associations and promoted, published and disseminated results and ideas to the disciplinary community. Although the Subject Centres met with varying engagement and readership, a recent study of SoTL in three countries by Brawley et al (2009) could point to the Subject Centres as the bridge into pedagogical research for the HE community that his own country, Australia, lacked. The underlying and unresolved issue is that such a move can be accused of amateurism: of a field only involving those who do not have a grounding in, still less a profound grasp of, educational research disciplines in which methods and paradigms are taken from or combine educational sociology, psychology, philosophy, ethnography, phenomenography, theories of learning, teaching and curriculum design. There is, of course, a long European and Australian tradition of rigorous and theoretically grounded pedagogical research. The question is, should SoTL researchers see themselves as part of that tradition? Should they be open to the charge that they are reinventing the wheel and failing to stand on the shoulders of giants. Are they, that is to say, to see themselves ashmmm, vertically, academically speaking, challenged? This question seems to me a vital one, and one that PRHE serves to raise. Giants, reinventing wheels are part of a discourse of established truth that those joining the research field must take as givens. But, such metaphors surely belong, rather to that of Pink Floydanother brick in the wall For one of the things underlying such complaints is a sense of pedagogic research as fitting into a unified, Kuhnian paradigm. This can only be understood in a context of pedagogy as having an accretive knowledge147

Parker

and evidence-base, such that researchers need to start from understanding both the bricks (what we know, as the result of the work of giants and lesser mortals, available in the literature) and the gaps signalling work to be done. The Carnegies Mary Huber and the HEA Anthropology Subject Centres David Mills wrote, rather, of pedagogic research as an Educational Trading Zone (Mills and Huber 2005). This inspiring model offers a vision of researchers of teaching and learning entering the zone to barter barter ideas, methodologies and paradigms drawn from their own subject discipline and from those they picked up last time they visited the zone. There are examples of scientists interacting with humanists, educationalists with linguists. Of course such exchanges are not deep interdisciplinary re-disciplining but they can suggest new questions to be asked and offer new ways of addressing them. This seems to me to offer a diametrically opposite model to that of a strictly hierarchical pedagogic institute with rigorously controlled entry levels and progression bars. SoTL, rather, operates as a community: a community of practice, of discourse, of mutual and reciprocal learning, of friendly collaboration and mentoring. Does it not seem appropriate that the theoretical underpinning should be of a trading zone rather than a Kuhnian unified research paradigm? This raises a question for PRHE, which clearly and successfully runs a high powered forum for exchange of ideas. The question is, where does and where should, pedagogical research in higher education position itself? As a separate and separatist research discipline? And with, incorporating or against the broad church of SoTL?

Scholarship and PRHE

The first issue is I think, what work scholarship of teaching and learning is being made to do. The infinitely generous and generative Lewis Elton has pointed to the root meaning of scholarship, used in all these contexts as a translation of Humboldts Wissenschaft. Translation, in this as in all contexts, of scholarship is a cultural not just a linguistic matter. There are many definitions of Scholarship from the German (as Elton says, the Oxford English Dictionary defines it as a translation of the German word Wissenschaft, and it can be traced back to Humboldts famous prescription for the future University of Berlin (Humboldt 1810, English translation: Humboldt 1970) Humboldt was concerned with both research and teaching, and he established a fundamental dichotomy between university and school according to which the university in contrast to school treats scholarship always in terms of not yet completely solved problems, whether in research or teaching, while school is concerned essentially with agreed and accepted knowledge. This is an empowering definition: scholarship is the tackling of disciplinary issues raised in teaching as in research. This immediately drives a coach and horses through the barriers that have arisen recently between teaching and research, whereby research quarries new knowledge and teaching transmits it to the next generation. (I have argued in public meetings about the need for a comparable attention to teaching- informed research as well as the more common and funded research-informed teaching. I have each time been corrected by the chair, who presumed I meant research-informed teaching in all cases, a one-way and strictly hierarchical process. Similarly, I have raised questions about knowledge transfer from research to educational settings. Of a new Arts and Humanities Research Council KT initiative, I asked what of the funds allocated by the council over 148

Is there a Theory of SoTL?

the last decade had not resulted in knowledge being transferred via books, teaching and public intellectual performances? That was not what the fund was for, I was told: it was to enable research to be transferred to, for example A level colleges and teaching institutions. I have always lost those exchanges. It seems so clear to me energy and reflective vitality can be transferred to disciplinary research hierarchs; teaching can inform the research agenda by setting it new and life-renewing questions. Investigations of teaching and learning initiatives can ask basic questions about the value and nuances of disciplinary processes, ones that may be taken for granted in the sort of consensual, conservative research proposal. For one cannot ask basic questions while asking for major research grants to answer paradigm-set questions. Kuhn lays down the parameters of the paradigm as the disciplinary epistemological model which both sets the questions to be asked and the disciplinary means to answer them. But reflections on what is not working in a curriculum or assessment strategy, on problems with establishing examination criteria or marking disputes which can frequently be the starting point for much SoTL action research, can move the discipline on in fundamental ways. For seemingly everyday teaching processes carry with them wide-ranging implications for the knowledge- and meaning-making epistemological framework of the discipline. It is commonly said that if one has a problem in ones research, one applies for a research grant; if one has a problem in ones teaching, one keeps very quiet and hope it goes away. But such problems may reveal an epistemological fault line in the discipline. Coffin et al (2007) reported on a fascinating problem with their project to collect disciplinespecific examination criteria The problem came when interview data on marking criteria were compared to the same examiners actual marking. They rarely matched up; extensive interviews revealed the epistemological and disciplinary structures of thought that underlay and brought together the seemingly disparate structures of criteria and judgements. For example, in Geography, one criterion was the exact reference to location but one script was awarded a first without a single map reference. Exploration revealed that the examiner had intended the criterion to denote a fine locational sense and he had rewarded a script that encapsulated a precise sense of place.

Problems with expert knowledge?

The theoretical frame for the scholarship of teaching and learning is to some extent practical: theorised scholarship is that which is accepted as validly grounded in SoTL as judged by the reviewers of conference paper applications and journal editors and referees. Given the 50 years plus of pedagogical research in higher education in Scandinavia, UK and Australia, should SOTL conferences and publications accept papers not grounded in such pedagogical research? There are a variety of answers, and a variety of grounds for answer, of this question. For the Lund academy, for example, the answer is simple: no. They have taken Carolin Krebers model of excellent, expert and scholar teacher (Kreber 2002) and made such categories the essential stages for entry and progress in a Scholarship of Teaching and Learning Institute. All teachers accepted after a tough application process, are deemed excellent, the baseline criterion. They progress to expert status after presentation of a folder of scholarship research and reflection; scholar teachers are those whose 149

Parker

knowledge of scholarship of teaching and learning qualifies them to teach and evaluate others. So the baseline for entrance to the Teaching and Learning Institute is informed understanding of and reflection on scholarship, the prerequisite for adding to the database of the discipline. However, research syntheses such as Little, Parker and Richardson (2008) for the Higher Education Academy point to problems with such a smooth progression from excellent to expert to scholar. Excellent teaching, in itself is a disputable label, but even if one takes a consensual rather than an idiosyncratic definition of excellence (and many teachers would refute the idea of a univocal, single definition of something as various, singular and personal as teaching), the relationship of excellence to scholarship is problematic. Some studies (Little et al 2008) seem to say that knowledge of teaching and learning scholarship is unrelated or even in inverse proportion to excellence! This may be explained by the common sense perception that knowing about it is not the same as being able to do it, in any performance medium. In fact, a study by M. Ferrari in The Pursuit of Excellence Through Education (2002 NJ) suggests that charisma and enthusiasm are the single outstanding qualities rated by students-poor old scholars of teaching and learning! Kreber has made a different point, querying the progressive scale from excellent to expert to scholar. For she points to the sheer investment in time and attention required to be an excellent teacher, given that preparation, feedback, advice, reflection and evaluation are heavily-demanding processes. With teachers also being expected to research their own discipline, bring in project money and fulfil administrative and tutorial roles, where can the time be found to become expert scholars of teaching and learning? This was a question that struck a chord with pedagogical researchers at the PRHE conference.

Conclusion
Anyone who has worked on an HE project with researchers from different traditions educational psychology, sociology, policy, philosophy, phenomenography, ethnography, student learning and so on, can vouch for the variegations and disagreements that inevitably arise. The questions and results look very different depending on how they are interpreted. This is a richness, if you take Mills and Huber's educational trading zone approach, but is seriously problematic if you conceive of pedagogical research as necessarily fitting into a Kuhnian paradigm. What SoTL and Gibbs have done is raise the question of exclusiveness and expertise. Rigour, validation, contextualisation, appropriately deployed methodology, properly gathered and evaluated data, peer publication......of course!!! But the question facing all of us involved in pedagogic research, and especially those of us involved in publishing, is whether we can cope with a plurality of goods, a plurality of approaches. That greatest and most problematic teacher, Socrates paralysed his interlocuter by saying "Splendid! I asked for one definition of excellence and you have given me a swarm of them". Paralysed, because the object of the exercise was to exclude and to essentialise. That may be the right way to go in building a theoretical frame in philosophy but not, I would argue, in Pedagogy.

150

Is there a Theory of SoTL?

References
Boyer, E. L. (1990) Scholarship reconsidered: priorities of the professorate, San Francisco: Jossey-Bass, p. 16 Brawley, S. (2009) SOTL and national difference: musings from three historians from three countries, Arts and Humanities in Higher Education 8 (1) Coffin, C., Purser, E., Donohue, J., Skillen, J., Peake, K., Dean, M., Mitchell, S., Developing academic literacy in context: movements between practice & theory http://www.schreibzentrum.de/eataw2007/schedule/sat2/coffin.html Ferrari, M (2002)The Pursuit of Excellence Thro Education (NJ) Humboldt, W. von (1970) On the spirit and organisational framework of intellectual institutions in Berlin, Minerva 8 : 242 267. Kreber, C. (2002) Teaching excellence, teaching expertise and the scholarship of teaching, Innovative Higher Education 27(1): 5-23. Little, B., Parker, J., Richardson, J. (2008) Excellence in HE: international literature review http://www.heacademy.ac.uk/ourwork/policy/excellence (accessed 15 May 2008) Mills, D and Huber, M.T. (2005) Anthropology and the educational trading zone: disciplinarity, pedagogy and professionalism, Arts and Humanities in Higher Education, 4 (2) 9 - 32. Pace, David (2006) The internationalizaton of History teaching through the scholarship of teaching and learning: creating institutions to unite the efforts of a discipline, Arts and Humanities in Higher Education vol 6 (3) Shulman, L. (2005) The signature pedagogies of the professions of law, medicine, engineering, and the clergy: potential lessons for the education of teachers. National Research Councils Centre for Education. http://hub.mspnet.org/index.cfm/11172. (accessed 15 May 2008)

151

Prime Volume 3(2)

The Interrelationships between Assessment Marks within a First-Year Undergraduate Programme: Some Implications for Aggregating Marks and Recording Achievement
Kevin Rowley and Andy Bell Manchester Metropolitan University, Department of Psychology and Social Change

Summary
Mark characteristics and mark relationships were explored for individual assessment elements within a first-year psychology undergraduate programme. There were large differences in mean marks across these assessment elements and large differences across units in the correlation between elements of unit assessment. Consideration is given to how these differences may be interpreted. The correlations between elements of assessment suggest that, in general, assessments within units do not form any more meaningful and coherent components than may be found across assessments outside of the units. Unit assessment coherence appears to be strongly influenced by the variety of assessment methods used and by other artifacts of the assessment process (e.g. mark reliability and differentiation). These influences prevent the emergence of coherent unit components and confound the interpretation of assessment element marks. In this context, the overall aggregate mark for the year may achieve a meaningfulness and usefulness that is not present in a more detailed profile of marks. This suggested relative usefulness of an overall aggregate mark is discussed in contrast to the Burgess Groups recommendation for recording student achievement as a profile of component elements without an overall summative judgment.

The assessment strategy and the meaning of marks

An anonymous database of marks for a cohort of undergraduate Psychology students provided the basis for an exploration of the relationships between marks for assessment elements. All students gave consent for their assessment marks to be used for such research purposes. The database consists of 76 students who entered university in 2004 and graduated in 2007. The following account addresses the first-year performance of these students. Table 1 below provides an outline of the assessment strategy for the nine units within this first-year for 2004. This table illustrates the diversity of assessments both within and between the units. Such an approach is consistent with the QAA (2006) code of practice which states that: To test a wide range of intended learning outcomes, diversity of assessment practice between and within different subjects is to be expected and welcomed, requiring and enabling students to demonstrate their capabilities and achievements within each module or programme (p. 4). The marks for each assessment are indicators of a level of performance assessed against the learning outcomes. The university regulations and procedures handbook states that a mark between 40 and 49 indicates that all learning outcomes have been achieved at a threshold level. A mark between 50 and 59 indicates that all learning outcomes have been achieved at a good level. The remaining pass outcomes are described as very good (60-69) and excellent/outstanding ( 70). Therefore, for example, giving a mark within the threshold level (i.e. between 40 and 49) is a matter of distinguishing between levels of 153

Rowley and Bell

performance that are above a marginal fail but below a minimum good. Furthermore, use of the term good, etc., implies that marks are ordinal (non-interval) judgments of quality, made with reference to a notion of what most people taking the assessment ought to be able to do if they have taken advantage of the learning opportunities. Marks can therefore be described as criterion-referenced in so far as they represent the achievement of specified learning outcomes. However, in so far as these outcomes are set with consideration to what is typically achievable, they will also reflect normreferenced expectations. A useful discussion of the differences in meaning between the term marks and the concepts of scores and grades is provided by McLachlan and Whiten (1999). Table 1: Units and outline assessment strategy for first-year BSc (Hons) Psychology (2004). Units and Credits Biopsychology (20) Assessments Weighting of Assessments 45:55 40:60 30:70 20:40:40 10:30:60 10:90 50:50 Overall Average 50:50

Average of Three Multiple-Choice Tests (MCTs) & Exam Child Development (10) Essay & Exam Cognition (10) Essay & Exam History & Philosophy (10) Group Presentation, Essay & Exam Individual Differences (10) Group Presentation, Essay & MCT Inquiry (10) SPSS test & Inquiry Exam Information Technology (IT) & Interpersonal Skills IT exercise & Reflective Diary Workshops (IPSWS) (20) Best Four from Five Practicals & Practicals (20) One group Practical Societal (10) Portfolio 1 & Portfolio 2

Summarising assessment performance

Given the ordinal nature of marks, it is often seen as inappropriate to calculate means and other parametric statistics (e.g. McLachlan and Whiten, 1999). However, as indicated by Fowell and Jolly (2000), the appropriate summary statistic for overall performance is a much debated area, and there are a number of advantages to using parametric statistics such as the mean and standard deviation. Table 2 provides the mean mark and standard deviation for the different assessments within each unit for the 2004 cohort. Overall mean performance across these assessments varies between threshold and outstanding. The two outstanding performances are within the SPSS assessment for the Inquiry unit and the group presentation assessment for the Individual Differences unit. These assignments appear to be more about mastery assessment rather than any normative assessment as outlined in the previous section. However, as shown in Table 1, these assessments are not heavily weighted within the relevant units. With these assessments excluded, the range of overall mean performance across the assessments is still large (i.e. threshold to very good). The standard deviations shown in Table 1 indicate that there is also considerable difference in the spread of marks across the assessments.

154

Marks within a First Year Undergraduate Programme

Table 2: Mean mark and standard deviation (SD) for the elements of assessment. Unit Assessment Biopsychology MCTs Biopsychology Exam Child Development Essay Child Development Exam Cognition Essay Cognition Exam History & Philosophy Group Presentation History & Philosophy Essay History & Philosophy Exam Individual Differences Group Presentation Individual Differences Essay Individual Differences Exam (MCT) SPSS test Methods/Statistics Exam IT exercise IPSW Reflective Diary Overall Average Societal Portfolio 1 Societal Portfolio 2 Mean Mark 62.08 56.55 57.71 55.82 52.39 49.29 62.96 48.38 56.55 76.64 55.92 54.30 95.20 59.51 57.58 55.86 59.08 60.14 53.37 SD 13.28 15.84 7.63 7.83 8.10 10.03 7.79 12.59 10.41 4.19 12.43 11.05 10.57 12.21 12.88 12.83 7.04 11.89 8.13

Biopsychology Child Development Cognition

History and Philosophy

Individual Differences

Inquiry IT & IPSW Practicals Societal

Table 2 has a similar format to a transcript of marks for recording individual student achievement. Within such a transcript, cohort mean marks for unit assessments are not usually shown and this is replaced with individual student marks on the assessments. Also usually shown within an individual transcript are the weighted overall unit mark and the weighted overall mark for the year (weighted by unit credit value). Such a transcript is similar in certain respects to the Higher Education Achievement Report (HEAR) proposed by the Burgess Group Final Report (2007). The transcript described above is similar to the proposed HEAR in that the HEAR will contain marks (in whatever form the institution chooses to use) awarded to the separate components of the honours degree (p.36). However, the Burgess group also proposes that the means of representing student achievement should be radically reformed ideally to replace the summative judgment with a more detailed set of information (p. 9). Thus, for example, the overall mark for the year would be removed from the transcript described above, requiring the user of the transcript to consider the profile of performance across components. Such a consideration may be made difficult by differences in means and standard deviations between the assessments, such as those presented in Table 1. There is also a question of what should constitute a component. For the transcript described above, the components are the different assessments within each unit and the weighted mark for each unit. To what extent do the assessments within units form meaningful and coherent components? 155

Rowley and Bell

One important source of information in addressing this question is obtained from examining the interrelationships between the assessment marks.

The interrelationships between assessment marks

The large differences in mean marks across the assessments shown in Table 1 could come about for a number of possible reasons. One possible explanation is that the students within this particular cohort on the whole performed rather better or rather worse than expected across the different assessments. This in turn could be due to better or worse learning and teaching associated with these assessments. However, other competing explanations include the possibility that the difficulty of the learning outcomes were not set at comparable levels across the assessments, or that the judgments of performance level were not comparable across the assessments. These two latter possibilities present a particular difficulty if assessments stand alone as components within a transcript of marks. Differences in mean marks across assessments present only one source of difficulty for the interpretation of assessment marks. The remainder of this paper will examine other sources of difficulty for the interpretation of marks, primarily through the use of correlation. Here, Pearsons correlation coefficient will be used as a simple index of linear relationship between marks for any two assessments completed by this cohort of students. This index is not affected by differences in means between the assessments, but rather assesses the extent to which students relative mark position is maintained across any two assessments. Examining such correlations between assessments provides useful information on the extent to which combinations of assessments form coherent components. When marks should correlate We should expect assessment mark correlations to reflect the extent to which there is commonality in the specific learning outcomes addressed by the assessments. We should also expect mark correlations to reflect the extent to which there is commonality between assessments in Psychology knowledge, Psychology specific skills and generic skills (see QAA Psychology Subject Benchmark Statement, 2002). Why marks between related assessments may not correlate Mark correlations that are expected on the basis of the above commonality will however be reduced for a number of possible reasons. For example, using an inappropriate assessment strategy for the intended learning outcomes will affect the expected correlations between assessment marks. This is a question of whether the assignments are meaningful (i.e. valid) assessments of the learning outcomes. However, even if the assessments are considered to be valid, there may be a large number of possible and equally valid alternative assessments for the learning outcomes. These alternative assessments can vary both by method (e.g. essay, exam, observation) and by task (i.e. the particular goal or content of the assessment). Shavelson, Gao and Baxter (1993) provide a useful analysis of the sampling variability of performance assessments and conclude that assessment scores are dependent upon the sampling of both the method and the task. 156

Marks within a First Year Undergraduate Programme

Similarly, Koretz (1998) states that student performance tends to be quite inconsistent across seemingly related complex tasks. Mark correlations will also be reduced by low marker reliability, both within a marker and between markers, and by any lack of differentiation in the marks (i.e. a narrow spread of marks). Temporary changes in individual student circumstances (e.g. health), that differentially affect the accurate demonstration of learning and skills across assessments, will also lower correlations between marks. One other possible reason for low correlations is any change in the relative positions of the students in their underlying learning and skills that are of relevance to the learning outcomes. Of course, changes in knowledge and skill are to be expected within education. However, any systematic change across the students will not affect the correlation between assessments, only changes in relative position. This could occur, for example, if some students leapfrogged others in learning and skills in the time between the related assessments. When considering the many possible reasons for why assessments with high commonality in learning outcomes and skills may nevertheless show weak mark correlation, it is only any change in relative position between students in relevant learning and skills that is a meaningful influence. All other influences outlined above may be considered artifacts of the assessment process. As previously discussed, an individual transcript of marks will usually show the weighted overall mark for each unit of study. This component mark thus suggests that the unit itself (i.e. usually a content domain) provides a sufficiently meaningful commonality of learning and skills across the assessment elements. Furthermore, if this is the only type of aggregate mark to be shown, then this suggests that aggregating within units produces a more meaningful component than aggregating either over the method of assessment (e.g. exam, presentation, etc.), or over the occasion of assessment (e.g. first, second or third term), or over all the assessments within the year. A comparison of overall correlations for marks from within and between units Table 3 provides the Pearson correlation between the marks obtained by the 76 students for assessments within each of the first-year units. Also shown in Table 3 is the average Pearson correlation between each assessment and all other assessments outside of the unit. The correlation for assessment marks within a unit range from .59 to .001 with a mean of .24. Thus, there is large variation between the units in the extent to which students relative performance in an element of unit assessment is related to performance in the other assessment element/s for the unit. There is also large variation between assessments in the extent to which they relate to other assessments outside of the unit (the outside-unit correlation). That is, the average correlation between each assessment and those assessments outside of the unit range from .36 to .08 with a mean of .22. Overall then, the mean within-unit correlation (i.e. .24) is very similar to the mean outside-unit correlation (i.e. .22). For some authors, the preferred procedure for obtaining the average of correlation coefficients is to transform each correlation to Fishers z coefficient, calculate the average of these z coefficients and finally transform this mean z back to its corresponding correlation coefficient (e.g. see Corey, Dunlap and Burke, 1998). Using such a procedure gives an average within-unit correlation of .23 and also gives an average outside-unit correlation of .23. These similar mean correlations suggest that, in general, 157

Rowley and Bell

marks from assessments within a unit are no more strongly related to each other than they are to marks outside of the unit. This in turn suggests that, in general, there is as much commonality for assessments outside of units as there is within units. Table 3: Interrelationships between assessment marks Correlation Average Correlation Unit between between Assessment Unit Assessment Assessments and all Assessments Element Within the Unit outside unit MCTs .36** .59** Biopsychology Exam .28** Essay .17 Child .17 Development Exam .19* Essay .27** .23* Cognition Exam .23* Presentation and Group .10 Essay .001 Presentation Presentation and Essay .20* Exam .37** History and Essay and Exam .17 Philosophy Mean correlation Exam .20* between History and Philosophy assessments .18 Presentation and Group .13 Essay (.15) Presentation Presentation and Essay .13 Exam (.16) Essay and Exam Individual (.05) Differences Mean correlation Exam (MCT) .34** between Individual Differences assessments (.12) SPSS test .08 .02 Inquiry Methods/Stats .28** Exam IT exercise .20* IPSW .33** IT & IPSW Reflective .26* Diary Practical 1 .25* Practicals .39** Practical 2 .20* Portfolio 1 .30** .53** Societal Portfolio 2 .30** Correlations with a single asterisk are significant at the 5% level Correlations with a double asterisk are significant at the 1% level 158

Marks within a First Year Undergraduate Programme

The above analysis for Practicals is based on two individual practicals completed by 76 students. Fewer students completed all five individual practicals, since the unit mark is made up of the best four from five individual practicals and a group report. Further evidence that assessments within units do not in general form any more coherent components than may be found between assessments across the year is presented within Table 4. This table shows which assessment within the year has the strongest and the weakest correlation with each element of unit assessment. Table 4 shows that the assessments with the strongest correlation are rarely found within the same unit. Thus, marks from assessments within a unit are usually more strongly correlated with marks from an assessment outside of the unit than with marks from assessments within the unit. The influence of assessment method One reason why there may be as much commonality for assessments outside of units as there is within units arises from the distribution of assessment methods across the year. That is, the previously described diversity of assessment methods within each unit is likely to lower the within-unit commonality, and any similarity in methods across different units is likely to increase the outside-unit commonality. This suggested important influence of assessment method may be illustrated by a consideration of the two units shown within Table 3 that possess the strongest within-unit correlations. Table 3 shows that the Biopsychology unit has the strongest correlation between assessment elements, where the multiple-choice tests (MCTs) and exam correlate at .59. The unit with the second strongest correlation between the assessment elements is Societal, where the two portfolios correlate at .53. For Biopsychology, the mct mark is derived from an average of three tests taken over the first two-terms of the year, and the exam is a traditional unseen written-paper taken in the third term. These two assessment methods possess features that provide plausible explanations for the relatively high correlation between these assessments. For the Biopsychology MCTs, these features include the high marker reliability for each of the three objectively scored tests. Also, since the overall MCT mark is an average of three tests, the average MCT mark is likely to have greater validity (i.e. better represent MCT performance over the entire content domain and better represent performance over a sustained period) than any of the single multiple-choice tests. Finally, the average MCT mark has relatively high differentiation between the marks (see Table 2, where the Biopsychology MCT assessment has the second highest sd). This assessment therefore has the potential to strongly correlate with any other assessments that have a commonality with the learning and skills required within the MCTs. The Biopsychology exam does not possess the above advantages of confidence in marker reliability and confidence in the full representation of the content domain. However, this assessment does possess the very highest differentiation between marks (see Table 2), and in this respect this exam has the potential to strongly correlate with other assessments that have a commonality.

159

Rowley and Bell

Table 4: Assessments within the year with the strongest and the weakest correlation with each element of unit assessment.

Unit

Unit Assessment Element MCTs

Highest correlation with Unit element Individual Differences MCT Biopsychology MCTs Biopsychology MCTs Biopsychology MCTs Societal Portfolio 1 Individual Differences MCT History & Philosophy Exam Individual Differences MCT Child Development Exam IPSW Diary IPSW Diary Biopsychology MCTs IPSW Diary Practical 1 Societal Portfolio 1 Individual Differences Essay Inquiry Exam Biopsychology Exam Societal Portfolio 2 Societal Portfolio 1

Lowest correlation with Unit element Individual Differences Essay Individual Differences Presentation Inquiry SPSS Information Technology Inquiry SPSS Individual Differences Essay Inquiry SPSS Individual Differences Essay Individual Differences Essay Child Development Essay History and Philosophy Exam Individual Differences Essay Child Development Essay Inquiry SPSS Child Development Exam Cognition Exam Inquiry SPSS Inquiry SPSS History and Philosophy Group Presentation Individual Differences Essay

Biopsychology Exam Essay Exam Essay Cognition Exam Group Presentation History and Philosophy Essay Exam Group Presentation Individual Differences Essay Exam (MCT) SPSS test Inquiry Methods/Stats Exam IT exercise IT & IPSW IPSW Reflective Diary Practical 1 Practical 2 Portfolio 1 Societal Portfolio 2 160

Child Development

Practicals

Marks within a First Year Undergraduate Programme

The Biopsychology exam and MCTs appear to be quite different methods of assessment. However, in some respects these two assessments are quite similar. For example, both involve timed conditions and no access to materials once the assessment has begun. The skills necessary to meet these common conditions will contribute to the commonality between the two methods of assessment. Also, the broad knowledge of biopsychology as assessed by the MCTs, in so far as it is necessary or helpful for the exam, will also contribute to the commonality between the assessments. For Biopsychology then, the relatively high correlation between the assessments within the unit appears to be the result of three factors: Firstly, the assessment process for each element of assessment produces marks which suggest the potential to relate to other assessments. Secondly, the methods of the assessment within the unit appear to require some common skills. Finally, the content domain learning assessed by one method within the unit appears to be relevant to the learning assessed by the second method. For the Societal unit, the two assessment elements share the same method (i.e. portfolio) and share a Societal Psychology content domain. In these respects the assessment elements for this unit may be considered parallel assessments. Thus, the relatively high correlation between assessment marks may be expected. Furthermore, learning and skills obtained from completing the first portfolio assessment, including the use of feedback from the marker, can directly inform the completion of the second portfolio. For this unit, no initial assumptions were made about the potential of each assessment to relate to other assessments (e.g. due to marker reliability or mark differentiation). However, this potential is now evidenced by the expected relatively high correlation. The two units discussed above (i.e. Biopsychology and Societal) have relatively high within-unit mark correlations that appears to result partly from a relatively high commonality in assessment content domain, but perhaps especially from a relatively high commonality in assessment method within these units. Those units shown within Table 3 that have relatively low within-unit correlations use quite different methods of assessment within the unit. For example, for all four units that employ an essay and an exam, the within-unit correlation between these two methods of assessment is at .23, or more frequently below this value. Thus, within-unit correlation appears to be strongly influenced by the similarity of the assessment method. There may be a number of advantages to having similar assessment methods within any individual unit; for example, the feedback from one assessment can directly inform the completion and improvement of the next assessment within the unit. However it may not be possible to fully represent and assess the learning outcomes within a unit through the use of similar assessments. The influence of other artifacts of the assessment process The previous section indicated that the assessment process should produce marks that possess the potential to relate to other assessments. As discussed previously, mark reliability and mark differentiation are two important contributors to this potential. The influence of these factors on mark relationships is further illustrated by the assessment relationships shown in Table 4. Here, the MCTs for Biopsychology and Individual Differences not only correlate most strongly with each other, but also have the strongest correlation with four other assessments that vary both by method (i.e. are either an essay or essay-type exam) and by content domain (i.e. do not belong to either the Biopsychology unit or the Individual Differences unit). Some correlation between the 161

Rowley and Bell

Biopsychology and Individual Differences MCTs may be expected since they share an assessment method. For these MCTs, the similarity in method of assessment appears to be a more important influence than differences in content domain. However, the relatively strong correlation between this method of assessment and other assessments that differ both in method and in content domain may be the result of the high marker reliability, and to some extent greater differentiation in marks, for these objectively scored tests. That is, although MCTs within a unit may have relatively low commonality of required learning and skills with an exam in a different unit, or especially an essay within a different unit, any commonality that does exist is not undermined by low marker reliability and low mark differentiation from at least one of the assessments (i.e. the MCTs). Furthermore, the relatively high correlation between the MCTs and other apparently quite different assessments, suggests that the commonality operating here may not be primarily due to any direct relationship between acquired knowledge and skills that are common to the assessments. Rather, such commonality may be primarily due to the indirect influence of student differences in such general factors as approaches to study and study skills. The above discussion suggests that if an element of assessment has low correlations within the unit and low correlations across all other assessments then there may be an issue with that assessments validity or reliability and differentiation. Although such an assessment may be meaningfully assessing a content domain that is quite specific and useful, this is perhaps unlikely, especially if there is some expected commonality with some other assessment that appears to share required knowledge or skills (e.g. due to a shared assessment method).

Conclusions
The above discussion suggests that mark relationships reflect the various influences of assessment validity; assessment method; assessment content; marker reliability; mark differentiation; the commonality in acquired knowledge and skills that are directly relevant to the learning outcomes and the commonality that arises from indirectly relevant factors such as student approaches to study. The above discussion is consistent with the view that elements of assessment within a unit that assess similar learning outcomes, using similar methods and content, should give higher correlations between marks and produce more coherent aggregate mark components in comparison to units without these characteristics. However, the evidence from the first-year mark relationships presented above suggests that, in general, assessments within units do not form any more meaningful and coherent components than may be found across assessments outside of the units. Unit assessment coherence appears to be strongly influenced by the variety of assessment methods used and by other artifacts of the assessment process (e.g. mark reliability and differentiation). These above influences appear to prevent the emergence of coherent unit components and confound the interpretation of assessment element marks. In this context, the overall aggregate mark for the year achieves a meaningfulness and usefulness that may not be present in a more detailed profile of marks. That is, if particular groupings of assessments cannot be identified, then an overall aggregate mark will be the most representative and reliable index of the commonality across assessments (e.g. see Nunnally and Bernstein, 1994). Although such an aggregate mark may misrepresent marks from a specific 162

Marks within a First Year Undergraduate Programme

assessment, the above discussion suggests that assessment specific marks may in large part be the result of assessment artifacts. Should these findings be supported across other years and programmes, then there would appear to be a number of difficulties with the Burgess groups suggestion that the overall summative judgment is replaced with a more detailed set of information.

References
Burgess Group final report (2007). Beyond the honours degree classification. London: Universities UK. Corey, D. M., Dunlap, W. P., and Burke, M. J. (1998). Averaging correlations: Expected values and bias in combined Pearson rs and Fishers z transformations. The Journal of General Psychology, 125: 3, 245-261. Fowell, S. and Jolly, B. (2000). Combining marks, scores and grades. Reviewing common practices reveals some bad habits. Medical Education, 34: 785-786. Koretz, D. (1998). Large-scale Portfolio Assessments in the US: evidence pertaining to the quality of measurement. Assessment in Education: Principles, Policy & Practice, 5: 3, 309-334. McLachlan and Whiten (2000). Marks, scores and grades: scaling and aggregating student assessment outcomes. Medical Education, 34:788-797. Nunnally, J.C. & Bernstein, I.H. (1994). Psychometric Theory, 3rd ed. London: McGrawHill. Shavelson, R.J., Gao, X. and Baxter, G.P. (1993). Sampling Variability of Performance Assessments. Journal of Educational Measurement, 30: 215-232. Quality Assurance Agency for Higher Education (2002). Subject Benchmark Statement for Psychology. Gloucester: QAAHE Quality Assurance Agency for Higher Education (2006). Code of practice for the assurance of academic quality and standards in higher education. Section 6: Assessment of Students. Gloucester: QAAHE.

163

PRIME Volume 3(2)

Poster Abstract

Transitions to Identity in Student - Writers

Tony Wailey and Susana Sambade London College of Communication This paper, explores the "in between world" of text, image and student expectation, (Deleuze 2003). It will focus on a group of electives that featured Creative Writing and were offered to all second year students of BA honours and FdA programmes across the London College of Communication, University of the Arts. The main focus of the paper is to consider students expectations: if or how attending these electives affected their ideas of writer identity, was in the way they expected or if it was significantly different. The research explores the work of Etienne Wenger (1991) whose sense of situated knowledge is at the heart of a transition to professional development, meaning perspectives and ownership of identity. In between worlds are areas of transition, in emigration studies of families caught between home and far horizon, in recent cultural writing between the cosmopolitan and the locale (Buck-Moss, 1989 -Panesar 2004)) and are equally the province of learners' negotiating the middle passage of their degree. The methodology used in the research aims to capture multiple perspectives professional writers and student individual voices- and draws upon the work of Ricoeur (2000). It aims to allow participants to give an account of their worldview and how they might negotiate the interweaving paths of professional identity. The research builds upon other academic work illustrating how students not only socially construct the world of creative writing and the relationship between text and image but how this itself establishes frameworks of cultural capital outside of the students specific academic discipline or indeed, life arena. The "in between world" is where the real work takes place and can be located as one of the sources of student expectation.(Wenger,2005)

References
Buck-Moss, S. (1989) The dialectics of seeing: Walter Benjamin and The Arcades Project, Cambridge, MA, MIT Press. Deleuze, J. (2003 ed) The Logic of Sensation, London, Continuum Books Lave, J. & Wenger, E. (1991)Situated learning: legitimate peripheral participation, Cambridge, Cambridge University Press. Panesar J. & Wailey, T. (2004) Migrant Women, London, SoftNet Books, Ricouer (2000) The Just Chicago, University (http://plato.stanford.edu/entries/ricoeur accessed 5 Feb2007) of Chicago Press

Wenger, E. (2005) Communities of Practice, Cambridge, Cambridge University Press 165

Prime Volume 3(2)

(Re)constructing Babel: Mapping the Dynamics of Teacher Training

Beatrix Fahnert, Jeremy Hilton and Joe OMahoney Cardiff University

Abstract
Professionalisation of teaching in Higher Education, corresponding quality assurance and training requirements of the individual teachers were identified as the main discourses associated with outsourced formal teacher training at Cardiff University. Individual and collective needs were researched in a case study involving three Schools and analysed by means of Soft Systems Methodology to draw together the commonalities of the different positions, enabling debate amongst stakeholders (teachers, home Departments/ Schools, training provider) and across disciplines. Once established the directed flow of information can be used as a formal means of communication and a potential audit tool towards meeting the collective needs.

Introduction
In line with the increasing professionalisation of the teaching, and other, professions (Eraut, 1994), Higher Education teachers are required by their employers to undertake formal training in the professional values and practices identified by the Higher Education Academy (HEA). This training (e.g. modules of Post-Graduate Certificate in University Learning and Teaching courses) is mostly outsourced to a specialist Learning and Teaching Unit (or similar phrasing) rather than being conducted locally within each Department/ School. However, agreement on what form this education will take should involve complex institutional negotiation considering differing ideologies, discourses and resources, especially given that many of the students (i.e. teacher training participants) are educators themselves. Understanding and representing such diverse perspectives provides a challenge for training providers and participants alike (Clark, 1984; Shils, 1983; Neumann, 2001). The alignment of such training courses with the HEA can place them in a position of tension, if not contradiction, with those they seek to train. On the one hand, the courses are usually accredited by the HEA and, therefore, aligned to the principles which the HEA has identified as underpinning professional practice in Higher Education. On the other hand, however, the alignment with HEA principles in no way guarantees that the skills developed by the teacher are those which are required by his or her Department/ School or students. The local institutions to which the teacher must return after being trained have varied and changing educational needs for their teachers, and their input into HEA accreditation requirements is often indirect and informal. In terms of the relevant literature, studies which assess the separation of the teaching institution from the training institution generally note the extent to which continuous improvement of teaching standards is related to the presence of communication and support mechanisms between trainers, trainees and the teaching institutions. For example, Prosser et al. (2006) note the difficulties involved with providing training that fits with the needs of all stakeholders and point out that participating institutions are often uninformed of exactly how, 167

Fanhert, Hilton and OMahoney and in what subjects, their teachers are being trained. On the positive side, they note how participants often carry strong ideas back to their institutions to improve the teaching strategies from within. Others, such as Prebble et al. (2004) support this by noting that teacher-training courses lead to long-term benefits only if the trainees receive support from their own Departments/ Schools. This, they found, is especially so of peer-support. Whilst there is significant research into the extent to which teacher training has positive or negative effects on those who participate, for example Rust (2000) or Gibbs and Coffey (2004), there is virtually no research into the issues of perspective and interpretation associated with outsourcing training to a centralised body within a university. Attempting to answer some of these open questions this paper explores the processes by which the realities of teacher education are both constructed and negotiated by stakeholders in the professional education process. Based upon a case-study at Cardiff University, the paper reports on the dynamics which underpin the formation and interpretation of teacher training from different perspectives: university Departments/ Schools, teachers that attended the course and the trainers themselves. The study used 8 semi-structured interviews with senior staff to ascertain the different needs and interpretations of these stakeholders and then deployed a questionnaire (n=52) to compare the expectations of the various stakeholders and the reality of the course. In seeking to explain the differences between these institutional frames, the paper points to three overlapping yet conflicting discourses: the professionalisation of teacher training (Schon, 1987), the quality assurance needs of the Schools (Biggs, 2001) and the functional requirements of the teachers themselves. Having established the differences between the expectations and interpretations of the stakeholders, the paper seeks to represent the commonalities of their needs using a Soft Systems Methodology (Wilson, 2001; Checkland, 1999). This resulted in a high level conceptual framework which could be used as both a benchmark against which to compare current practice and as a tool for translating the differing discourses into a common and shared representation.

Methodology
Since August 2004 Cardiff University requires all probationary teaching staff to complete the first module of a Postgraduate Certificate in University Teaching and Learning (PCUTL). The teaching of the course is undertaken centrally by the Training and Development Section in the HR Division. The study focused on answering the following research questions: a. Who are the key stakeholders in the Cardiff PCUTL course? b. How do these stakeholders construct and interpret their understanding of PCUTL? c. Can there be commensurability between these differing interpretations? The methodology started from a constructivist perspective, assuming whilst different interpretations would exist between PCUTL stakeholders it may be possible to forge a common understanding between different groups (for a methodology overview see Appendix One). Such a foundation enables a methodology which assumes that shared communities of practice exist between different groups, thus allowing inter-disciplinary discussions regarding 168

Mapping the Dynamics of Teacher Training the same object - in this case, teacher training (Wenger, 1998). The process of the methodology was exploratory, seeking to develop an understanding of where the different interpretations were located iteratively rather than seeking to impose a top-down a priori categorisation. To this end, exploratory semi-structured interviews were conducted with the Heads of School and Heads of Teaching at three university Schools (Biosciences, Business and Computer Science). Once themes had been tentatively developed a questionnaire was created using SSM (as described later) and deployed to lecturers within the Schools and those undertaking the PCUTL course (n=52). The enquiry aimed at capturing the discourses, requirements and perspectives associated with different stakeholders involved in both teaching strategy and practice (see Table 1).
Table 1: The Methodological Framework Strategic Perspective School Perspective Teacher Perspective

Interviews with Head of School and Questionnaire to School lecturers to Head of Teaching to identify prioritise the identified training needs. training needs. Interviews with PCUTL Programme Questionnaire to PCUTL participants to Manager and Section Manager to assess relevance and quality of training identify training provision. provision.

PCUTL Perspective

Similarities and Differences

The findings from the survey (see Appendix Two) indicated a number of similarities and differences between the expectations of students/ participants, Schools and PCUTL managers. In terms of consistencies, most stakeholders agreed on what being a good lecturer means. Despite this, a number of differences existed between the perspectives in terms of what was expected from teacher training. For example, participants did not expect all of their training support to come from PCUTL and felt instead that guidance in some areas should be provided by Schools. For example, whilst over 50% felt the following attributes were essential to being a good lecturer, less than 50% felt these attributes should be developed at PCUTL: Subject matter (i.e. the trainees own subject matter expertise) Research-led teaching (i.e. teaching informed by the trainees own research) Industry practice and needs (i.e. the needs of the industry environment) Pedagogic research (i.e. the trainees own research into their own teaching) Being organised (for example, project and time management) Being scholarly Similarly, whilst 70% or more of lecturers agreed that lessons on teaching technology, health and safety and dealing with problem situations should be taught by PCUTL, less than 50% of participants felt this was the case. Whilst more time could be spent outlining the similarities and differences between participants, Schools and PCUTL management, it is more interesting to turn to the qualitative data (i.e. interviews with stakeholders) and discover the reasons for these differences.

169

Fanhert, Hilton and OMahoney

Underlying Discourses
The differing priorities in terms of teacher training of PCUTL, Schools and teachers as revealed by the survey may, from a traditional consultancy perspective, seem to reflect poor or ineffective communication of the lecturers training needs (given the challenges faced in teaching) via their Schools to their outsourced training provider PCUTL. Whilst there was (very occasional) communication from Schools to PCUTL (especially when it was set up) there was no formal process in place to ensure that teacher-training requirements are generated from teachers and passed to PCUTL. Indeed some Schools intend only to review provision once a number of teachers have completed PCUTL and the benefits can be ascertained. Optimal communication is crucial and to be expected in a system where the first priority is the effective teaching of students facilitated by optimised training of their teachers. Instead, interviews and survey data showed that each group of stakeholders is founded within institutional frameworks generated through distinct discourses. PCUTL discourse, for example is rooted in the HEA definition of professionalism. PCUTL was to conform to the HEA Professional Standards Framework and the course including assessment criteria are explicitly structured around these values. The professionalisation debate clearly points to the aims of standardisation and control (Baume 2006) as well as occupational ring-fencing (Jeffcutt, 2004). School management on the other hand is focused on the discourse of accreditation achieved in a short a time as possible. Thus School managers want to ensure via provision of a basic training that teaching is certainly of a minimum standard; their main priority is to ensure sufficiently clear processes which will support the accreditors quality audits (see Table 2). Turning to the teachers themselves, their underlying discourse appears to be that of teaching itself. The needs they expressed were generally of a practical nature focusing on teaching students well: using technology, being a strong communicator, and dealing with problem situations. This is not to say that there are no other discourses present in the teacher-student relationship. For example, the pressures of research, publications and administration appear to press at least equally heavily on teachers time. However, as concerns the PCUTL relationship, the primary focus of teachers appears to be on educating their students well. Table 2: The Discourses
Main Discourse PCUTL Management School Management PCUTL Participants Professionalisation Purpose Structure

To maintain standards and HEA communicate good practice in the teaching discipline To have evidence of required AACSB / EQUIS / capabilities of their teachers AMBA / QAA/ other quality audits To activate students learning processes effectively as possible. deep Student feedback/ Selfas assessment/ Course QA and QE/ Mentoring/ Appraisal/ Peer review

Accreditation

Teaching

170

Mapping the Dynamics of Teacher Training Whilst the discourses of accreditation and professionalisation do commit to supporting the quality of teaching, both commit to a minimum standard which states that a teacher should be capable of demonstrating X rather than actually being good at X. Thus a teacher can demonstrate respect for individual learners, commitment to developing learning communities etc., but still be a poor quality teacher. The demonstration is partially based on a reflective portfolio. One intended outcome of the teacher training (as well as the discourses of accreditation and professionalisation) is a self-directed continuation of the teachers professional development and informed by reflective practice taking into account e.g. student and peer feedback. The feedback procedures are in place in Schools as part of quality enhancement and assurance. It is the teachers responsibility to aspire to be a good quality teacher making use of what was learnt in the training when acting on the feedback. Thus directed and informed communication of certain aspects between all stakeholders (discourses) is required in order for all to benefit from the training provided to teachers. Designing a framework in support of a communication that efficiently meets these requirements could positively combine all agendas without adding work or administration. An approach, Soft Systems Methodology, was explored through a proof of concept to test its suitability for providing such a framework.

Soft Systems Methodology: Searching for Common Ground

Despite the finding that PCUTL, School Management and teachers have differing fundamental discourses, this does not mean that these positions are incommensurable. Indeed to a great extent, all parties agreed that one requirement of the PCUTL course should be that teachers improve the quality of their teaching. To ensure all parties gain the benefits they require of PCUTL, they need to know what they require, and the managers of the PCUTL programme need to know the collective set of requirements to provide a complete offering. This is required so all parties can get what they need from PCUTL without detriment to others. Therefore, there is a need to analyse the collective needs and agree a universal set through dialogue. Soft Systems Methodology (SSM) (Wilson 2001) is used in this context to develop an objective model and shared vision as well as achieve a common language against which to compare the provision of teacher training through PCUTL. Two statements of purpose (Root Definitions) had been developed (included in Appendix One). From these activities were modelled that one might consider undertaking in order to achieve the purpose defined. These had then been used to inform the design of the survey questionnaires. The benefit of this approach is that an explicit mental model is produced which leads to defensible outcomes. As the Root Definitions were based on the interviews, objectivity in thinking could be maintained. Given the scope of the project the method was not applied as rigorously as one might, but in a light manner as a proof of concept; for example, Heads of School or PCUTL participants were not approached in a follow up session to gain their agreement to the Root Definitions. This is planned to be the subject of further studies. However, the activities could be identified with confidence on the basis of the model. It has to be stressed that this is not a model of reality, or a solution, but merely a first attempt to produce a common understanding of what the system might be (see Appendix One; Figure 171

Fanhert, Hilton and OMahoney 2). The next step was to consider the activities identified. This consideration informed the generation of questions for the questionnaire. The response to the questionnaire and the results gained validated the use of SSM in this manner, and the next step will be to use SSM more rigorously. PCUTL would need a repository of the various discourse opinions to inform PCUTL management. The development of the PCUTL programme to encompass the repository will enable Schools and PCUTL participants to tailor their learning e.g. core and optional aspects to fit their needs. The repository should also be used to capture results to demonstrate the effectiveness of the PCUTL programme. Linking the activities in the Conceptual Model to the repository is through analysis via a table (such as Figure 1) Figure 1: Analysis Tool
A Activity B Is it done? C Whom is it done by? D How is it done? E PCUTL offering F Quality criteria G How well is it done? H Potential change I Impact of change

1. The activities from the conceptual model (e.g. such as Figure 2 in Appendix One) are listed under A. 2. The various stakeholders are consulted to agree the objective criteria by which the success of the activities should be judged and listed under F. 3. With the above agreed, relevant parties are asked to consider whether the activity is carried out (to be listed under B), and if not, whether it should. Moreover, it is recorded who carries out the activity (under C) and how it is carried out (under D). If an activity is currently not carried out, and it is felt it should be, it is also recorded how it might be carried out and whom by. 4. Some of these activities will result in specific learning opportunities for PCUTL participants through core or optional topics (list under E). 5. Using the quality criteria, an opinion as to how well that activity is carried out is recorded (under G). 6. The result of the previous step will lead to a consideration of potential changes (list under H) to improve the success of the activity, often whilst . considering the impact of any such change (list under I).

Such an analysis can be undertaken based on the collected data and to engage with PCUTL management and the Schools involved to date to evaluate the outcomes. Thus SSM also enables an audit trail from the outcomes back to the original statements of purpose. Should any of the stakeholders change their opinion of the purpose of PCUTL, including HEA accreditation requirements, then by restating the Root Definitions, and developing a new conceptual model, changes in activity can be identified and the repository of PCUTL needs to be updated accordingly. This would then be reflected in revised offerings to PCUTL participants.

172

Mapping the Dynamics of Teacher Training

Conclusions
Through the use of interviews and a survey, this paper demonstrates the differences in priorities of the key stakeholders in an outsourced teacher-training function. Rather than put these differences down to initial poor communication, the paper suggests that the differing stakeholders are founded in differing discourses which help determine their priorities. However, ongoing communication among the stakeholders has to be adapted by means of a common language that acknowledges the existence of different initial priorities to support the identified and agreed common priority that PCUTL participants should improve the quality of their teaching. This is the number one priority of the teachers themselves and in the interest of PCUTL and School management as well, but within the framework of professionalisation and accreditation, respectively. After outlining the differences between these discourses the paper attempted to move the debate forward by arguing that, whilst differences in priorities exist, all stakeholders are committed, in varying degrees, to the development of teaching quality. To this end, a system was designed which could act as an agreed basis for moving the discussions forward. The light use of SSM was useful in bringing together the various discourses and presented an objective view representing the commonalities of their needs, albeit at a high level. It did enable the differences between the expectations and interpretations of the different stakeholders to be identified through the use of questionnaires. The use of SSM also resulted in a high level conceptual framework which could be used as both a benchmark against which to compare current practice and as a tool for translating the differing discourses into a common and shared representation. This is to be further explored through subsequent research which will examine the potential of using SSM to bridge discourses in creating commensurate understandings across different perspectives. By seeking to depict the commonalities of a system which can satisfy the three perspectives, SSM does not aim to neutralise the discourses. Instead it aims at providing a flow of information which connects different stakeholders at a strategic and operational level. The paper accepts that SSM has its own discourse, language and structure in other respects, but holds that this is amenable to constructing a common ground between competing perspectives.

References
Baume, D. (2006) Towards the end of the last non-professions, International Journal for Academic Development, 11 (1): 57-60. Biggs, J. (2001) The reflective institution: assuring and enhancing the quality of teaching and learning, Higher Education, 41 (3): 221-238. Checkland, P. (1999) Soft Systems Methodology in Action, Chichester: John Wiley and Sons Ltd. Clark, B. (1984) Perspectives on Higher Education: eight disciplinary and comparative views, Berkeley, CA: University of California Press. Department of Education and Skills (2003) The Future of Higher Education. White Paper, January.

173

Fanhert, Hilton and OMahoney Eraut, M. (1994) Developing Professional Knowledge and Competence, London: Falmer Press. Gibbs, G. and Coffey, M. (2004). The impact of training of university teachers on their teaching skills, their approach to teaching and the approach to learning of their students, Active Learning in Higher Education, 5 (1): 87-100. Jeffcutt, P. (2004) The Foundations of Management Knowledge, London: Routledge. Neumann, R. (2001) Disciplinary differences and university teaching, Studies in Higher Education, 26 (2): 135-146. Prebble, T., Hargraves, H., Leach, L., Naidoo, K. Suddaby, G. and Zepke, N. (2004) Impact of Student Support Services and Academic Development Programmes on Student Outcomes in Undergraduate Tertiary Study: A Synthesis of the Research. Research Report, Research Division, New Zealand Ministry of Education. Prosser, M., Rickinson, M., Bence, V., Hanbury, A. and Kulej, M. (2006) Formative evaluation of accredited programmes, Report, The Higher Education Academy. Rust, C. (2000) Do Initial Training Courses Have an Impact on University Teaching? The Evidence From Two Evaluative Studies of One Course, Innovations In Education & Training International, 37 (3): 254-262. Schon, D. (1987) Educating the Reflective Practitioner: Toward a New Design in Teaching and Learning in the Professions, San Francisco: Jossey-Bass. Shils, E. (1983) The Academic Ethic, Chicago, IL: Chicago University Press. Wenger, E. (1998) Communities of practice: Learning, meaning and identity, Cambridge: Cambridge University Press. Wilson, B. (2001) Soft Systems Methodology: Conceptual Model Building and Its Contribution. Chichester: John Wiley and Sons Ltd.

174

Mapping the Dynamics of Teacher Training

Appendix One: Method Sequence

1. Interviews were undertaken with the Heads of each School and, where available, the Heads of Teaching. These were qualitative and semi-structured. Interview Questions What are the general teaching needs of the School? What is the process for identifying and managing teaching needs/ skills? What would be your idea of an ideal teacher? How is this vision put into practice in a research-led environment? What are your teachers education needs? How do you know? What do you understand PCUTL to be? What do you want PCUTL to do for your teaching-education needs? What do you believe your teachers get from PCUTL? Is there any other teaching-education provision in the School? Were you/ the School involved in designing the PCUTL course? 2. Root Definitions of the purpose of PCUTL were developed from the interview results. From these, a conceptual model (see Figure 2) was developed to identify common requirements across the three Schools. Root Definitions for PCUTL RD1 Assumed PCUTL viewpoint A system, owned by Cardiff University, operated by relevant Cardiff University staff, visiting experts, skilled and experienced mentors, and recently appointed lecturers to develop effective teaching and learning skills in recently appointed lecturers by providing a course based on seminars on a range of topics, including learning outcomes, course preparation and delivery, assessment, and educational theory, requiring a group project, and culminating in the preparation of an assessed portfolio, within the constraints of relevant Cardiff University policy, the needs of the various Schools within Cardiff University and the requirements of a research-led University. RD2 Schools viewpoint A system owned by the Head of School, operated by relevant Cardiff University staff, visiting experts, skilled and experienced mentors, and recently appointed lecturers, to engender in recently appointed lecturers the skills and attributes required for quality teaching and student learning, by providing the necessary knowledge and support for personal development such that the recently appointed lecturers have the knowledge, skills and attitude required to maintain the necessary professional standards as befits a world-class research-led university. 3. Lecturers within each School were surveyed by means of questionnaires based on the SSM results to prioritise these requirements, to state whether the requirement identified by the Heads was important to being a good lecturer and whether they believed that meeting the requirements should be part of the PCUTL provision. This was an important step because there would be no point assessing whether a requirement was adequately catered for by PCUTL if it was unfeasible to provide it.

175

Fanhert, Hilton and OMahoney 4. PCUTL Programme Manager and Section Manager were interviewed using the questionnaire items to assess whether there was a strategic intention to provide the stated requirements. Details were sought on how the requirement was provided (e.g. workshops, assignments and coursework) and whether the provision of the requirement was measured.
5. PCUTL participants were asked using the questionnaire to assess whether (in their opinion

the requirement is provided by PCUTL and how well and whether it should be provided by PCUTL. Figure 2: Conceptual Model

Appendix Two: Findings

Interviews with the Heads of School and Heads of Teaching revealed remarkable consistency regarding what they believed good lecturing involved. These were split into requirements regarding knowledge (to know) and attitudes (to be):

176

Mapping the Dynamics of Teacher Training

Table 3: Knowledge Requirements of Good Lecturers

Subject matter Educational theory Teaching technology Learning outcomes Educational landscape Teaching and delivery methods Research led teaching Industry practice and need Health and safety Student behaviour Course design Assessment setting Pedagogic research Problem situations Appropriate teaching level Relevant teaching Know one's subject matter thoroughly Know different educational theory and how it informs practice Know how to use different teaching technologies (e.g. white-boards, PowerPoint) Know how to construct and assess learning outcomes Know about the changing educational environment (e.g. accreditation, student no.'s, overseas students) Know different methods of teaching delivery Know how to provide teaching based upon one's own research Know relevant industry requirements for employees and current practice Know health and safety aspects/ regulations for teaching practice Know how students learn and respond in different teaching environments Know how to design and structure different types of courses and programs Know how to set assignments at an appropriate level whilst maintaining standards Know how to undertake research into different learning and teaching issues Know how to deal with disruptive students Know how to provide different teaching styles for different class sizes, levels and student nationalities Know how to show why one's teaching area is relevant and important

Table 4: Attitudinal Requirements of Good Lecturers

Supportive Scholarly Inspirational Proactive Reflective Skilled Organised Professional To be responsive to students, showing flexibility and care To be learned and intellectual To be motivated and challenge/ engage students To take the initiative, be confident and provide leadership to students To be reflective and manage one's development needs To be a strong communicator and presenter To be able to manage time and self-organisation effectively To be a professional teacher

When these requirements were put to lecturers themselves there was, again, consistency, with none scoring less than a 50% agreement that the requirement was, indeed, important in becoming a good lecturer. It should be noted that 19 of the 23 requirements scored over 60% agreement.

177

Fanhert, Hilton and OMahoney When lecturers and PCUTL participants were asked if they believed that the provision of the required skills and knowledge should be part of PCUTL training, the following requirements scored over 60% agreement levels: Table 5: Requirements Lecturers believe should be provided by PCUTL
Educational theory Teaching technology Learning outcomes Educational landscape Teaching and delivery Health and safety Student behaviour Course design Assessment setting Problem situations Appropriate teaching Relevant teaching Inspirational Proactive Reflective Skilled Professional Supportive Know different educational theory and how it informs practice Know how to use different teaching technologies Know how to construct and assess learning outcomes Know about the changing educational environment Know different methods of teaching delivery Know health and safety aspects/ regulations for teaching practice Know how students learn and respond in different environments Know how to design and structure different courses and programs Know how to set assignments at an appropriate level Know how to deal with disruptive students Know how to provide different teaching styles Know why one's teaching area is relevant and important To be motivated and challenge/ engage students To take the initiative, be confident and provide leadership To be reflective and manage one's development needs To be a strong communicator and presenter To be a professional teacher To be responsive to students, showing flexibility and care

Lecturers and participants agreed that the following should NOT be provided by centralised teacher training: - Knowledge regarding the lecturers subject matter - Research-led teaching - Specific industry practices - Pedagogic research Teachers felt not enough training is provided in the following areas: - Knowing about the educational landscape - Dealing with problem situations - Being motivated and inspirational - Being a strong communicator and presenter PCUTL do not currently aim to provide training on Dealing with problem situationsand a strategic decision would need to be made on this for future activities. Being a strong communicator and presenter is fundamental to lecturing and these skills could be built more centrally into the PCUTL provision of training.

178

Mapping the Dynamics of Teacher Training Schools might review the provision of: - Being knowledgeable in subject matter - Providing research-led teaching - Knowing industry expectations and practice - Undertaking pedagogic research - Being organised - Being scholarly A formal feedback loop (e.g. by means of the introduced model) connecting PCUTL participants, School and PCUTL management could help the combination of the discourses and has been found not to be currently in place. The following requirements are considered the most important ones in becoming a good teacher and are well provided by PCUTL: - Knowing how educational theory informs practice - Using learning outcomes - Teaching delivery methods - Knowing how students behave in different environments - Designing courses - Being reflective - Being a professional teacher - Being supportive of students - Participants should focus on the training provided. Table 6: Questionnaire results
LECTURERS 33 % returned
How important to being a good lecturer? Should this be provided?

PCUTL PARTICIPANTS 22 % returned

Is this provided by PCUTL? Should this be provided? How well is this provided?

PCUTL MANAGERS
Is this provided by PCUTL? Is this output measured?

Rate 1 5 Subject matter Educational theory

Know one's subject matter thoroughly Know different educational theory and how it informs practice Know how to use different teaching technologies (e.g. white-boards, PowerPoint) Know how to construct and assess learning outcomes Know about the changing educational environment (e.g. accreditation, student no.'s, overseas students)

Yes in % 2

Yes in % 8

Yes in % 8

Rate 1 5 2.2

Yes / No Yes

Yes / No Yes

4.9

2.7

90

83

75

3.4

Yes

Yes

Teaching technology Learning outcomes

3.7

76

25

67

2.8

Yes

No

3.5

88

92

83

4.2

Yes

Yes

Educational landscape

2.5

57

25

67

Yes

Yes

179

Fanhert, Hilton and OMahoney

Teaching and delivery methods

Know different methods of teaching delivery Know how to provide teaching based upon one's own research Know relevant industry requirements for employees and current practice Know health and safety aspects/ regulations for teaching practice Know how students learn and respond in different teaching environments Know how to design and structure different types of courses and programs Know how to set assignments at an appropriate level whilst maintaining standards Know how to undertake research into different learning and teaching issues Know how to deal with disruptive students Know how to provide different teaching styles for different class sizes, levels and student nationalities Know how to show why one's teaching area is relevant and important To be motivated and challenge / engage students To take the initiative, be confident and provide leadership to students To be reflective and manage one's development needs To be a strong communicator and presenter

4.3

100

92

83

3.9

Yes

Research led teaching Industry practice and need

3.7

27

50

1.3

Yes

2.9

22

33

42

1.7

Yes

Health and safety

3.1

72

50

67

3.7

Yes

Yes Yes (if included in portfolio) Yes (if included in portfolio) Yes (if included in portfolio)

Student behaviour

4.1

93

83

83

3.7

Yes

Yes

Course design

4.1

84

100

75

3.5

Yes

Yes

Assessment setting

4.5

86

92

83

3.4

Yes

Yes

Pedagogic research

2.5

46

67

75

3.1

Yes

Problem situations

3.8

92

33

100

No

Yes Yes (if included in portfolio)

Appropriate teaching level

4.1

100

83

75

3.1

Yes

Yes

Relevant teaching Inspirational

3.9 4.8

51 64

17 58

67 50

2.6 2.3

Yes Yes

Yes Yes

Proactive

4.4

58

17

58

2.5

No

Reflective Skilled

3.9 4.5

68 78

75 33

83 58

4 1.8

Yes Yes

Yes Yes (if included)

180

Mapping the Dynamics of Teacher Training

To be able to manage time and self-organisation effectively To be a professional teacher To be responsive to students, showing flexibility and care To be learned and intellectual

Organised Professional

4.3 3.4

35 53

8 83

50 83

1.5 3.9

No Yes Yes Yes (if included in portfolio) Yes

Supportive Scholarly

4.5 3.4

51 11

75 42

67 67

3.8 3

Yes Yes

181