You are on page 1of 9

Online Assignment Management: An Evolutionary Tale

David Jones Faculty of Informatics and Communication Central Queensland University d.jones@cqu.edu.au Sandy Behrens Faculty of Informatics and Communication Central Queensland University s.behrens@cqu.edu.au

Abstract
Due to the potential benefits there has been widespread interest in the use of online assignment submission and management (OASM). However, much of the reported work is limited to small-scale use usually in courses taught by innovative staff. This paper draws on experience with OASM in a growing number of courses taught by a number of different staff from different disciplines. It aims to investigate the process by which OASM is adopted across an organization and identify the factors that influence its growth, adoption, and adaptation. In particular the paper demonstrates that the introduction and successful use of OASM requires changes in practice, the ability to cater for a wide range of requirements, and a continual questioning of the appropriateness of the resulting changes to practice.

1. Introduction
Assessment for many University courses includes some form of assignment which students complete, individually or as groups, and submit for marking and feedback. Assignment management involves the collection, date stamping, redistribution to tutors for marking, collation of results, and return of assignments to students [1]. Tregobov [2] breaks the process down into four stages: submission, recording, marking, and return. Online assignment submission and management (OASM) involves the use of the World-Wide Web, the Internet and computers in this process. There are a number of problems with traditional approaches to the submission and management of student assessment, especially in a distance education setting [3]. Application of OASM promises to address many of the problems of traditional approaches and offers the potential to adopt a wide range of new practices. Consequently, there has been widespread interest in OASM [1-20, 2830]. However, much of that literature is of a limited nature. Usually describing the features, characteristics and initial use of these systems in a limited number of courses taught by a small number of innovative staff that are usually the authors of the subsequent papers.

Implementing technology, even implementing technology well, provides no guarantee that the system will be used or be effective [23]. There is a long history of failed technology-based innovations in education [24]. Many such projects fail due to the innovators underestimating the consequences of new technologies [26] and failing to accommodate environmental and contextual factors affecting implementation [22]. This paper seeks to contribute to the understanding of OASM by proposing a model that encapsulates the issues, challenges and opportunities encountered when OASM is adopted by a wider range of faculty. This model has grown out of the use of OASM within the Faculty of Informatics and Communication (Infocom) at Central Queensland University (CQU). An important aspect of the model is that the adoption and use of OASM is an ongoing, evolutionary tale that is continually driven by these issues, challenges and opportunities. As a result any decision, which limits that evolution, can limit future benefits and adaptation. The paper commences with a summary of the potential benefits of OASM and previous work in the area. It then introduces the model that seeks to explain the factors driving the evolution of OASM. The following section uses the model to describe Infocom's experience with OASM since 1994. Finally, the paper briefly discusses the implications of this model.

2. Previous work
All teaching and learning requires administrative support [21]. Much of this administrative support is to some degree transparent but if performed inefficiently can become immediately obvious and can distract students and staff from the learning process [1]. Online Assignment Submission and Management (OASM) is one administrative task that can consume much of a course coordinator's time [1]. Reducing the amount of time consumed, addressing other problems, and benefiting from new practices are some of the reasons that have driven many staff to adopt and use OASM. Darbyshire [1] reports finding over 6000 hits from web search engines for pages describing online assignment submission procedures to students. In

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

addition there is a large amount of literature written on the topic. This section outlines some of the reasons driving the adoption of OASM and provides an overview of much of this previous work.

2.1. Reasons for using OASM


All of the reasons for adopting OASM described in the following paragraphs are based on three main technological characteristics of OASM. 1.Almost uniformly quick transmission of assignments providing geographic independence for students and staff. 2.The digital format of assignments and the inexpensive nature of computer storage enabling the creation of a single central storage place for student assignments and related data. 3.The information processing capabilities of computers that allow for a wide range of analysis, manipulation and reporting. The most obvious advantage offered by online assignment management is that it offers faster transportation of assignments than traditional, physical methods. In a study focusing on the use of online assignment management in a programming course Price and Peter [17] found that students reported a decrease in assignment turnaround time from 2 weeks down to 5 to 7 days. Chi-Sang et al [7] report a reduction in the assignment submission cycle from 2-3 weeks to 1-2 hours. The best result achieved through OASM at CQU resulted in the return of 72 assignments from students throughout Australia and South-East Asia within just over 3 days (1 working day). The resource intensive nature of assessment can often decrease the quality of the feedback provided to students [16]. Much of the marking process is mechanical, repetitious, and a perfect candidate for the application of information technology. Through appropriate application of information technology it is possible to reduce costs and increase both the quantity and quality of assessment and feedback to students [21]. One Infocom course coordinator reports the following form his experience of using OASM: Students are very happy with that, they get plenty of feedback and it's doable by the markers. No negative feedback from markers or students.....some students were saying this is the best feedback I've had in any course.. When supported appropriately with resources, experience, and minimal automation it has been found that marking online assignments takes 20-30% less time than marking traditional assignments [3]. Summons et al [18] reports tutors marking being reduced by an average of 67% by using a semi-automated marking process. The

greater the automation used the greater the reduction in marking time. For a number of reasons achieving accuracy in assessment is vital [19]. Any problems, perceived or otherwise, with the accuracy of assessment reduces student confidence. A CQU student reports: I had 2 assignments marked incorrectly and another one the results were not recorded at all (this went unnoticed till I saw my final result), it makes me wonder just how many other errors are slipping by unnoticed. Appropriate reporting mechanisms combined with online assignment management increase the transparency of the marking process by allowing students to view the progress of their assignment. Systems with built-in automatic logging of the process increase accountability [17]. Automation of manual processes such as transcription of results can help reduce errors. Electronic marking of assignments also removes problems associated with the illegibility of handwritten marker comments [17]. A CQU staff member reports on a related problem: a tutor in Melbourne went missing, this was a paper-based submission, and just disappeared with about ten assignments and I had no marks and I had no assignments either. If they had been electronically submitted it would have been really easy just to pull it off the web page and send it down to the other marker... Another staff member adds: Just the flexibility, you know, if a marker makes a real hash of things we can jump in and do it ourselves or get someone else to do it...... Upon graduation most CQU graduates from the regional campuses move away to capital cities. This means that there is a limited population of qualified graduates in regional areas. This can make it very difficult to find appropriate markers for advanced level courses. Use of OASM offers a solution to this problem as one Infocom staff member discovered. another advantage is that one of my students is now a graduate. He's now working in Canberra. He's keen to help me out from that distance, so he's going to do some marking for me. Large class sizes make it necessary to introduce a large number of markers. Maintaining consistency between multiple markers is a major problem in large classes [20]. Mason and Woit [15] believe that online marking of assignments can lead to improved marking consistency and integrates well with online mark reporting. Moderation of assignment marking is an increasing problem at CQU where an individual course can have over 10 markers spread geographically throughout Australia and South-East Asia. OASM can help in the moderation process as two Infocom staff found.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

...an electronic copy of their assessment is really, really handy because I can do parallel moderation with the marker, so the marker can be away marking and I can be moderating or assessing what I think the marks would be, and I can actually look at the difference, so there's no lag to the moderation. OASM enables a single submission process that can be used by all students regardless of physical location. As one Infocom staff member found It creates an equality between on-campus students and flex students no matter where they are because they all want to be able to submit at the same time. So theres nothing like as long as your assignment is post-marked by Monday you will be accepted in when the assignment was due in by Friday afternoon. There is an equality in that, it will be Friday afternoon, your assignments are here instantaneously, no excuse otherwise. And I think that they detect that as well, they all know that theyve all got the same deadline. So thats good. Paper-based submission introduces a number of other problems as two Infocom staff report also just not dealing with stacks of paper, you stick it on a disc and take it home, that's your marking on a disc and you can take it home rather than this great pile of rubbish. Yes, well there's still assignments upstairs that the student's haven't collected. In fact, there are still a heap left over in that box there from Winter term, people never bothered to come back and collect them. At least when you email them all back, they're gone. A study by the Center of Academic Integrity at Duke University in 1999 found that 68% of the 2,100 students polled said that they had committed at least one academic offense such as plagiarizing [25]. CQU academics, like many around the world, believe the ease of access to information provided by the World-Wide Web along with a number of other factors is increasing the levels of student plagiarism. Definitely, definitely there is a general feeling that we have problems, plagiarism problems, on the International campuses where we are identifying a number of incidents.......So, I'm sure we're all looking for methods for dealing with that problem. However, using the information processing capabilities of computers, drawing on electronic copies of student assignments submitted via OASM, and using access to Web search engines detecting plagiarism can be a relatively straightforward task. There is a growing range of available systems, both commercial and free, that enables staff to perform plagiarism detection [27].

Apart from detecting plagiarism OASM also offers the potential to avoid it. With support from the technology and appropriate assignment design it is possible to generated individual specialized assignment items that allow students to collaborate without fear of plagiarism [14].

2.1. The literature


With all these possible advantages it is not surprising that there has been widespread interest in using OASM. This section provides a brief overview of much of the literature covering OASM. As stated previously little of this literature attempts to examine the process by which OASM is developed and used throughout an organization. We have broken the literature on OASM we have found into four categories: 1.System descriptions and initial use. These papers offer a description of the features of a particular system. These systems generally differ on the basis of technical characteristics and capabilities. The papers describe the initial use of the system within a small group of courses [1, 2, 3, 6, 14, 28]. 2.Investigations of electronic marking. Given the time consuming nature of marking it is not surprising that the next most popular paper category examines the use of computers to fully or semi-automate the marking process [11, 16-19, 29, 30]. 3.A feature of a course management system. Most of the systems described in first category are stand alone, locally developed systems. Recent years has seen the advent of integrated course management systems that attempt to offer all features for Web-based education including online assignment submission. Papers in this category describe the features of such course management systems including brief coverage of its online assignment submission features [7-9]. 4.Part of an online teaching experience. OASM is described as part of the experience of teaching a course along with a collection of other online technologies [4,5,10].

3. The model
Drawing on existing publications [3, 4, 5], results of student surveys, interviews with staff, and system statistics a range of factors have been identified as influencing the evolution of OASM within Infocom. Those factors have been categorized and the model shown in Figure 1 is an attempt to represent their relationship.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

Infrastructure Individuals Course System Experience Society Technology Organisational


Figure 1. Factors influencing OASM Evolution The rest of this section briefly explains each of the components of the model in Figure 1. The factors identified in Figure 1 interact in a complex and interdependent way to influence the evolutionary development of OASM. The model proposes that the evolutionary development of assignment submission and management starts with the existing system. Be it traditional or OASM. The experience gained with the existing system will drive whether or not changes are required. It is the assumptions, characteristics, conceptions, and perceptions of the individuals using the system, which are important. If they perceive few problems with the existing system then the requirement for change is reduced. Staff who are not familiar with technology or prefer traditional methods will not adopt OASM. In many cases the perceptions of individual staff will be influenced by the characteristics of the courses they teach. Characteristics including delivery modes, pedagogy, content, and assessment methods will influence the evolution of OASM. A course offered entirely to oncampus students has less of a need to use OASM than a distance education class. A pedagogy based around live presentations (for example, a drama class) is not conducive to using OASM. The current state of technology will influence what is possible. For example the advent of viruses in Word documents encourages an adoption of different file formats. The availability of simple and cheap computers and Internet access are a requirement for the adoption of OASM. The available infrastructure for the implementation of OASM also plays a large part. The absence of sufficient servers, network bandwidth, or technically competent support staff also inhibits the adoption of OASM. The policies, procedures, and structure of the organization in which OASM is used will also influence its evolution. An organization, which limits academic freedom and experimentation, can limit the growth of OASM. The wider context for OASM is set by the

society in which the organization operates. Legal requirements, perceptions of technology, and other issues from the wider society can also inhibit or encourage OASM evolution.

4. Evolutionary development of OASM


New System
The use of OASM examined in this paper began in 1994 with the use of email. Since then it has undergone an evolutionary development process influenced by the factors identified in Figure 1. This section provides an indication of how these factors have influence the evolution of OASM at CQU through five separate stages. Each stage starts with a description of the experience with the existing system. This is followed by a description of how that experience, along with the other factors from Figure 1, influenced the development of the next stage in OASM. In the eight years since 1994 OASM at CQU has grown from ad hoc use of email by 20 students in one course to a fully integrated, Web-based interface used by over 2300 students in 11 different courses.

4.1. Manual email (1994-1995)


Experience. Traditional print-based submission of assignments at CQU in 1994 suffered from a number of problems which when combined increased staff workload and assignment turnaround time [3]. Particularly frustrating was the 4 to 6 days an assignment would take to travel for a distance education student and back. Course. One of CQU's courses, taught by the first author, was an advanced level computing course in Systems Administration with 54 distance students and 45 on-campus students. The provision and maintenance of Internet-based services was part of the content for the course and students were required to have access to a computer. All of the assessment for the course involved the production of computer files. Individuals. Both students and staff in this course had significantly greater familiarity and access to computer technology than the general and university population. The casual marker employed was relatively inexperienced with the technologies and had some difficulties. The course coordinator had naive ideas about the promise and impact of technology. Technology. In 1995 SLIP/PPP connections to the Internet were slowly becoming available. Internet service providers were not common and most students relied on long distance phone calls to University modems. The primitive nature of email programs and huge variety in encoding styles for attachments made submitting computer files somewhat problematic.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

Society. During 1995 the major Australian telecommunications company introduced a special long distance rate for phone calls late at night that made Internet access considerably cheaper for students. Infrastructure. The course coordinator had a personal UNIX server already hosting email lists for the department that was used to host mail folders. Organizational. Individual academics had the freedom to innovate and experiment within their courses that allowed the adoption of email based assignment submission. New System. As a result of these factors the use of email for the submission of assignments was made optional for distance students in this single course. Assignment files were attached to messages and sent directly to the course coordinators email address. The assignments were manually forwarded to and from markers and students. Only 20 of the 54 distance students made use of email assignment submission.

submission process was also used in the 1996 offering of that initial course. Technology. During this time GUI-based Internet tools were becoming available and interest in the Web was becoming more prevalent. The combination of increasing numbers and increasing Internet access led to over 50 students using the system during the second half of 1995 and the first half of 1996. Individuals. Due to the relationship between the two courses the majority of students using this system in 1996 had used the system previously. The course coordinator now had increased experience with online assignment submission. In particular, the importance of changing practice and the value of sufficient training was known. A new marker with greater familiarity and access to technology was employed. Infrastructure, Organizational, and Society. Remained much the same during this period.

4.2. Automated email (1995-1996)


Experience. Some of the problems experienced with manual email submission was similar to those reported by Young, McSporran, and Destow [10]. These problems included: students using the wrong address, vulnerability to virus infection, duplication of administration effort, and significant administrative burden for the lecturers who had to confirm receipt by email. The major problem was that even with only 20 students using the manual email submission system the amount of work for academic staff was significantly greater than the traditional approach [3]. This workload was increased by the primitive nature of the technology and resulting problems. The average turnaround time on four assignments was 25 days, a little bit worse than the average turnaround time for the traditional approaches. New System. The major addition to the new system was an increase in the level of automation. To submit an assignment students would send an email to a specified email address with a special subject line. A script would receive the students submission email, send a receipt to the student, forward the assignment to the marker, and update a web page that showed submission and marking progress. This new system was optional in the second half of 1995 and made compulsory in the first half of 1996 as part of a totally online approach to teaching and learning [4]. Course. In the second half of 1995 the first author was responsible for another advanced computing course (Operating Systems). This course, with 250 students was a pre-requisite for the course that used the manual email submission process. The new automated email

4.3. Non-integrated Web (1996-1997)


Experience. With the automated email system and a capable marker it was possible to reduce the average turnaround time during this period to 9 days (including weekends). The average turnaround time for the traditional submission method in other courses at this time was 20+ days. The difference is not only due to the benefits of OASM but also attributed to a more organized marking process. The form of automation still left too much freedom and contributed to increased workload due to student mistakes including: using the wrong subject format, wrong message structure, and attaching files in the wrong format. The initial use of both OASM and traditional methods created extra work for the marker. Due to problems with email attachment formats and the non-standard and primitive nature of the tools marking online assignments was taking 2 to 10 minutes longer than marking traditional assignments. The report page though, which allowed students to see how they were progressing compared to other students, was well received. New System. The use of email, even with automation, was for various reasons problematic. With the increasing presence of the Web the OASM system moved entirely onto the Web. However, this system was not integrated with any organizational system so students used just their student number for assignment submission. Technology. Around this time the Web and browsers such as Netscape were becoming prevalent. The technology was becoming more user friendly. In the first half of 1996 76% of students in the Systems Administration course had Internet access from home before the course started. At this time the first Word viruses were becoming popular and forced a move to using RTF files rather than Word documents. Early

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

versions of the Internet Explorer Web browser caused problems since it did not support the file upload standard. As a result students using Internet Explorer had to revert to manual email submission. Course. OASM was still being used in the same two courses. Both courses now had large accompanying web sites and OASM was made compulsory for both courses. Individuals. Staff and students are becoming more familiar with the Web and related technology. The marker now had two or three semesters experience with online marking. By 1997 all students had access to the Internet from home or work. Students are starting to expect 24/7 availability of Websites. In the second half of 1996 the coordinator had time relief to develop web-based tools and systems to support other staff. Organizational. The importance of Internet technologies to the operations of the University was starting to be realized. Along with time relief for the originator of OASM a departmental web server is purchased. However, no long term funding is provided for maintenance that made maintaining a 24/7 operation required significant unpaid effort. A partnership with a commercial company leads to the addition of new campuses and growing student numbers. The academic structure of the organization is restructured. Society. Many small Internet Service Providers (ISPs) are becoming available. Infrastructure. A new faculty server is available however uptime is a problem. There is growing problems with the number of University provided dial-up modems available for students. Students slowly start moving towards using ISPs.

4.4. Integrated Web (1997-2001)


Experience. The new and compulsory nature of OASM caused some initial resentment amongst oncampus students who did not see the benefits of OASM. As the technology became easier to use this resentment decreased. Student use of OASM peaked at over 100 submissions for a single assignment in early 1997. Even with the web submission process 25% of students still submitted the wrong file types. Turnaround time was reduced to 8 days for all assignments using OASM. One assignment had an average turnaround time of 2 days. With improved technology and processes marking online assignments was found to be 20-30% faster than paperbased assignments [3]. Distance students were extremely positive about the fast turnaround time with some disquiet amongst on-campus students. One student was concerned about privacy issues due to the absence of a username/password system. New System. The next evolution of the OASM system was to integrate it with CQU's student record system.

This integration enabled the system to know which courses a student was enrolled in and also to make use of a global username/password available to all students. Courses. During this time staff teaching other courses started becoming interested in using OASM. In the first half of 2000 over 770 students in 6 different courses used OASM. Individuals. Other academic staff became involved for the first time. These new staff and their markers did not have experience with OASM. During this time the creator of the OASM system spent time on a number of other projects so during this four year period little work was done on the OASM system beyond its initial integration with the CQU student records system. The marker involved in the 1996-1997 OASM system joined CQU's central Information Technology division working on CQU's student record system. Organizational. Primarily due to the past connections outlined in the previous paragraph a good relationship existed between the academic staff and CQU's central Information Technology division. This relationship made it much easier to negotiate access to their databases. Gaining permission from the data owner was more problematic and after many formal approaches was achieved by interrupting the head of that division at lunch. Infocom employed a Webmaster and expended more money on a larger Web server. All Infocom courses now had a primitive, generic website. Numbers at international campuses grew until they eventually provide more than 50% of all students in courses. Consequently there are more staff, both teaching and markers, involved with courses. Society. Larger, nationwide companies were beginning to replace smaller, local Internet Service Providers (ISPs). Local call access to the Internet becomes available to all parts of Australia. The Internet and World-Wide Web became widely popular. Australian federal and state governments were under pressure to better support regional Australia. Technology. By 2001 a student without Internet access in computing courses becomes the exception. The Web has arrived has a major tool for students and the ease-of-use of the available tools had dramatically increased. Infrastructure. During this time University infrastructure did not keep up with the growth in popularity. Problems were caused by too few dialup modems for student use, limited bandwidth, too few support staff, and under-resourced servers.

4.5. Evolution (2001-now)


Experience. 2000 and early 2001 saw a growing number of staff and courses using OASM without an

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

appropriate increase in supporting infrastructure. This caused considerable difficulties and contributed to a negative experience for many students. The new staff members limited experience with OASM combined with the infrastructure problems meant that average turnaround times for OASM assignments were usually somewhat more than the average turnaround time for the traditional method. New System. Significant money was spent by CQU to improve infrastructure including both bandwidth and servers. The major changes to OASM were the addition of extra services such as copy detection, reporting, and support for multiple markers. Organizational. A support team of four people has been set up within Infocom to support the use of OASM and other online technologies. This is a move initiated by the Faculty Dean and this alone enables further development of the system. One head of school asks for a presentation about OASM to be made to her staff. This encourages more staff to adopt OASM. This increase is mainly due to the growing problems with the multicampus operations of CQU and the administrative headaches it causes for teaching staff. CQU adopts a new student records system that requires significant changes to the OASM system. The requirement that staff at international campuses mark their student's assignments is written into contracts and thus must be supported by OASM. The new server equipment is shared amongst three different parts of CQU and requires some work for this relationship to operate appropriately. Courses. By 2002 OASM is being used in 18 different courses offerings ranging from 1st year courses in computer programming and web development, through 2nd year Multimedia Design and Human Computer Interface courses, to post-graduate courses in database design. Individuals. Greater than 50% of the students in most Infocom courses have English has a second or third language. The majority of the courses using OASM are first year courses. These factors combined require a continual effort to improve the interface and operation of OASM. Staff concerns about plagiarism grow with students found to be adopting practices such as selling past assignment solutions on E-Bay. Each staff member that adopts OASM brings with them their own vision of how assignment management, marking and return should be performed. Infrastructure. The large University server is more than sufficient for current requirements. Bandwidth to some of the campuses is limited and cause problems. Technology. Access to copy detection software is available during 2002. The growing use of OLE objects and images in student assignments lead to students submitting very large RTF files. In mid-2002 virus detection software is made available on the main server

for the first time. This enables the replacement of RTF files with Word documents. Society. The Australian government strengthens reporting requirements for full fee paying overseas students (those studying at CQU's international campuses) to include a requirement that the University must be able to identify students who have not submitted assessment. This requires the addition of a reporting mechanism for administrative staff. The government also embarks on a push to increase quality assurance systems within Universities. Due to the governmental auditing requirements there this has sparked an increased interest in OASM due to the automated tracking of assignments at all stages.

4.6. The future


Experience. With increasing numbers, in terms of students, courses, and markers, using OASM the first half of 2002 had numerous problems. The average turnaround time for all assignments submitted via OASM in the first half of 2002 was 29 days. As 2002 progressed both students and staff (teaching and marking) have become more familiar with OASM and better understand the medium. The average turnaround time for assignments so far in the second half of 2002 is down to 18 days. Staff members are increasingly making use of copy detection and are commenting on the reduction in the level of student plagiarism. OASM is becoming a standard part of the organization with better support and wider acceptance. From the start of 2002 to late September over 3500 different students in 18 different course offerings have made use of OASM. Students see OASM as one of the most popular of all online learning innovations at CQU. New System. Immediate development plans for OASM include increasing the integration with various other services, the provision of additional management facilities for staff, and improvements to the interface and flexibility of the system. All of these changes are being driven by the different ideas and suggestions generated by the variety of staff using OASM.

5. Discussion
A wide range of factors, as described by Figure 1, has influenced the development of OASM within Infocom. The ability for Infocom's OASM system to evolve in response to these factors has resulted in a system that better serves the needs of students, staff, and the organization. Such evolutionary development has only been possible through the presence of appropriately skilled individuals supported through sufficient resources, infrastructure, organizational policies, and a flexible technology.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

This raises questions about the suitability of integrated course management systems such as WebCT and Blackboard. Appropriately resourced in-house products are able to provide more functionality than these integrated systems [1] and be modified in response to unique local conditions. Since no company will adequately address all the possibilities for online assessment it is critical that such a system have open system architectures that support interoperable modules [12]. Course management system vendors are recognizing this fact by adopting more open architectures. The need for evolutionary development of OASM is not limited to the technology and its features. Experience with OASM has shown that initially, faculty attempt to recreate existing practice with the new technology. This is often because changing both practice and medium is seen as being too difficult. However, as experience grows staff begin to question basic assumptions and start to look at using the unique characteristics of OASM and alternate assessment approaches. If OASM systems accommodate only classical approaches they will constrain instructors and not allow the development of more innovative approaches [12]. A system that only accommodates "innovative" approaches will not allow staff to adapt gradually [12].

[3]

[4] [5] [6]

[7]

[8]

[9] [10]

6. Conclusions
Drawing on experience with online assignment submission and management (OASM) since 1994 this paper has developed a model that encapsulates seven factors that have contributed to the evolutionary development of OASM. It has used that model to show how OASM within an individual organization has changed from submission via email in a single course by 20 students to an integrated, web management facility incorporating support for copy detection and distributed markers used in a single calendar year in 18 different course offerings by 3500 different students.
[11]

[12] [13]

[14] [15]

7. References
List and number all bibliographical references in 9point Times, single-spaced, at the end of your paper. When referenced in the text, enclose the citation number in square brackets, for example [1]. Where appropriate, include the name(s) of editors of referenced books.
[1] P. Darbyshire, Distributed Web-Based Assignment Management, Web-Based Learning and Teaching Technologies: Opportunities and Challenges, A. Aggarwal, Ed. Hershy, PA: Idea Group Publishing, 2000. A. Tregobov, The Web-Based Assignment Submission Systems, presented at NAWeb'98, University of New Brunswick, Canada, 1998.

[16]

[17]

[2]

[18]

D. Jones, B. Jamieson, Three Generations of Online Assignment Management, Proceedings of ASCILITE'97, R. Kevill, R. Oliver, R. Phillips, Eds. Perth, Australia: pp 317-323, 1997. D. Jones, Solving some problems of University Education: A Case Study, Proceedings of AusWeb96, R. Debreceny, A. Ellis, Ed. Gold Coast, pp 243-252, 1996. D. Jones, Solving some problems with University Education: Part II, Proceedings of Ausweb'99, A. Ellis, Ed. Balina, Australia, 1999. R. Byrnes, B. Lo, J. Dimbleby, Flexible Assignment Submission in Distance Learning, presented at the IFIP World Conference on Computers in Education VI, WCCE'95, 1995. M. Chi-sang, Y. Kin Sun, C. Siu Leung, L. Chow, The Electronic Learning Environment for IT-based course in Hong Kong, Proceedings of EdMedia'99, R. Oliver, B. Collis Eds., AACE, Seattle, USA, June, 1999, p13841385. G. Burnett, K. Burnett, Integrated Online Course Delivery System: Florida State University School of Information Studies, Proceedings of EdMedia'99, R. Oliver, B. Collis Eds., AACE, Seattle, USA, June, 1999, pp 1511-1512. R. Laws, Totally Integrated Internet Courses, Proceedings of EdMedia'99, R. Oliver, B. Collis Eds., AACE, Seattle, USA, June, 1999, pp 787-791 S. Young, M. McSporran, R. Dewstow (2000), Suffering Remotely: Challenges when teaching on-line, Proceedings of EdMedia'2000, J. Bourdeau, R. Heller Eds., pp 1590-1591 J. Palmer, R. Williams, H. Dreher, Automated Essay Grading Applied to a First Year University Subject - How can we do it better?, Proceedings of Informing Science, InSITE - "Where Parallels Intersect", June 2002 M. Hopper, Assessment in WWW-Based Learning Systems: Opportunities and Challenges, Journal of Universal Computer Science, 4(4), 1998, pp 330-348 P. Davies, S. Hansen, G. Salter, K. Simpson, Online assessment with large classes: issues, methodologies and case studies, Proceedings of WebNet99, AACE, Honolulu, USA, 1999, pp 1498-1499. J. English, P. Siviter, Experience with an Automatically Assessed Course, Proceedings of ITiCSE'2000, ACM, Helsinki, Finland, 2000, pp 168-171. D. Mason, D. Woit, Providing mark-up and feedback to students with online marking, Proceedings of the 30th SIGCSE Technical Symposium on Computer Science Education, ACM, 2000, pp: 3-6. B. Oliver, G. Mitchell, Setting the PASE - The Value of Computer Aided Assessment, Proceedings of the First Australasian Conference on Computer Science Education, ACM, Sydney, Australia, 1996, pp 103-110. B. Price, M. Petre, Teaching Programming through Paperless Assignments: an empirical evaluation of instructor feedback, SIGCSE Bulletin, 29(3), September 1997, pp 94-99. P. Summons, J. Coldwell, C. Bruff, F. Henskens, Automating Assessment and Marking of Spreadsheet Concepts, Proceedings of ACSE'97, Melbourne, Australia, 1997, pp 178-184.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

[19] J. Preston, R. Shackleford, Improving On-line Assessment: an Investigation of Existing Marking Methodologies, Proceedings of ITiCSE'99, Cracow, Poland, 1999, pp 29-32 [20] J. Preston, Evaluation software: improving consistency and reliability of performance rating, Proceedings of ITiCSE'97, ACM, Uppsala, Sweeden, 1997. [21] D. Jones, C. McCormack, Class Management: The Forgotten Task, Proceedings of the Third International North American Web Conference, University of New Brunswick, Canada, 1997, pp 109-125. [22] D. Jonassen, Designing Constructivist Learning Environments. Instructional Theories and Models. C. M. Reigeluth. Mahwah, NJ, Lawrence Erlbaum, 1998. [23] R. Kling, J. Allen, Can Computer Science Solve Organizational Problems? The Case for Organizational Informatics, in Computerization and Controversy, Rob Kling (editor),, Academic Press, San Diego, pp 261-276 [24] T. Reeves, A Research Agenda for Interactive Learning in the New Millennium. Proceedings of EdMedia'99, R. Oliver, B. Collis Eds., AACE, Seattle, USA, June, 1999. [25] D. Quan, Universities combat rising Internet plagiarism, The Ottawa Citizen, 16 Feb. 2001, final edition. [26] L. Sproull, S. Kiesler, Connections: new ways of working in the networked organization. Cambridge, MIT Press, 1991. [27] J. Young, The Cat-and-Mouse Game of Plagiarism Detection, The Chronicle of Higher Education, July 6, 2001, A26. [28] M. Roantree, T. Keyes, Automated collection of coursework using the Web, Proceedings of ITiCSE'98, Dublin, Ireland, Pages 206-208 [29] M. Joy and M. Luck, Effective electronic marking for on-line assessment, Proceedings of ITiCSE'98, Dublin, Ireland, Pages 134-138 [30] D. Jackson, A Semi-Automated Approach to Online Assessment, Proceedings of ITiCSE'2000, Helsinki Finland, 2000, pp 164-167.

Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS03)

0-7695-1874-5/03 $17.00 2002 IEEE

You might also like