Intl. Inform. & Libr. Rev. (2002), 34, 247^254 doi:10.1006/iilr.2002.0205 Available online at http://www.idealibrary.

com on

Website Quality Evaluations: Criteria and T ools

NICOLAE-GEORGE DRAGULANESCU*

A B ST R AC T The quantity of information distributed today on the Internet, especially through the World Wide Web, is rising much more quickly than the quantity of information available through other mass media. Just as we do not like or believe everything we read in the newspapers, see on TV, or hear on the radio, we cannot like or believe everything on the Internet. Consequently, it is necessary to critically assess the quality of websites and to determine the extent to which they meet our needs and requirements. This paper is an attempt to analyze information evaluation concept and process-based on some quality management concepts, rools and methods in order to propose some basic criteria to evaluate website quality and to select suitable websites.
r 2002 Elsevier Science Ltd. All rights reserved.

S IGN I FIC A NC E

AND

S C OP E

OF A N

E VA LUATION P RO C E S S

Evaluation is a complex and critical process. Every human being uses evaluation as a judgmental process involving such concepts as quality, truthfulness, and accuracy. Evaluating, for example, an information product, a service, a person, or an organization should not be a burdensome task. University professors and students should be the most eective information evaluators, and if not, they have to be trained to do so. Evaluative judgments are frequently followed by important deliberative and decision-making processes. Examples include: 1. Judging the merits of an argument, a person, a product or service, or an organization.
*Fulbright Visiting Scholar at University of Pittsburgh, USA, Quality Auditor of European Organization for Quality, Assessor of European Foundation for Quality Management, University of Polytechnics, Bucharest, Str. Obcina Mare 1/15, 77.346 Bucharest 66, Romania. E-mail: nicut@pcnet.ro; nicudrag@artelecom.net

1057^2317 /02/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.

248

N.-G. DRAGULANESCU

2. Deciding what action to take (for instance, to buy or not to buy a good/product or a service provided by a certain company). 3. Establishing the credibility of a position or course of action. 4. Deciding to defer the decision to act or not to act in order to obtain additional information. There are in English at least six dierent verbs having the same basic meaning of to form a judgment of worth or signicance, and also specic additional meanings that must be considered.
K

K K

to evaluate implies considered judgment in setting a value on a person or thing; to rate involves determining the rank of a person or thing when she/he or it is judged in relation to others of the same kind; to estimate may imply judgment based on rather rough calculations (but this term lacks the denitiveness of other terms); to appraise stresses expert judgments; to assess implies authoritative judgments (such as setting a monetary value on something as a basis for taxation); and to assay refers to a careful examination (such as chemical analysis of ore to determine its content).

These last three verbs can refer, in extended senses, to a critical analysis or appraisal. Bloom1 in his hierarchical model of educational objectives, includes evaluation as a basic dimension of the thinking/cognitive process. According to this model, the hierarchy of thinking skills peaks with evaluation as the most sophisticated of cognitive processes (including those of understanding, appreciation, analysis, and synthesis). Debons2 refers to these in his basic DIK-needs concept, i.e., Data^Information^ Knowledge needs. The critical dierence between evaluation and all the other abovelisted skills is the inclusion of criteria and values. Evaluation is dened as the making of judgments about the value-for [sic] some purpose-of [sic] ideas, works, solutions, methods, material, etc. It involves the use of criteria as well as standards for appraising the extent to which particulars are accurate, eective, economical, or satisfying. The judgments may be either those determined by the evaluator or those which are given to him .
Bloom, S.B. et al. (1956) T axonomy of educational objectives: the classication of educational goals. New Y ork: David McKay. 2 Debons, A. (1988) Information science. An integrated view. Boston: G.K. Hall&Co.
1

WEBSITE QUALITY EVALUATIONS

249

Studies in cognitive psychology, metacognition, and critical thinking have established the possibility that evaluative thinking may be inuenced by several factors (listed in order of their importance):3 (I) prior knowledge (which has the strongest inuence); (II) format preferences (for example, scholarly books and journal articles are much more important for researchers than popular press books, magazines, and web resources); (III) epistemology (which investigates the nature and the origin of knowledge); (IV) aect (which is most common during initialization of the evaluation process); and (V) beliefs (epistemological, methodological, etc.). All these inuences may impact or change the course of evaluative processes and their outcomes, sometimes in rather dramatic ways. Q UA LI T Y OF I N F OR M AT ION Both notions of information and quality have become broader and broader over the years and generated a lot of questionable denitions. According to Dragulanescu,4 the information conceptFas it was employed in dierent disciplines, during many years, for designing a state, a product, or a processFhad a heteroclite, ambiguous, polyvalent, and unclear character, despite its considerable heuristic value. The word information, as used today, reects dierent perspectives. This includes such terms as commodity, energy, communication, facts, data, knowledge, etc. If it is used as a commodity, information (i.e., that which is included in texts, sounds, images, etc.) constitutes an economic value. Consequently, information can be sold and purchased, as an information product or information service . According to Debons,5 if an individual or organization has sole possession of a particular body of information, that information may enable whoever holds it to achieve objectives. Information can thus provide control over objects and persons! The next point is how the quality of these products and services can be properly measured. T oday, the international standard ISO 9000
3 Fitzgerald, M.A. (2000) The cognitive process of information evaluation in doctoral students: A collective case study. Journal of Education for Library and Information Science 41(3) (Summer 2000, 170^186). 4 Dragulanescu, N. (1999) Information science and technology. Genesis and evolution (in French). Bucharest/Romania: AGIR Publishing House. 5 Debons, A. (1988) Op. cit.

250

N.-G. DRAGULANESCU

denes quality as being the totality of characteristics of an entity (a general notion including either a product/service, or a process, an activity, a system, an organization, a person or a combination thereof) that bear on its ability to satisfy stated and implied needs (of customers and other stakeholders). This internationally accepted denition refers both to the customer/ user (of the product/service) and to the provider of the product/service. Indeed, quality concerns the product/service to the degree that it complies with the specications to the adequacy of its usage, as well as to the number of attributes aiming for excellence at a competitive price (because the customer/user is purchasing exclusively products/services having the highest quality/price ratio). Quality also represents a new strategic philosophy of company management, based on the overall commitment of the management and employees towards continuous improvement. This commitment has an ultimate objective: the customers satisfaction in all the phases of a products life cycle and in all the sectors of the company. A product is the result of a series of activities which denes its processes.6 Consequently, the information as a commodity could be considered to be an immaterial product, realized and sold by a supplying company, purchased and resold by one or more providers, and used by its end user This immaterial product is the so-called information . product and should not be confused with the material product accompanying it (i.e., the material carrier of information: the paper of a book or a journal, an object, a signal, etc.). The quality of both these immaterial and material products has to be evaluated within the information market by both the user and the provider, using specic criteria and tools. The model of the External Provider^Customer/User Relationship could be very useful for someone interested in understanding how to dene and improve the quality of both immaterial and material products. Based on this model, Dragulanescu7 concluded that an information product (texts, sounds, images) and its material information carrier (e.g., the paper of a book or journal, the audio-cassette or the videocassette, etc.) are considered to be a quality information product solely if the provider has taken into consideration both the stated and implied needs of the customers/users (before designing, manufacturing, and supplying the product) as well as their reactions (after having supplied the product) In other words, an information product/carrier is considered .
Dragulanescu, N. & Niculescu, C. (2000) Quality management (in Romanian). Bucharest/ Romania: Niculescu Publishing House. 7 Dragulanescu, N. (1999) Op. cit.
6

WEBSITE QUALITY EVALUATIONS

251

1) eeds/requirementsRRR

1) NEEDS / REQUIREMENTS

INFORMATION PROVIDER

2) MATERIAL PRODUCT + IMMATERIAL PRODUCT

3) REACTIONS

INFORMATION CUSTOMER / USER

FIGURE 1. The External Provider^Customer/User Relationship. (Figures 1, 2 and 3 are representing the object sequence of actions to be implemented in order to obtain/ improve the quality of both the material and immaterial products components of an information product.)

to be a quality information product/carrier only if it meets the stated and implied needs of customers/users (Fig. 1). Human beings, according to Debons,8 have fundamentally only one basic information need and multiple knowledge needs. The information need is to be aware of and how the capacity to respond to questions of what?, where?, when? and who? The knowledge needs, on the other hand, are to understand, to apply, to analyze, to synthesize, and to evaluate. Satisfying these needs relates to the capacity to respond to questions of why and how? Another classication of very basic needs of human beings could be:9 (a) the need to know (for example, who are they, where are they coming from, when, where, how, and why are certain phenomena and events happening, etc.); and (b) the need to act (in order to improve continuously their living conditions). According to a Romanian saying, knowing without acting could be useless, but acting without knowing could be dangerous! Information suppliers/providers and information customers/users are establishing partnerships whose informational interactions have eciency and eectiveness depending both on the partners capabilities and on the partnership interaction context.10 Q UA LI T Y ON W E B SI T E S The World Wide Web is a very complex information technology network currently consisting of several hundred million web pages and over a
8 9

Debons, A. (1988) Op. cit. Dragulanescu, N. (1999) Op. cit. 10 Dragulanescu, N. (1999) Op. cit.

252

N.-G. DRAGULANESCU

hundred million users. Each day, users search websites in order to nd the most convenient, relevant, and up-to-date information they need. On the web, users typically forage for information by navigating from page to page, along web links. The content of pages associated with these links is usually presented to the user by some snippets of text or graphic. Based on the above-mentioned, quality management concepts and models, I propose to identify values and criteria enabling a website surfer/user to evaluate and select websites and pages. T oday it is possible to nd on web pages, for example:
K K

K K

marketing information on products, services, organizations, etc.; texts, sounds, and/or images of mass-media productions (including electronic documents and journals); personal web pages; and, as e-mail messages, newsgroup postings, etc.

The layout of a web page has three main components: the header, the body, and the footer. By thoroughly investigating these components, it is possible to nd answers to questions like:
K K K K K K

Who is the sites author (or contact person)? When was the site created/reviewed? Who sponsors the site/page? What are the links provided? Who is the intended audience of the site/page What is the purpose of the sites information?

T oday, almost anyone can publish a website and, unfortunately, there are so far no formal standards on minimal criteria for so doing. Some websites use an e-mail link to a webmaster in order to facilitate the interactivity, i.e., the communication of the user with the provider in order to allow the continuous improvement of site quality, based on reactions collected in this way from dierent users. Further analysis could imply each criterion seen as a result which has some specic enablers, to identify, dene and evaluate. After careful evaluation of all these results and enablers, an eective evaluator is able to make comparative judgments related to the quality of dierent websites and their information providers. An ideal website may be impossible, but surely an attempt to satisfy basic user needs is feasible. T do so is in the spirit of total quality mano agement (TQM) and this attempt could be continuously made easier and faster by applying quality management specic tools, methods, and techniques (Table I).11
11

Dragulanescu, N. & Niculescu, C. (2000) Op. cit.

WEBSITE QUALITY EVALUATIONS

253

TABLE I
EVALUATION CRITERIA 1. ACCURACY (extent/degree of information exactness and correctness) QUESTIONS TO BE ANSWERED BY USER F Does the author mention his information sources? F Is it possible for you to check if these sources are legitimate? F Is the authors background related to the covered topics? F Did the author indicate clearly the research method and how he collected and processed data (only for research-based data)? F Is the author known? F What do you know about the author (as for example: his background, position, aliation, publications, etc)? F Is it possible, for you, to determine the authors expertise and credibility? F Are all sites topics explored in depth? F Are all sites links relevant, appropriate, comprehensive and operational? F How relevant and comprehensive is, for you, the sites information? F F F F When was created and reviewed (last time) the site? Is the copyright date displayed? Are all mentioned resources available? Are all of sites links relating to correct INTERNET addresses?

2. AUTHORITY (extent/degree to which the author could be considered an expert in his eld). 3. COVERAGE (extent/degree to which topics was observed, analyzed and reported) 4. CURRENTNESS (extent/degree to which the distributed information is belonging to the time now in progress) 5. DENSITY (extent/degree to which comprehensive and relevant information is displayed on each sites page) 6. INTERACTIVITY (extent/degree to which is operating the bidirectional communication between user and author)

F Is text or graphic content predominant on each page? F Is enough comprehensive, for you, text/graphic information displayed on each page? F Is enough useful, for you, the displayed text/graphic? F How much advertising information is included on sites pages? F Is it possible, for you, to nd at least one e-mail link to Web site author or to the webmaster? F Is this link active? F Is this link quickly operational? F Did you receive an answer to the message you sent to author/webmaster?

254

N.-G. DRAGULANESCU

TABLE I F Continued
7. OBJECTIVITY (extent/degree of sites author objectivity versus his subjectivity) 8. PROMPTNESS (time delay needed for site nding and its pages displaying ) F What is the real goal of the site? F What is the real purpose of the sites author? F Are you enough condent in authors objectivity? F Was the site URL found enough quickly? F Is it possible to change immediately displayed pages? F Is it possible, for you, to contact, within a reasonable delay, all provided links?

W E B SIT E S S E A RC H OR L I B R A RY P U BL IC AT ION S S E A RC H ? T oday, searching for information for a project on websites seems to be a lot more ecient and eective than traditional, library-based methods. But, is this expectation realistic? In my opinion, web and library research are complementary. The web can be a good source of quick overviews but the library is necessary for in-depth analysis. Finding in a library the information we need for a project could determine, in itself, the success of our project. Such a search can be exhausting and envigorating, yet also produce unexpected, serendipitous results that may be useful in re-evaluating the researchers assumptions or may even be useful for the researchers future projects. In poring over vast quantities of information, we are more likely to nd alternative points of view and are also more likely to experience a more reective and critical approach to the research than on the web, with its veneer of instant expertise and instant credibility. In contrast, searching for information on websites, for the same above-mentioned project, is to ask the various search engines, metacrawlers, and directories to nd specic pieces of information that support our assumptions. Accidental discoveries do happen, but they are more rare on the web. On the other hand, the web is great for reading abstracts and getting a snapshot of a subject. C ONC LUSION We, as professors, want students to learn to nd information on all sides of an issue and to recognize complexities and nuances. We want students to learn a vast variety of subjects, but we also need them to master at least one. The challenge for us, as educators, is to integrate old and new ways of learning. However, in order to be able to act in this manner, we should learn, rst of all, how to evaluate critically website information quality.