Higher-order thinking essentially means thinking that takes place in the higher-levels of the hierarchy of cognitive processing.

Blooms Taxonomy is the most widely accepted hierarchical arrangement of this sort in education and it can be viewed as a continuum of thinking skills starting with knowledge-level thinking and moving eventually to evaluation-level of thinking. A common example, used by Dr. Chuck Weiderhold of the application of the major categories in Blooms Taxonomy, is show below, applying the taxonomy to the Pledge of Allegiance: Knowledge statements ask the student to recite the pledge. Example: Say the pledge. Comprehension statements ask the student to explain the meaning of words contained in the pledge. Example: Explain what indivisible, liberty, and justice mean. Application statements ask the student to apply understandings. Example: Create your own pledge to something you believe in. Analysis statements ask the student to interpret word meanings in relation to context. Example: Discuss the meaning of and to the Republic for which it stands in terms of its importance to the pledge. Synthesis statements ask the student to apply concepts in a new setting. Example: Write a contract between yourself and a friend that includes an allegiance to a symbol that stands for something you both believe in. Evaluation statements ask the student to judge the relative merits of the content and concepts contained in the subject. Example: Describe the purpose of the pledge and assess how well it achieves that purpose. Suggest improvements. (Wiederhold, C. (1997). The Q-Matrix/Cooperative Learning & Higher-Level Thinking. San Clemente, CA: Kagan Cooperative Learning.) When we promote higher-order thinking then, we are simply promoting thinking, along with the teaching methodologies that promote such thinking, that takes place at the higher levels of the hierarchy just provided, notably application, analysis, synthesis, and evaluation. Critical/creative/constructive thinking is closely related to higher-order thinking; they are actually inseparable. Critical/creative/constructive thinking simply means thinking processes that progress upward in the given direction. First one critically analyzes the knowledge, information, or situation. Then they creatively consider possible next-step options, and then finally, they construct a new product, decision, direction, or value. The evaluation step listed above with the Pledge of Allegiance would require this sort of thinking.

Reading Beyond the Lines Another way to look at higher-order thinking is to look at the reading process in typical terms and then extend the terms one step to reach higher-order thinking. That is, being able to read, being literate, typically means having the ability to decode words and understand their meanings individually and collectively. Being able to read and to

comprehend the reading is generally considered thinking and involves reading the lines and reading between the lines. Higher-order thinking or literacy though, is the next crucial step, often not even thought of in the reading process, that being reading beyond the lines. This is so crucial because it is in reading beyond the lines that reading the lines and reading between the lines have their real value.

Instructional Elements for Fostering Higher-Order Thinking in the Classroom (Synthesized from Teaching Children to Be Literate: A Reflective Approach, by Anthony and Ula Manzo, 1995) 1. Remember to ask for it; that is, for discovery, invention, and artistic/literary creation. 2. Great curiosity and new ideas with enthusiasm; these can often lead to the most valuable teachable moments. 3. Expose learners to new twists on old patterns and invite looking at old patterns from new angles. 4. Constructively critique new ideas because they almost always require some finetuning. 5. Reset our expectations to the fact that there will be many more misses than hits when reaching for workable new ideas. 6. Learn to invite contrary, or opposing, positions; new possibilities are often discovered in this way and existing thoughts, patterns, and beliefs can be tested and strengthened.

Questions that Invite Higher-Order Thinking (Synthesized from Teaching Children to Be Literate: A Reflective Approach, by Anthony and Ula Manzo, 1995) How is this study like another you/we have read? This question encourages students to make connections and see analogies. Does this story/information make you aware of any problems that need attention? This amounts to asking students to see themselves as active participants in problem identification as well as problem solving. What does this mean to you and how might it affect others? This pair of questions gives students a chance to express their own interests but also to empathetically consider and understand the views of, and possible consequences to, others. Is there anything wrong with this solution, and how else might this problem be solved? These questions are the heart of successful critical analysis.

 What more needs to be known or done to understand or do this better? This is a pointed request for creative problem solving that invites thinking beyond the lines. What is a contrary way of seeing this? Being able to examine issues from multiple points of view helps the students to clarify their thoughts.

Questioning for Quality Thinking at Each Level of Blooms Taxonomy Knowledge: Identification and recall of information Who, what, when, where, how? Describe ___________________. Comprehension: Organization and selection of facts and ideas Retell ___________ in your own words. What is the main idea of ___________________? Application: Use of facts, rules, principles How is __________ and example of _______________? How is __________ related to _________________? Why is _________________ significant? Analysis: Separation of the whole into component parts What are the parts or features of ________________? Classify _______________ according to ________________. Outline/diagram/web ____________________. How does ______________ compare/contrast with __________________? What evidence can you list for _____________________? Synthesis: Combination of ideas to form a new whole What would you predict/infer from __________________? What ideas can you add to __________________? How would you create/design a new __________________? What might happen if you combine _______________ with ________________?

What solutions would you suggest for __________________? Evaluation: Development of opinions, judgments, or decisions Do you agree with _________________? What do you think about _______________? What is the most important _____________? Prioritize ________________. How would you decide about ________________? What criteria would you use to assess ______________________?

Head-on Approaches to Teaching Higher-Order Thinking (Synthesized from Teaching Children to Be Literate: A Reflective Approach, by Anthony and Ula Manzo, 1995)

Thinking Thursdays o Consider setting aside a given amount of time on a regular basis to try some of these direct approaches to teaching critical and creative thinking. Word Creation: o Define the word squallizmotex and explain how your definition fits the word. o If dried grapes are called raisins, and dried beef is called beef jerky, what would you call these items if they were dried: lemons, pineapple, watermelon, chicken. Unusual Uses: o Have students try to think of as many unusual uses as they can for common objects such as bricks, used toys, old tennis balls, soda bottles, and 8-track cassette tapes. Circumstances and Consequences: What would happen if . . . o school was on weekends and not during the week? o water stuck like glue? o gravity took a day off? o there were no colors? o everyone in the country could vote on every issue that is now decided by government representatives? Product Improvements: o How could school desks be improved? o How could living room furniture be improved to provide better storage and even exercise while watching television?

How can we better equip book-carrying bags to handle lunches and other needs that you can think of?

Systems and Social Improvements: o A sample question that could lead into plenty of higher-level discussion and a good give-and-take of views and needs could be: How can schools be made more fun without hurting learning?

Higher-Order Thinking & REAP Read-Encode-Annotate-Ponder (REAP) is a teaching method developed by M.G. Eanet & A.V. Manzo at University of Missouri- Kansas City. It is a strategy developed for students to use to improve writing, thinking, and reading. As a teaching method, it is intended to teach students a variety of ways to respond to any text. The responses are brief and poignant ways to critique or annotate what they have read. There are different types of annotations which range from simple summary (reconstructive) to highly challenging critical-creative responses (constructive). Value of Annotating In writing annotations the readers discriminate and synthesize ideas presented by the author, then translate it into their own language. Writing and annotations enrich reflective thinking and reading. The readers analyze the author's purpose and explore their own feelings about the written material. Students who write about what they have learned gain from the reading process. Consequently, writing should be an integral part (a vital component) in the classroom setting. Writing serves as a catalyst in improving one's reading, thinking and comprehension abilities. Learning the routine to write after reading ignites ACTIVE THINKING before, during and after a reading selection. Annotations ensure meaningful reading and encourage clear and concise thinking and writing. Annotations enhance reader's knowledge base as well as improve thinking and writing skills. Steps in REAP: R: Read to discern the writer's message. E: Encode the message by translating it into your own words. A: Annotate by cogently writing the message in notes for yourself, or in a thought book or on an electronic response system. P: Ponder, or further reflect on what you have read and written, through discussion and by reviewing others' response to the same materials and/or your own annotation. Using REAP as a Rubric for Monitoring Progress Toward Higher-Order Thinking REAP may be used as a way to monitor a student's progress toward higher-order thinking. By using examples of the various types of annotations, a teacher may compare and appraise the characteristic way in which the student responds to text. The annotation types listed above are roughly in order of difficulty. Lower numbers indicate more

concrete thinking (or literalness) and higher numbers more personal and abstract patterns of response. Annotation Types Reconstructive... requires literal-level response to a text. Constructive... requires reading and thinking between and beyond the lines. -----------------------------------------------------------------------Reconstructive Responses 1. Summary response. States the basic message of the selection in brief form. In fiction, it is the basic story line; in nonfiction, it is a simple statement of the main ideas. 2. Precise response. Briefly states the author's basic idea or theme, with all unnecessary words removed. The result is a crisp, telegram like message. 3. Attention-getting or heuristic response. Restates a snappy portion of the selection that makes the reader want to respond. It is best to use the author's own words. 4. Question response. Turns the main point of the story or information into an organizing question that the selection answers. Constructive Responses 5. Personal view or transactional response. Answers the question "How do your views and feelings compare with what you perceive the author to have said?" 6. Critical response. Supports, reject s, or questions the main idea, and tells why. The first sentence of this type of response should restate the author's position. The next sentence should state the writer's position. Additional sentences should explain how the two differ. 7. Contrary response. Attempts to state a logical alternative position, even if it is not one that the student necessarily supports. 8. Intention response. States and briefly explains what the responder thinks is the author's intention, plan, and purpose in writing the selection. This is a special version of the critical response that causes the reader/responder to try to think like the author or from the author's perspective. 9. Motivation response. States what may have caused the author to create or write the story or selection. This is another special version of critical responding. It is an attempt to discover the author's personal agenda and hence areas of writing or unwitting biases. 10. Discovery response. States one or more practical questions that need to be answered before the story or facts can be judged for accuracy or worth. This type of response to text is the mode of thinking that leads to more reading and research and occasionally to a reformulated position or view.

11. Creative response. Suggests different and perhaps better solutions or views and/or connections and applications to prior learning and experiences. Students usually need some guidance and/or examples to produce this type of response. Once they begin thinking in this way, the results can be remarkably constructive.

Writing to Promote Higher-Order Thinking (Synthesized from Teaching Children to Be Literate: A Reflective Approach, by Anthony and Ula Manzo, 1995) Advantages

Writing activates the readers background knowledge before reading/thinking. Writing builds anticipation of upcoming learning events. Writing raises the readers level of intellectual activity. Writing encourages meaningful comparisons of the students perspective with that of the writer (in reading situations) Writing helps students better formulate their world view. Writing allows students to examine their perspectives on key issues. Writing builds metacognitive as well as cognitive abilities because writing forces deeper levels of introspection, analysis, and synthesis than any other mediational process.

Suggestions Related to Using Writing to Promote Higher-Order Thinking

Write daily or frequently rather than sporadically. Write for real audiences and purposes. Allot sufficient time for stages of thought and editing to occur. Encourage peer review Write with an initial emphasis on thinking rather than on proofreading and editing.

Contributed by Barbara Fowler, Longview Community College. Bloom's Revised Taxonomy divides the way people learn into three domains. One of these is the cognitive domain which emphasizes intellectual outcomes. This domain is further divided into categories or levels. The key words used and the type of questions asked may aid in the establishment and encouragement of critical thinking, especially in the higher levels.

Level 1: Remembering - exhibits previously learned material by recalling facts, terms, basic concepts and answers. Key words: who, what, why, when, omit, where, which, choose, find, how, define, label, show, spell, list, match, name, relate, tell, recall, select

Questions: What is . . . ? How is . . . ? Where is . . . ? When did _______ happen? How did ______ happen? How would you explain . . . ? Why did . . . ? How would you describe . . . ? When did . . . ? Can you recall . . . ? How would you show . . . ? Can you select . . . ? Who were the main . . . ? Can you list three . . . ? Which one . . . ? Who was . . . ?

Level 2: Understanding - demonstrating understanding of facts and ideas by organizing, comparing, translating, interpreting, giving descriptions and stating main ideas. Key words: compare, contrast, demonstrate, interpret, explain, extend, illustrate, infer, outline, relate, rephrase, translate, summarize, show, classify Questions: How would you classify the type of . . . ? How would you compare . . . ? contrast . . . ? Will you state or interpret in your own words . . . ? How would you rephrase the meaning . . . ? What facts or ideas show . . . ? What is the main idea of . . . ? Which statements support . . . ? Can you explain what is happening . . . what is meant . . .? What can you say about . . . ? Which is the best answer . . . ? How would you summarize . . . ?

Level 3: Applying - solving problems by applying acquired knowledge, facts, techniques and rules in a different way. Key words: apply, build, choose, construct, develop, interview, make use of, organize, experiment with, plan, select, solve, utilize, model, identify Questions: How would you use . . . ? What examples can you find to . . . ? How would you solve _______ using what you have learned . . . ? How would you organize _______ to show . . . ? How would you show your understanding of . . . ? What approach would you use to . . . ? How would you apply what you learned to develop . . . ? What other way would you plan to . . . ? What would result if . . . ? Can you make use of the facts to . . . ? What elements would you choose to change . . . ? What facts would you select to show . . . ? What questions would you ask in an interview with . . . ?

Level 4: Analyzing - examining and breaking information into parts by identifying motives or causes; making inferences and finding evidence to support generalizations. Key words: analyze, categorize, classify, compare, contrast, discover, dissect, divide, examine, inspect, simplify, survey, take part in, test for, distinguish, list, distinction, theme, relationships, function, motive, inference, assumption, conclusion Questions: What are the parts or features of . . . ? How is _______ related to . . . ? Why do you think . . . ? What is the theme . . . ?

What motive is there . . . ? Can you list the parts . . . ? What inference can you make . . . ? What conclusions can you draw . . . ? How would you classify . . . ? How would you categorize . . . ? Can you identify the difference parts . . . ? What evidence can you find . . . ? What is the relationship between . . . ? Can you make a distinction between . . . ? What is the function of . . . ? What ideas justify . . . ?

Level 5: Evaluating - presenting and defending opinions by making judgments about information, validity of ideas or quality of work based on a set of criteria. Key Words: award, choose, conclude, criticize, decide, defend, determine, dispute, evaluate, judge, justify, measure, compare, mark, rate, recommend, rule on, select, agree, interpret, explain, appraise, prioritize, opinion, ,support, importance, criteria, prove, disprove, assess, influence, perceive, value, estimate, influence, deduct Questions: Do you agree with the actions . . . ? with the outcomes . . . ? What is your opinion of . . . ? How would you prove . . . ? disprove . . . ? Can you assess the value or importance of . . . ? Would it be better if . . . ? Why did they (the character) choose . . . ? What would you recommend . . . ? How would you rate the . . . ?

What would you cite to defend the actions . . . ? How would you evaluate . . . ? How could you determine . . . ? What choice would you have made . . . ? What would you select . . . ? How would you prioritize . . . ? What judgment would you make about . . . ? Based on what you know, how would you explain . . . ? What information would you use to support the view . . . ? How would you justify . . . ? What data was used to make the conclusion . . . ? Why was it better that . . . ? How would you prioritize the facts . . . ? How would you compare the ideas . . . ? people . . . ?

Level 6: Creating - compiling information together in a different way by combining elements in a new pattern or proposing alternative solutions. Key Words: build, choose, combine, compile, compose, construct, create, design, develop, estimate, formulate, imagine, invent, make up, originate, plan, predict, propose, solve, solution, suppose, discuss, modify, change, original, improve, adapt, minimize, maximize, delete, theorize, elaborate, test, improve, happen, change Questions: What changes would you make to solve . . . ? How would you improve . . . ? What would happen if . . . ? Can you elaborate on the reason . . . ? Can you propose an alternative . . . ? Can you invent . . . ?

How would you adapt ________ to create a different . . . ? How could you change (modify) the plot (plan) . . . ? What could be done to minimize (maximize) . . . ? What way would you design . . . ? What could be combined to improve (change) . . . ? Suppose you could _______ what would you do . . . ? How would you test . . . ? Can you formulate a theory for . . . ? Can you predict the outcome if . . . ? How would you estimate the results for . . . ? What facts can you compile . . . ? Can you construct a model that would change . . . ? Can you think of an original way for the . . . ?

Teaching and Researching Higher-Order Thinking in a Virtual Environment University of Manitoba Abstract An innovation in studying the teaching and learning process has been developed at the University of Manitoba. Computer-Aided Personalized System of Instruction (CAPSI) targets questions or problems within small units of study material to initiate composed rather than option-based responses. This system integrates peer review with evaluation by instructor and teaching assistants. Current research focuses on increasing thinking levels by students in courses using CAPSI.

Introduction The Keller ( ) personalized system of instruction (PSI) is a method of self-paced learning in which students proceed through course material at their own pace by writing unit assignments on study questions or problems given to the students beforehand. Other students act as reviewers or tutors by giving feedback on the unit assignments. PSI is a mastery system since students demonstrate mastery on a given unit before they can proceed to the next unit. Research has shown that mastery learning in general and the

Keller system in particular produce superior learning ( ). Bloom's ( ) taxonomy in the cognitive domain is a system for categorizing the thinking levels required by specific questions, problems, or exercises. Bloom identified six major categories of thinking: (1) knowledge, (2) comprehension, (3) application, (4) analysis, (5) synthesis, and (6) evaluation. These categories are roughly hierarchical. For example, to be able to creatively put together several basic concepts to create a new idea (level 5), one must have a good understanding or comprehension (level 2) of those basic concepts. Although Blooms taxonomy is not the only possible way to classify thinking levels, it is widely known and used in education, and therefore provides a good starting point for teaching higher-order thinking and studying its development. This paper describes a method for teaching and studying the teaching and learning process, called computer-aided personalized system of instruction ( ), that combines Keller's PSI and Bloom's taxonomy. Combining the Keller system with Blooms taxonomy presented some problems that required a technological solution. First, the Keller system requires a great deal of routine administrative work to maintain. Second, adding the thinking-level dimension increases the administrative work required. By automating repetitive tasks, computer technology increases the efficiency of the process. Perhaps even more importantly, computer technology makes it possible to study the process in a comprehensive systematic manner. The Method As originally developed by Keller, PSI uses students in a more advanced course as reviewers of assignments by students in less advanced courses. This made sense administratively, because the more advanced course provided a source of individuals who had presumably mastered the material in the less advanced course. With computer technology, however, a more advanced course is unnecessary. Each student's position in the course is available instantaneously. This enables the CAPSI program to use students in the same course as peer reviewers. An added benefit of using computer technology is that students do not have to be at one specific location at one specific time. CAPSItaught courses at the University of Manitoba are conducted through the Internet. An important feature of CAPSI is its quality control potential. In courses at the University of Manitoba, the program requires that a unit assignment be marked by the instructor or teaching assistant or by two peer reviewers. If two peer reviewers mark it, both must independently agree that the assignment is a pass in order for the program to record it as a pass. In addition, all assignments are automatically recorded to disc for the instructor to sample and evaluate. There is also a built-in appeal process for arguing the validity of a given answer. The program is applicable to any course topic and any set of questions or problems. The instructor inputs questions or problems and certain parameters, such as the number of units in the course, the course credit for each unit assignment, the course credit for peer reviewing, and whether there are to be examinations or projects in the course and their respective course credits. The program then automates all the administrative functions of the course. Thus, the study material (e.g., text, videos, lectures) along with the questions, exercises, or problems selected or generated by the instructor form the basis or core of the system. The type of learning that students can acquire from the course will be highly dependent on this core. If the instructor writes questions that require only rote learning (level 1, or knowledge, in Bloom's taxonomy), for example, students will be unlikely to advance above the rote

level. For this reason CAPSI is designed for constructed or composed solutions or answers rather than option-based (e.g., true-false, multiple choice) responses. However, a method for ensuring that students would learn and interact with the material at the highest possible level of thinking was still needed. Hence, a modified form of Bloom's taxonomy was integrated into the system. There were several reasons for modifying the taxonomy. One is that there are reliability problems with the taxonomy (e.g, ). Another is the complexity of the taxonomy, which makes it difficult to apply. Of course, one would expect a classification of thinking levels to be complex. However, it seemed better to simplify the taxonomy and make it more reliable for the purpose of integrating it with CAPSI. It is anticipated that refinement and elaboration of this modified taxonomy will result from research on its use within the CAPSI program. The taxonomy as currently used with CAPSI is as follows: 1. Rote knowledge: the answer is word-for-word or closely paraphrased from the study material. 2. Comprehension: the answer is in the students own words. 3. Application: a concept is applied to a new problem or situation. Examples would be illustrating a concept with a new example (e.g., one not in the study material) and applying an equation to a new problem. 4. Analysis: breaking down a concept into its parts. This occurs when, for example, one compares and contrasts two or more concepts. 5. Synthesis: integrating two or more concepts to form something new. An example would be combining several styles of painting to produce a new style. 6. Evaluation: providing reasoned argument for or against a given position. An example would be an argument considering the pros and cons of cloning research from an ethical perspective or from a scientific perspective. Flowcharts were constructed constructed that permitted the thinking levels of both questions and answers to be assessed with good reliability (). This has set the stage for research on ways to raise the level of thinking at which students respond to questions in CAPSI-taught courses. Research Issues The first foray into investigating increasing thinking levels in student answers consisted in providing students with the modified taxonomy, the thinking level required by each question, and a system of bonus points for each question answered above the level outlined ( ). For example, if a question asked for an example of a concept without specifying that the example had to be original (i.e., not in the study material), this question would be considered to be at level 2. Thus, if a student gave an original example this would be answering at level 3, and therefore would be answering above the level of the question. This procedure successfully increased the levels at which students answered the questions. This shows that students are able to increase their demonstrated thinking levels. Using CAPSI as an instrument for probing students thinking levels in a course, we are in a position to study variables thought to be important in helping students advance their thinking levels. For example, we might use CAPSI to examine whether some study materials and media are more effective at promoting higher-order thinking than others are. Research issues that can be examine include: Are textbooks that are written in a manner that initiates thinking or leads the reader through the discovery process more effective in facilitating higher-order thinking

than those that present a comprehensive coverage of factual material? Are lectures or discussion groups, or some mixture of the two, more effective at promoting higher-order thinking? Are live presentations more effective than videos? Are face-to-face discussions more effective than on-line discussions? These are questions that need to be answered as we advance into the technological age of education. Another important research area concerns the questions, exercises, and problems in a course. Research questions that might be studied here area include: What is the most effective proportion of each category of thinking level? For example, a large proportion of evaluation questions (level 6) might be detrimental because, given the hierarchical nature of the taxonomy, students may not be adequately prepared to successfully address questions at the highest level. What is the most effective way of sequencing the question levels for a given unit in the study guide? For example, would higher-level thinking be more effectively promoted by having students answer all rote questions (level 1) first, then all comprehension questions (level 2) next, etc., or would interspersing the levels be more effective? The information obtained by research on these issues would likely be applicable to courses taught with various other methods, not just those taught using CAPSI. The social milieu in of a CAPSI-taught course also provides a rich source of variables that may have an impact on higher-order thinking. In that milieu are the students, acting both as learners and as peer reviewers, and the instructor and (if there is one) teaching assistant who oversee the system and perform evaluative and feedback functions. Research shows that, overall, students perform their peer-reviewing duties effectively ( ) and that there is a large amount of compliance with feedback that students as learners receive from other students as peer reviewers, and from the instructor ( ). There is, however, considerable room for improvement, and research is in progress on this. Analysis of archived data shows that students in a CAPSI-taught course receive much more substantive feedback on their work than would be possible in a course taught by traditional methods. This interactive nature of CAPSI fits a social constructivist model of knowledge generation through interaction with others ( ). Much of the knowledge generation occurs on the part of the peer reviewers, who find (often to their surprise) that reading other students answers or solutions and commenting on them initiates their own learning and higher-order thinking. Another important area of study, therefore, is on the effects of the peer-review component of CAPSI in the development of higher-order thinking.