Running head: ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

Assessing Second Language Skills: Test Proposal Bridget Schuberg Colorado State University

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Assessing Second Language Skills: Test Proposal Assessments and tests have become an essential component of Western education in the

twentieth century. In fact, the use of objective tests is often considered one of the most influential and crucial steps in the modernization of education (Spolsky, 1978, p. v). Though they are used for a variety of reasons, two of the most common are to provide teachers and researchers with information about effective forms of instruction and to help in decision-making processes related to student ability (Miller, Linn, & Gronlund, 2000; Alderson, 2000). The ultimate goal of a test is to be able to derive an inference from a student's performance on a test task about his/her ability to perform a real-life task (Bachman & Palmer, 1996). The need for authenticity, or correspondence between the test task and a real-life task, is particularly central in determining the usefulness of a test. I previously served as a project assistant to the development of CSU's Kitchen Resource Guide: An Introduction to the Kitchen for Non-Native Speakers of English. To complete this resource guide, project leader Andy Fuller analyzed the entire recipe and instruction book used in Colorado State University kitchens, and identified the most frequent off-list, kitchen-specific vocabulary with which English language learners (ELLs) would most likely lack familiarity. Using this information, the project team created a picture dictionary separated into different categories: general kitchen; food and ingredients; equipment; time, temperature, and weight; holding, serving, and storing, and cooking and preparation. Many of the non-native workers who benefited from the creation of this dictionary also have taken part in the semester-long Workplace English Training (WET) program at CSU, which aims to provide international workers in CSU kitchens with the language skills they will need to succeed in their jobs. Due to both my participation in the development of the kitchen resource guide and my commitment to intern for the WET program in the future, I have an interest in creating a test that would be useful

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

to this same group of ELLs. In this particular context, vocabulary is the most necessary to assess, as a majority of the tasks these workers must perform require an understanding of recipes they are given to follow. Vocabulary knowledge is often assumed to be the foundation of language ability. While competence in other language areas (e.g. grammar, phonology) is not necessarily a prerequisite in either understanding or being understood in a language, vocabulary knowledge is absolutely critical in both receptive and productive tasks. Though for a while, vocabulary tended to be relatively ignored in favor of a greater focus on grammatical and communicative competence, the view of the role of vocabulary as an index of language ability has been renewed once again (Read, 1997). Due to the relative ease with which they can be written, vocabulary items have been one of the most common components in objective language testing from the beginning (Read, 1997, p. 99). Popular test types for assessing vocabulary include multiple-choice tests, cloze tests, in which words are deleted from a reading passage at either a fixed ratio or based on their lexical content, and tests which require test-takers to match vocabulary words with short definitions, such as Nation's Vocabulary Levels Test (Nation, 1990, pp. 261-272). The scores from such objective test items are also desirable as they are considered to have a high rate of test-retest, inter-rater, and intra-rater reliability (Read, 1997, p. 99). However, testing vocabulary is not without difficulty. Some argue that vocabulary knowledge is not truly indicative of language ability, as it is only a lower-order enabling skill' (Read, 1997, p. 104), and as such, is not as valuable as other skills in the development of communicative competence. Furthermore, certain scholars such as Nattinger and DeCarrico (1992, as cited in Read, 1997), believe that groups of words and lexical units play a more important role in spoken and written discourse than do individual words in isolation.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Additionally, context plays an important role in the definition of a word. Therefore, to present vocabulary items in a more developed discourse context decreases the chances that a pure measure of vocabulary untainted by reading ability has been measured (Read, 1997, p. 105). Consequently, the two abilities serve complementary roles at times. For the purposes of my test and the vocabulary my test will include, the context is already established and known for the test-taker. Because I will be testing specialized vocabulary that has been found to be highly

frequent in CSU kitchens (CSUs Kitchen Resource Guide, 2012), the necessary step of selecting target words to both teach and test is already complete. Organization In the rest of this paper, I will provide a description of the test, including its purpose, its type, how scores from the test are to be interpreted, the TLU domain the test is meant to represent, the definition of the construct my test aims to assess, how the test is designed, the test task, and how the items were written. Next, I will discuss how the procedures involving a hypothetical piloting of the test. Then, I will examine the types of analysis that would be appropriate if I were to administer the test. Lastly, I will look at the types of data that would help me argue for the validity of my test. Description of the Test Test Purpose The language purpose of this test is to determine whether or not the test-taker has control of the general skills necessary to succeed in his/her work environment (in a CSU kitchen or dining hall). More specifically, the test-taker will need to demonstrate that he/she is able to recognize words related to food, ingredients, equipment, time, temperatures, weights, holding, serving, storing, cooking, and preparation. More broadly, this test will facilitate the process of transitioning into a new job for English language learners.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

Inference. Because the vocabulary test-takers are asked to translate constitutes over 90% of the words they will encounter in the cookbooks, one should, in theory, be able to infer that the test-takers who pass the test have an understanding of a majority of the words needed to complete the tasks demanded of them and will therefore be able to complete the task of preparing food effectively. Use. As defined by Stoynoff and Chapelle (2005), a score represents a summary of an examinee's performance across one or more tasks on a test (p. 18). The scores from this test should be used only for placement purposesto decide whether an employee possesses the language skills necessary to fulfill his/her workplace responsibilities, or whether he/she needs to enroll in the Workplace English Training program sponsored by Colorado State University. The Workplace English Training program is a 2 hour/week, 10 week course offered for free to English language learners every semester. As Bachman and Palmer (1996) assert, it is extremely important that the inferences meant to be made on the basis of test scores is clearly established prior to administering a test. It is critical that this test not be used for selection of employees; to do so would greatly diminish the chance of non-native speakers being selected. Furthermore, to use this test as a selection aid would create a high-stakes testing environment, increasing the anxiety of the test-taker and thereby potentially reducing both the construct validity and test-retest reliability. Impact. A test's impact is defined as the positive and/or negative effects a test has on the individuals who take the test, a society, and on educational systems (Bachman & Palmer, 1996). On a micro-level, I believe the potential negative effects of my proposed test would be minimal for many reasons. First of all, the test is merely a placement test, not a selection test; failure to pass the test does not result in unemployment, but merely in a short course that comes at no financial cost to the test-taker. Completion of the course is not a prerequisite to

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL employment; in other words, the learner does not need to have finished the Workplace English course before beginning his/her job. Furthermore, the course does not assume any English proficiency; therefore, test-takers need not worry about his/her English level as they might in a

typical English classroom. Finally, the aforementioned CSU Kitchen Resource Guide is available to at least all employees who do not speak English as their native language; it is from this resource guide that the majority of test items are based. I believe that so long as the test-takers are made aware of the low-stakes situation of the test, the impact of this test on individuals should not be negative. I also believe this test could have positive effects at the macro-level, as it could result in greater employment for non-native speakers of English, benefiting both traditionally underprivileged groups and thereby the larger Fort Collins community. Moreover, an increased participation in the Workplace English Training program could provide more opportunities to aspiring teachers, while helping both CSU educators and employers to more precisely identify specific areas of need for language learners, particularly regarding workplace English. Type of Test Because this test aims to measure what a person already knows about workplace language and is not based on any particular syllabus, it is by definition a proficiency test. This test is based on the belief that reading ability and vocabulary knowledge are necessary components of working in a CSU kitchen, and that proficiency in these areas increases an employee's chance of success in the workplace. Interpretation of Scores Test takers will not be scored according to their relative knowledge in comparison to each other or to an external reference group, as no selection or admission decisions will have to be made on the basis of the test results. Instead, this test is designed to provide information on the

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

specific knowledge or skills possessed by an employee (Mitchell, 1982, p. 2), and is therefore a criterion-referenced test. Seventy-five percent of items will need to be answered correctly in order for the test taker to be eligible to opt out of Workplace English Training. Target Language Use Domain Language test developers must have a specific purpose and group of test takers in mind when deciding to create a test if they wish it to be useful. Most importantly, they must specify a situation or context in which the test taker will be using the language outside of the test itself (p. 18) to make the tasks truly useful. This outside situation is commonly referred to as the Target Language Use, or TLU, domain (Bachman & Palmer, 1996). The TLU domain that my test aims to represent is the workplace environment, particularly that of a kitchen or dining hall. The test will attempt to predict the test-taker's ability to successfully complete one TLU task: to prepare food according to a recipe book written in English while simultaneously following health and sanitation procedures (see Appendix B). Definition of Construct Explicitly delineating the precise nature of the ability we want to measure, also known as defining the construct, is a necessary step if we wish to be able to interpret test scores and make inferences about learner ability (Bachman & Palmer, 1996, p. 89). In relation to organizational competence, grammatical knowledge in the form of lexis will be tested, as the all of the test items will be focused on identification and recognition of vocabulary related to food and the workplace. Vocabulary breadth, not depth, will be the focus of this test. Familiarity with all English phonological and graphological units will be assumed, as will minimal topical knowledge of how to prepare food. Strategic competence, sociolinguistic knowledge, textual knowledge, and writing ability will not be tested.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Design of Test A Table of Specifications (TOS) was used in the development of the test to help with the sampling of items and to align content with objectives (Miller et al., 2009). Because the final vocabulary test called for only one task, the TOS indicates that this task composes 100% of the test. The Table of Specifications used for this test (Appendix A) delineates the objective regarding the vocabulary used in the test questions according to their part of speech. This hopes to be a useful distinction for those who will have access to the scores (most likely those running the Workplace English Training program); if a test-taker misses mostly verbs, the emphasis of the training should be on general kitchen actions; if the employee misses mostly nouns, trainers should attempt to focus on products and ingredients; if they seem to have the most trouble with adjectives, they should receive more descriptions of products; if the test-taker misses questions

asking about adverbs, specific manners of carrying out processes should be emphasized; lastly, if he or she mostly misses questions regarding prepositions correctly, he/she should receive more instruction regarding locations. The content column of the TOS uses the K1, K2, and K3 word lists to organize these items. (i.e. their classification according to their frequency in English). Number of parts. There is only one part to this test. As the principle jobs given to English language learners in CSU kitchens are food preparation and food storage, and because employees are expected to use the instructions on how to adequately perform these tasks that are written in cookbooks and handbooks, vocabulary knowledge is the fundamental basis of the employees ability to fulfill his/her workplace tasks. Therefore, only vocabulary knowledge is tested. Sequence of parts. The items are organized according to the frequency with which they appear in the cookbooks that all employees must use when preparing food (from high to low frequency). Should test-takers run out of time, if they complete the items in order, the questions

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL aiming to assess their knowledge of the most high-frequency words (and therefore the ones that are likely to be the most important to successful task performance in the TLU domain) will have been completed first.

Total time to administer test. The total time needed to administer this test is 95 minutes. Five minutes will be set aside before the test is administered for test-takers to read the directions and ask any pertinent questions they may have about the instructions, purpose, scoring, etc. Then, test-takers will be given 90 minutes to complete the test. Description of Test Task Instructions. The instructions are written on the test in the native language of the testtakers just before the actual test items. This is to eliminate any potential confusion about how to mark the correct answer, what the purpose of the test is, how much time they will have to take the test, and how the test will be scored. As the test contains only one task, the general directions apply to the entire test. Following the instructions, an example will be given for test-takers to follow should they still not understand. Input. Because the purpose of this test is to measure the employees ability to function in his/her place of work, the test does not ask the test-takers to define English words in terms of more English words; rather, test-takers must select the most accurate English translation of the word in their native language from a set of provided choices. As the overwhelming majority of employees in need of workplace English training have had an L1 of Spanish, the alternatives are written in Spanish. The focus of this test is on meaning, since these employees will be asked to perform reallife tasks based on their comprehension of the recipes they read in their cookbooks. Although dictionaries explaining kitchen-specific English words can now be found in all CSU kitchens, these were formed using Compleat LexTutor to target topic-specific, off-list vocabulary that

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL would be considered essential to the successful completion of the tasks performed by the

10

employees. While it is important to prioritize the words these employees are least likely to know but very likely to use frequently, it cannot be assumed that the non-native employees have knowledge of the meaning of all the K1 or K2 words used in the cookbooks. Moreover, employees must already understand certain words in English in order to be able to use the information provided in the CSU kitchen dictionaries all employees are provided with. Furthermore, cooking and baking are highly specific processes; misinterpretation of any word given in a recipe could potentially significantly alter the end result. Knowledge of the vocabulary words on this test, when combined with the words present in the kitchen dictionaries that the employees have access to, comprise almost 90% of the total words found in the cookbook. Expected response. Because the test-takers are at a beginning level in their English language learning, the output required in response to the input is minimal; test-takers are merely responsible for circling the best answer. As participants are not required to create his/her own language, the design of the test reflects the fact that the test-takers typically do not have to create their own language to complete their TLU tasks. Relationship between input and response. As the input is the same for all test-takers, this task is non-reciprocal; there is no interaction or feedback between the test-takers and the proctor/s. Only a sentence of context needs to be processed in order to answer the question correctly; therefore, the scope of the relationship is narrow. Finally, as the input supplies all the information needed for the test-takers to respond correctly, the relationship between the input and the response is direct. Scoring. Because this is an objective test in which test-takers can choose the correct answer from a set of alternatives, no scale or rubric is needed. Each correct response will earn one point, while an incorrect response will not earn any points. The total score possible is 90

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

11

points; test-takers must answer a total of 68 items correct for a total of 68 points (75%) in order to pass. After an administration of the test, the appropriateness of this cut-score could be reassessed by analyzing whether those who were considered masters of the test (i.e. those who passed) are also considered masters of the TLU task. Time for task by test-takers. Test-takers will be given 90 minutes to complete the test. The amount of time allotted was chosen according to the suggestion that beginning-level language learners (the target audience) typically are capable of answering around one multiplechoice question per minute. Item-Writing and Editing The CSU Kitchen Resource Guide includes off-list, kitchen-specific terminology. However, I wanted to create a test that would determine whether employees understood the most frequent K1, K2, and K3 words as well. In order to decide which vocabulary words should be included, I uploaded the .txt file of the cookbook workers at Corbett Kitchen are required to use when preparing, serving, holding, and storing food to Compleat Lex Tutor so as to determine which words that were not kitchen-specific (and were on the K1, K2, and K3 lists) occurred most frequently. Next to each vocabulary item on the test is an English sentence from the actual Corbett Kitchen cookbook to provide the test-takers with context for the word they must translate. In coming up with distracters, I often chose English words which look or sound similar to the correct answer translations based on inappropriate senses of the word. Alternatives are listed in alphabetical order. The proper letter choice does not appear more than three times in a row. When I finished constructing the items, a native Spanish-speaker looked at my answers to ensure the alternatives were logical, and that the answers were obvious only to those who truly knew the correct translations.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Pilot Test Procedure Participants The participants in a piloting of this test would have an L1 background of Spanish, in which they would most likely be of advanced proficiency (at least verbally). In order to be in

12

accordance with the target demographic of the actual test-takers of operational administrations of the test, many of them should be immigrant women in their 30s and 40s. The participants of the study may or may not have had prior exposure to the material that will be assessed to align with the background of the takers of the operational test. Administration The WET placement test could be piloted at before the beginning of the upcoming semester (Spring 2013), as the greatest number of hires in the CSU kitchens occurs at the beginning of each semester. This is feasible for both test administrators and test scorers; the WET program does not begin until approximately the fifth week of each semester, allowing ample time for the hiring, test administration, test scoring, item analysis, and item revision processes. The WET program takes place in the Palmer Center, the building dedicated to Housing and Dining Services, as this sector is the largest employer of non-native speakers on the CSU campus. A room in the Palmer Center is therefore where the test will be piloted, as the administrators in this building work in conjunction with the WET program and a room is typically already designated for these classes. Prior to the administration of the test, the test-takers will be informed about how the scores from the test will be used. They will be given five minutes to read the directions and ask any pertinent questions. Then, they will be given 90 minutes to complete the test. Scoring Procedures

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

13

As the test is composed solely of multiple choice items, no special training is required to to administer, proctor, or score it. After the 90 minutes allotted for the test has passed, the tests will be given to the coordinator of the WET program, to score using the answer key. After she determines how many each participant has answered correctly, she will fill out a score report form (Appendix D) to be handed back to the student which will indicate whether or not he/she has surpassed the cut score (in this case, 75% or 68/90) necessary to test out of the WET program. Proposed Test Results Item Statistics Item statistics are necessary for making decisions about the effectiveness of certain items so that a test may be revised to be more useful to all stakeholders. The basic purpose of a criterion-referenced test such as this one is to determine the amount of material that test-takers know or can do (Brown, 2003) for the purpose of making decisions with the results. Item difficulty would be calculated by indicating the percentage of students who got each item right. An items discriminating power could be determined by calculating the numbers of test-takers who got each question right in both the upper and lower groups. Using the same formula for item discrimination, the effectiveness of the distracters of each item could also be analyzed. Using the B-index, one could also determine how well each item contributes to the pass-fail decision made with this particular test. This information could also help to determine if the cut score for this test is appropriate or if it needs to be revised. Descriptive Statistics Descriptive statistics describe a set of scores; they help summarize performance so that interesting trends may be seen. They also serve as the basis for more powerful analyses (Miller et al., 2009). To calculate central tendency (e.g. how test-takers group), I would find the mean, or

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL the average score of data, by taking the sum of all scores and dividing it by the total number of points possible; then, I would plot the participants scores on a frequency polygon (line graph). To calculate variability, I would determine the standard deviation to see to what degree each

14

score deviates from the mean. Ideally, two thirds of the cases will have scores between the mean minus one standard deviation and the mean plus one standard deviation (Miller et al., 2009). Reliability and Standard Error of Measurement Reliability is a necessary component of an assessment, as it contributes to an assessments validity. Reliability measures attempt to reflect the generalization we can make across circumstances, tasks, and raters. In order to collect evidence of the internal consistency of this single assessment, the split-half method could be used to determine to what extent the items in the assessment measure similar characteristics (Miller et al., 2009). Test-retest reliability and inter-rater reliability are not relevant to this test. Lastly, I would calculate the standard error of measurement (SEM), which represents a variation of scores around the true score (i.e. if the test were free from error, what would a particular test takers score be?) Calculating the SEM is a way to gauge our confidence with the interpretation of an assessment; the higher the SEM, the less precise or reliable the assessment may be. Description of Masters/Non-Masters Criterion-referenced tests typically involve the classification of test-takers as either masters or non-masters (those who pass or those who fail). The agreement coefficient can be used to show the proportion of examinees who are considered masters or non-masters on multiple administrations of the same test. However, the proposed test would only be piloted once; therefore, the chart presented by Subkoviak (1998, p. 3) for calculating the agreement coefficients from a single administration (provided one has the z score and the reliability

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL coefficient r) is particularly helpful in determining the overall consistency of those test-takers who would have supposedly mastered the objectives. Discussion Critique of Item Performance

15

To argue for validity, the individual items on my test would have to be well-constructed. In order to determine this, I would calculate each items difficulty and discriminating power. Items on this test should ideally have a difficulty between .6 and .8 as there are four alternatives (Miller et al., 2009). For an item to be classified as discriminating positively, it would have to be answered correctly by the test-takers in the upper group than those in the lower group (Miller et al., 2009). The formula for item discrimination can also be used to determine the effectiveness of distracters; if the distracter attracts more students from the lower group than the upper group, it can be deemed effective. Lastly, the closer the B-index value is to 1.00, the more it contributes to the final pass/fail decision; the closer it is to -1.00, the less effective the item is in making these decisions. Once the test was piloted and these values could be calculated, one could speak more to teach items individual performance, and therefore, its effectiveness. Evaluation of Test Usefulness Reliability. The statistics produced from an analysis using either the split-half method or coefficient alpha could be examined to determine this tests internal consistency. The closer the correlation coefficients are to 1, the more reliable the test may be considered. The SEMs for each score range would also have to be analyzed in determining the tests reliability; the smaller the SEM, the higher the reliability. Once the test was piloted and these values could be calculated, more evidence for reliability could be accrued.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL This test is a selection test; therefore, its objectivity contributes to reliability. Its large

16

number of items (90) for its one construct could also be seen as contributing to reliability (Miller, Linn, & Gronlund, 2009). Construct-related evidence for validity. The construct that my test was purported to measure was the test-takers ability to understand and follow a written recipe. Because the items that the test task is based on are taken directly from the actual material employees use when completing the TLU task, we may consider the test task to be highly representative of the TLU domain; it therefore should theoretically have a high degree of authenticity. In order to argue for a high degree of construct validity, one would have to determine the success of the employees in this task and then correlate these employees scores on this test. In this way, one could see if the highest-scoring students with their rank in terms of the quality of their performance in their completion of the TLU task. Additionally, one could compare test scores between two groups: one that has been deemed unprepared to prepare food in a CSU kitchen, and one that has been successfully completing the tasks required of them; the scores of the latter group should be higher than the cut-score set for this test. Along these same lines, one could administer the same test to the same group of people at two different times: once before they worked in a CSU kitchen, and once after they have successfully been carrying out their tasks there; if the scores from the second administration are passing, we might be able to say the test is highly reliable. Consequential evidence for validity. As validity is defined as the appropriateness of the interpretation and use made of assessment results, impact is extremely important to consider when determining the degree of validity of a certain test use. In order to argue that this tests impact contributes to validity rather than hinders it, the aforementioned aspects of construct validity and reliability would need to be determined; in this way, one could argue that the

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL assessment scores are consistent, relevant to the decisions that are to be made, and are generalizable to the TLU task. A questionnaire about the test-takers feelings about their perceptions of the test could be used to collect feedback about the impact of the test on their attitudes toward their job and toward their language learning. The kitchen managers and those who work with the international employees could also provide feedback on the performance of

17

the test-takers who tested out of the WET program in order to determine if they believe the test is placing workers appropriately. Practicality. This test may be deemed practical based on the availability of employees to take the test, the availability of proctors to administer and monitor the test, the cost of printing and distributing the test, and the ease with which a room may be reserved for the administration of the test. This information could be ascertained after several attempts to administer the test. Additionally, the scores of the test are easily interpreted, and the score report form contributes to clarity. Achievement of Test Purpose The purpose of this test, as defined earlier, is to determine whether or not the test-takers have the vocabulary knowledge necessary to follow a recipe according to the sanitation guidelines. In order to decide whether or not the purpose has truly been achieved, then, one would have to run statistical analyses of the reliability and validity of the test items and results according to the aforementioned methods, and determine whether or not these numbers fall into the acceptable ranges. Reflection The most prominent theme that comes to mind when reflecting on my own test development process is that I never realized all of the components that are required of a wellmade test. I often just assumed the competence of test developers and those who make inferences

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL based on test scores, and thus never questioned whether the items truly tested the defined

18

construct, whether the test was reliable, and whether the use of the test could be considered valid. However, now that I know the many factors that must be considered when creating and justifying a tests use, I will look much more critically at the tests I am given or the tests I may be told to give to students. In the future, when I create tests, I know now to ensure that I allow enough time for its development, editing, piloting, analysis, and revision. I will also pay closer attention to the Table of Specifications. For this particular test, I felt that the frequency with which the words appear in the employee cookbook was sufficient justification for the items I wrote. However, the organization of the content and the objectives sections of my TOS might be seen as weak. Furthermore, since the items are organized from high to low frequency, perhaps it would be possible to weight the items differently, as understanding of certain words on the test may be more significant than others with regard to the TLU domain.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL References

19

Alderson, J. C. (2000). Chapter 7: Techniques for testing reading. In Assessing reading (pp. 202270). New York: Cambridge University Press. Bachman, L., & Palmer, A. (1996). Language testing in practice. New York: Oxford University Press. Brown, J.D. (2003). Criterion-referenced item analysis (online). JALT, 7, 18-24. Retrieved from http://jalt.org/test/bro_18.htm Miller, M.D., Linn, R., & Gronlund, N. (2008). Measurement and assessment in teaching.(Tenth Edition). Upper Saddle River, NJ: Merrill, Prentice Hall. Mitchell, B. (1982). A glossary of measurement terms. (Test Service Notebook 13). NY: Harcourt Brace Jovanovich, Inc. Nation, I.S.P. (1990). Teaching and learning vocabulary. New York: Heinle and Heinle. Nattinger, J.R. & DeCarrico, J.S. (1992). Lexical phrases and language teaching. Oxford: Oxford University Press. Read, J. (1997). Assessing vocabulary in a second language. In C. Clapham and D. Corson (Eds.), Encyclopedia of language and education: Vol. 7. Language testing and assessment (pp. 99-107). Norwell, MA: Kluwer Academic Publishers. Spolsky, B. (1978). Introduction: Linguists and language testers. In B Spolsky (Ed.), Approaches to language testing, (pp. v-x). Washington, D.C.: Center for Applied Linguistics. Stoynoff, S., & Chapelle, C. (2005). ESOL Tests and Testing. Alexandria, VA: TESOL, Ch. 1 Subkoviak, M. (1988). A practitioners guide to computation and interpretation of reliability indices for mastery tests. Journal of Educational Measurement, 25, 47-55.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Appendix A Table of Specifications Text Understanding meaning of vocabulary items in context Descriptions (Adjectives) K1 wordlist K2 Wordlist K3 wordlist # of items per section % of items per section 15 8 2 25 # of items % of items

20

General Actions (Verbs) 13 7 2 22

Objects (Nouns) 16 2 3 21

Specific Processes (Adverbs) 11 0 1 12

Locations (Prepositions) 9 1 0 10 64 18 8 90 71.1% 20% 8.9% 100%

~27.8%

~24.4%

~23.3%

~13.3%

~11.1%

100%

Items According to Type and Category Adjectives K1 wordlist 5, 12, 18, 19, 32, 33, 34, 43, 53, 57, 62, 63, 69, 73, 79 13, 26, 38, 41, 42, 50, 55, 60 36, 39 Verbs 1, 4, 7, 8, 46, 48, 51, 70, 76, 77, 83, 84, 90 14, 15, 20, 22, 35, 44, 72 40, 45 Nouns 3, 16, 24, 25, 27, 29, 30, 47, 49, 65, 66, 74, 82, 86, 87 17, 28, 78 Adverbs 31, 52, 54, 56, 59, 61, 71, 75, 80, 81, 85 Prepositions 6, 10, 11, 21, 23, 37, 58, 64, 88,

K2 wordlist

K3 wordlist

2, 67

68

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

21

Appendix B TLU Domain Description TLU Task #1:Using a recipe book to prepare food while following health and sanitation procedures. Characteristics of the setting physical characteristics kitchen Participants employees time of task morning, afternoon, evening Characteristics of the input Format Channel visual Form language Language target language (English) Length short sentences Type series of items degree of speededness normal speed (~160-180 wpm) Vehicle reproduced language of the input language characteristics organizational characteristics Grammatical full set of graphological features; simple and frequent vocabulary; statements and commands Textual pragmatic characteristics Functional instrumental Sociolinguistic topical characteristics food-related or workplace vocabulary Characteristics of the expected response Format Channel tactile (producing the end food product) Form non-language Language Length Type degree of speededness language of the expected response language characteristics organizational characteristics Grammatical

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Textual pragmatic characteristics Functional Sociolinguistic topical characteristics Relationship between input and response Reactivity scope of relationship directness of relationship

22

food-related and workplace-related vocabulary non-reciprocal narrow direct

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

23

Appendix C Test Nombre: ______________________________________________ Fecha: _______________ El propsito de este exmen es evaluar su saber del significado del vocabulario bsico que necesita para trabajar en una cocina de CSU. Tiene 90 minutos para tomar el exmen. Cada pregunta ser marcado correcto o incorrecto. Los resultados sern usados para determinar si necesita participar en el curso de Workplace English Training. Instrucciones: Lea cada oracin ingles. Despus de leer el oracin de contexto, trazar un crculo alrededor de la palabra/frase que representa la mejor traduccin de la palabra/frase en negrita. No adivine si no sabe la respuesta. Por ejemplo: 0. cup a. cuenco b. olla c. taza d. vaso Put the liquid in the cup.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 1. to need a. deber b. exigir c. necesitar d. querer 2. recipe a. advertencia b. lista c. mensaje d. receta 3. hands a. dedos b. manos d. platos d. utensilios 4. to begin a. cocinar b. continuar c. empezar d. terminar 5. clean a. limpio b. modificado c. nuevo d. sano 6. until a. antes b. despus c. hasta d. mientras 7. to add a. aadir b. exprimir c. medir d. sacar Wash hands prior to beginning preparation. Refrigerate and serve as needed.

24

Ensure that all preparation equipment and utensils needed for this recipe are cleaned and sanitized.

Wash hands prior to beginning preparation.

Place product in clean sanitized serving pans and cover.

Keep chilled at 41F or below until ready to use.

Add lemon juice, water, and salt.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 8. to keep a. calentar b. congelar c. conservar d. mantener 9. below a. bajo b. ms c. por d. sobre 10. when a. antes de b. cuando c. despus de d. donde 11. during a. antes de b. durante c. despus de d. si 12. ready a. disponible b. listo c. deseado d. necesario 13. internal a. alto b. ambiente c. extremo d. interior 14. to maintain a. anotar b. mantener c. medir d. observar Maintain temperature at 140F or above during holding and serving. Keep chilled at 41F or below until ready to use.

25

Keep chilled at 41F or below until ready to use.

Follow all appropriate sanitation procedures when handling and serving product.

Keep chilled at 41F or below until ready to use.

Cook to internal temperature of 135F.

Maintain temperature at 140F or above during holding and serving.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 15. to reach a. extenderse b. contactar c. llegar a d. sobrepasar 16. seconds a. secondarios b. secos c. segundos d. siguientes 17. directions a. direcciones b. distribuciones c. instrucciones d. introducciones 18. hot a. caliente b. congelado c. fro d. tibio 19. cold a. caliente b. congelado c. fro d. tibio 20. to remove a. agregar b. reemplazar c. remover d. sacar 21. over a. abajo de b. al lado de c. dentro de d. encima de Prepare crepes according to recipe directions. Ensure that skillet reaches an internal temperature of 165F for 15 seconds. Cooking time will take from 5-7 minutes to reach 165F internal temperature.

26

Add hot water.

Keep cold.

Remove from oven.

Spread 2lb 4oz mixture over each brownie.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 22. to cool a. agitar b. calentar c. enfriar d. mezclar 23. before a. antes de b. despus de c. mientras d. si 24. top a. centro b. fondo c. parte de abajo d. parte superior 25. time a. poca b. medidas c. tiempo d. veces 26. fresh a. cido b. delicioso c. dulce d. fresco 27. pieces a. filas b. pedazos c. pilas d. tiras 28. bowl a. cacerola b. cuenco c. olla d. taza 29. half a. mitad b. cuarto c. tercio d. todo Remove starch and cool completely.

27

Pan roast allowing golden color to form before turning.

Spread a thin layer of sour cream over the top.

The amount of time should not exceed 4 hours.

Prepare fresh lime juice according to recipe direction.

Divide dough into 2 lb 2 oz pieces.

Combine salt and sugar in a bowl.

Cut in half.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 30. amount a. calidad b. cantidad c. montn d. poquitn 31. well a. bien b. bueno c. lo mejor d. mejor 32. best a. bien b. bueno c. lo mejor d. mejor 33. green a. amarillo b. blanco c. prpura d. verde 34. yellow a. amarillo b. blanco c. prpura d. verde 35. to select a. empujar b. escoger c. tirar d. tocar 36. smooth a. con azcar b. con grumos c. sin azcar d. sin grumos

28

Cooking time will vary depending on the amount of product in the oven.

Mix well.

Best served fresh.

Prepare chopped green onions.

Prepare recipe for diced yellow onions.

Select "pan fried" from SCC menu.

Puree until smooth.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 37. under a. al lado de b. alrededor de c. debajo de d. encima de 38. golden a. crudo b. dorado c. medio crudo d. quemado 39. medium a. bajo b. mximo c. mediano d. mnimo 40. layer a. capa b. parte c. regin d. seccin 41. red a. amarillo b. naranja c. rojo d. verde 42. dry a. blando b. firme c. mojado d. seco 43. small a. corto b. grande c. largo d. pequeo 44. to spread a. doblar b. mezclar c. partir d. untar Run under cold water.

29

Bake at 375F for 10 to 12 minutes, or until puffed and golden.

Heat dry saute pan over medium heat for 2 minutes.

Spread a thin layer of sour cream over the top.

Prepare red pepper strips.

Strawberries should be dry.

Cut tofu into small rectangles.

Spread across sheet pan evenly in single layer.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 45. to toss a. lanzar b. mezclar c. organizar d. tirar 46. to check a. controlar b. determinar c. servir d. verificar 47. day a. da b. maana c. noche d. tarde 48. to make a. frer b. hacer c. hornear d. marcar 49. side a. dorso b. frente c. lado d. rincn 50. thick a. delgado b. duro c. fino d. grueso 51. to turn off a. abrir b. apagar c. cerrar d. encender Toss and stir until all ingredients are hot.

30

Check internal temperature each time.

Best served same day.

Make sauce by whisking soy, sriracha, fresh ginger and olive oil together.

Brown well on both sides until 165F internal temperature is reached.

Slice onions and peppers in thick strips about 2 inches wide.

Turn off heat and let steep for 1-2 hours.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 52. very a. demasiado b. muy c. nunca d. siempre 53. done a. caliente b. cocido c. seco d. mojado 54. lightly a. claramente b. ligeramente c. oscuramente d. suavemente 55. brown a. correoso b. crocante c. dorado d. quebradizo 56. fully a. a mitad b. achicharrado c. enteramente d. parcialmente 57. same a. diferente b. limpio c. mismo d. nuevo 58. through a. al lado de b. a travs de c. con d. encima de 59. evenly a. equitativamente b. finalmente c. lentamente d. rpidamente Drain tofu very well.

31

When noodles are done, drain and rinse with cold water.

Lightly season chicken with salt and pepper.

Deep fry at 350F for 3 minutes, until crisp and lightly brown.

Bake shell for 20 minutes until almost fully baked.

In same pan, saute onions and garlic.

Strain sauce through china cap.

Evenly distribute the carrots, cucumbers and green onions.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 60. soft a. aromtico b. blando c. crujiente d. dorado 61. briefly a. brevemente b. cuidadosamente c. lentamente d. rpidamente 62. high a. alto b. extremo c. lento d. mediano 63. low a. alto b. extremo c. lento d. mediano 64. while a. antes de b. despus de c. mientras d. por 65. bag a. bolsa b. caja c. nevera d. congelador 66. bottom a. fondo b. lado c. parte superior d. rincn Saute onions until soft and fragrant over medium heat.

32

Warm in the oven briefly to re-crisp pastry.

Heat nonstick pan over medium high heat.

Mix on low for 3-4 minutes.

Add pasta while gradually stirring.

Thaw chicken in bags in cooler.

Scrape bottom of pan to incorporate any browned bits.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 67. flavor a. aparicin b. color c. olor d. sabor 68. gently a. cuidadosamente b. generosamente c. rpidamente d. totalmente 69. large a. corto b. grande c. largo d. pequeo 70. to leave a. calentar b. cuidar c. dejar d. salir 71. more a. ms b. menos c. muchos d. pocos 72. to transfer a. cambiar b. pasar c. transferir d. reemplazar 73. warm a. caliente b. fro c. helado d. tibio 74. box a. bolsa b. caja c. paquete d. libro Full cake flavor is best at room temperature.

33

Gently smooth glaze along top and sides of cake.

Scale 4 pounds mixture into large roasting pans.

Leave at room temperature for immediate service.

Continue cooking for 10 more minutes.

Then transfer pizza to cutting board.

Pour in warm water.

Cook according to box instructions.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 75. carefully a. cuidadosamente b. finalmente c. lentamente d. rpidamente 76. to cover a. dejar b. mezclar c. sacar d. tapar 77. to lay a. cortar b. estirar c. poner d. separar 78. plate a. bandeja c. bol c. plata d. plato 79. flat a. blando b. duro c. liso d. plano 80. at least a. ms de b. menos de c. por lo mximo d. por lo menos 81. directly a. inmediatamente b. directamente c. luego d. primero Carefully fold eggs into noodles.

34

Cover bowl and let set at room temperature.

Lay sausage on sheet tray.

Turn the pan upside down, rolling the omelet onto the plate.

Grill on flat top until golden brown and cheese is melted.

Preheat oven for at least 1 hour before using.

Roast directly on oven racks.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 82. ends a. macas b. moho c. pednculos d. puntas 83. to form a. formar b. hacer al grill c. hornear d. separar 84. to order a. pedir b. poder c. poner d. preguntar 85. quickly a. cuidadosamente b. generosamente c. lentamente d. rpidamente 86. size a. altura b. distancia c. longitud d. tamao 87. speed a. calor b. grado c. nivel d. velocidad 88. between a. al lado de b. enfrente de c. entre d. atrs de 89. center a. centro b. fondo c. lado d. parte superior Cut off ends.

35

Form into 1.75 oz patties.

Assemble and serve when ordered.

Cook quickly until just done

Medium Time: 17-25 minutes (depending on batch size.)

Beat for 2-3 more minutes on low speed.

Place wings on pan with space between them.

Cake should be just firm in the center when done.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 90. to continue a. continuar b. empezar c. terminar d. parar Continue baking for 5-10 minutes until cheese is melted.

36

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Answer Key 1. c. necesitar 2. d. receta 3. b. manos 4. c. empezar 5. a. limpio 6. c. hasta 7. a. aadir 8. d. mantener 9. a. bajo 10. b. cuando 11. b. durante 12. b. listo 13. d. interior 14. b. mantener 15. c. llegar a 16. c. segundos 17. c. instrucciones 18. a. caliente 19. c. fro 20. d. sacar 21. d. encima de 22. c. enfriar 23. a. antes de 24. d. parte superior 25. c. tiempo 26. d. fresco 27. b. pedazos 28. b. cuenco 29. a. mitad 30. b. cantidad 31. a. bien 32. c. lo mejor 33. d. verde 34. a. amarillo 35. b. escoger 36. d. sin grumos 37. c. debajo de 38. b. dorado 39. c. mediano 40. a. capa 41. c. rojo 42. d. seco 43. d. pequeo 44. d. untar 45. b. mezclar 46. d. verificar 47. a. da 48. b. hacer 49. c. lado 50. d. grueso 51. b. apagar 52. b. muy 53. c. cocido 54. b. ligeramente 55. d. dorado 56. c. enteramente 57. c. mismo 58. b. a travs de 59. a.equitativamente 60. b. blando 61. a. brevemente 62. a. alto 63. c. lento 64. c. mientras 65. a. bolsa 66. a. fondo 67. d. sabor 68. a. cuidadosamente 69. b. grande 70. c. dejar 71. a. ms 72. b. pasar 73. d. tibio 74. b. caja 75. a. cuidadosamente 76. d. tapar 77. c. poner 78. d. plato 79. d. plano 80. d. por lo menos 81. b. directamente 82. d. puntas 83. a. formar 84. a. pedir 85. d. rpidamente 86. d. tamao 87. d. velocidad 88. c. entre 89. a. centro 90. a. continuar

37

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

38

Translation of Test Name _________________________________________________ Date __________________ The purpose of this exam is to test your knowledge of the meaning of basic kitchen vocabulary you will need to work in a CSU kitchen. You will have 90 minutes to take the test. Each response will be marked either correct or incorrect. The results will be used to determine if you need to participate in the Workplace English course. Directions: Read each English sentence. After reading the sentence of context, circle the answer which represents the best translation of the word/phrase in bold. Do not guess if you do not know the answer. For example : 0. cup a. bowl b. pot c. cup d. glass 1. to need a. to have to b. to demand c. to need d. to want 2. recipe a. warning b. list c. message d. recipe 3. hands a. fingers b. hands c. plates d. utensils 4. to begin a. to cook b. to continue c. to begin d. to finish Wash hands prior to beginning preparation. Put the liquid in the cup.

Refrigerate and serve as needed.

Ensure that all preparation equipment and utensils needed for this recipe are cleaned and sanitized.

Wash hands prior to beginning preparation.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL

39

5. clean a. clean b. modified c. new d. sanitary 6. until a. before b. after c. until d. while 7. to add a. to add b. to squeeze c. to measure d. to take out

Place product in clean sanitized serving pans and cover.

Keep chilled at 41F or below until ready to use.

Add lemon juice, water, and salt.

8. to keep Keep chilled at 41F or below until ready to use. a. to heat b. to freeze c. to conserve d. to keep/maintain (something cold) 9. below a. below b. more c. through d. above 10. when a. before b. when c. after d. where Keep chilled at 41F or below until ready to use.

Follow all appropriate sanitation procedures when handling and serving product.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 11. during a. before b. during c. after d. if 12. ready a. available b. ready c. desired d. necessary 13. internal a. high b. room c. extreme d. internal 14. to maintain a. to write down b. to maintain c. to measure d. to observe 15. to reach a. to spread b. to contact c. to reach d. to surpass 16. seconds a. secondary b. dry c. seconds d. following 17. directions a. addresses b. distributions c. directions d. introductions Prepare crepes according to recipe directions. Ensure that skillet reaches an internal temperature of 165F for 15 seconds. Maintain temperature at 140F or above during holding and serving.

40

Keep chilled at 41F or below until ready to use.

Cook to internal temperature of 135F.

Maintain temperature at 140F or above during holding and serving.

Cooking time will take from 5-7 minutes to reach 165F internal temperature.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 18. hot a. hot b. frozen c. cold d. warm 19. cold a. hot b. frozen c. cold d. warm 20. to remove a. to add b. to replace c. to stir d. to remove 21. over a. under b. next to c. inside d. over 22. to cool a. to shake b. to heat c. to cool d. to mix 23. before a. before b. after c. while d. if 24. top a. center b. end c. bottom d. top Add hot water.

41

Keep cold.

Remove from oven.

Spread 2lb 4oz mixture over each brownie.

Remove starch and cool completely.

Pan roast allowing golden color to form before turning.

Spread a thin layer of sour cream over the top.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 25. time The amount of time should not exceed 4 hours. a. time (in history) b. measurements c. time d. times (occurrences) 26. fresh a. acidic b. delicious c. sweet d. fresh 27. pieces a. rows b. pieces c. piles d. strips 28. bowl a. saucepan b. bowl c. pot d. cup 29. half a. a half b. a quarter c. a third d. a whole 30. amount a. quality b. amount c. a lot of d. a little bit of 31. well a. well b. good c. the best d. better Prepare fresh lime juice according to recipe direction.

42

Divide dough into 2 lb 2 oz pieces.

Combine salt and sugar in a bowl.

Cut in half.

Cooking time will vary depending on the amount of product in the oven.

Mix well.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 32. best a. well b. good c. the best d. better 33. green a. yellow b. white c. reddish-purple d. green 34. yellow a. yellow b. white c. reddish-purple d. green 35. to select a. to push b. to select c. to pull d. to touch 36. smooth a. with sugar b. not smooth c. sugarless d. smooth 37. under a. next to b. around c. under d. above 38. golden a. raw b. golden c. semi-cooked d. burned Best served fresh.

43

Prepare chopped green onions.

Prepare recipe for diced yellow onions.

Select "pan fried" from SCC menu.

Puree until smooth.

Run under cold water.

Bake at 375F for 10 to 12 minutes, or until puffed and golden.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 39. medium a. low b. maximum c. medium d. minimum 40. layer a. layer b. part c. region d. section 41. red a. yellow b. orange c. red d. green 42. dry a. soft b. firm c. wet d. dry 43. small a. short b. large c. long d. small 44. to spread a. to turn b. to combine c. to cut d. to spread 45. to toss a. to throw b. to toss c. to organize d. to throw out Heat dry saute pan over medium heat for 2 minutes.

44

Spread a thin layer of sour cream over the top.

Prepare red pepper strips.

Strawberries should be dry.

Cut tofu into small rectangles.

Spread across sheet pan evenly in single layer.

Toss and stir until all ingredients are hot.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 46. to check a. to control b. to determine c. to serve d. to check 47. day a. day b. morning c. night d. afternoon 48. to make a. to cool b. to make c. to bake d. to mark 49. side a. back b. front c. side d. corner 50. thick a. thin b. hard c. fine d. thick 51. to turn off a. to open b. to turn off c. to close d. to turn on 52. very a. too b. very c. never d. always Check internal temperature each time.

45

Best served same day.

Make sauce by whisking soy, sriracha, fresh ginger and olive oil together.

Brown well on both sides until 165F internal temperature is reached.

Slice onions and peppers in thick strips about 2 inches wide.

Turn off heat and let steep for 1-2 hours.

If carrot is very slender, don't cut it lengthwise.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 53. done a. hot b. done c. dry d. wet 54. lightly a. lightly (in color) b. lightly c. darkly d. gently 55. brown a. crunchy b. toasted c. crumbly d. brown 56. fully a. halfway b. burned c. fully d. partially 57. same a. different b. clean c. same d. new 58. through a. next to b. through c. with d. on top of 59. evenly a. evenly b. finally c. slowly d. quickly When noodles are done, drain and rinse with cold water.

46

Lightly season chicken with salt and pepper.

Deep fry at 350F for 3 minutes, until crisp and lightly brown.

Bake shell for 20 minutes until almost fully baked.

In same pan, saute onions and garlic.

Strain sauce through china cap.

Evenly distribute the carrots, cucumbers and green onions.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 60. soft a. fragrant b. soft c. crunchy d. golden 61. briefly a. briefly b. carefully c. slowly d. quickly 62. high a. high b. extreme c. low (heat) d. medium 63. low a. high b. extreme c. low d. medium 64. while a. before b. after c. while d. through 65. bag a. bag b. box c. refrigerator d. freezer 66. bottom a. bottom b. side c. top d. corner Saute onions until soft and fragrant over medium heat.

47

Warm in the oven briefly to re-crisp pastry.

Heat nonstick pan over medium high heat.

Mix on low for 3-4 minutes.

Add pasta while gradually stirring.

Thaw chicken in bags in cooler.

Scrape bottom of pan to incorporate any browned bits.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 67. flavor a. appearance b. color c. smell d. flavor 68. gently a. gently b. generously c. quickly d. totally 69. large a. short b. large c. long d. small Full cake flavor is best at room temperature.

48

Gently smooth glaze along top and sides of cake.

Scale 4 pounds mixture into large roasting pans.

70. to leave Leave at room temperature for immediate service. a. to heat b. to care for c. to leave (something) d. to leave (a place) 71. more a. more b. fewer c. many d. few Continue cooking for 10 more minutes.

72. to transfer Then transfer pizza to cutting board. a. to change b. to transfer (something) c. to transfer (a person) d. replace 73. warm a. hot b. cold c. frozen d. warm Pour in warm water.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 74. box a. bag b. box c. packet d. book 75. carefully a. carefully b. finally c. slowly d. quickly 76. to cover a. to leave b. to mix c. to take out d. to cover 77. to lay a. to cut b. to stretch c. to lay d. to separate 78. plate a. tray b. bowl c. silver d. plate 79. flat a. soft b. hard c. smooth d. flat 80. at least a. more than b. less than c. at the most d. at least Cook according to box instructions.

49

Carefully fold eggs into noodles.

Cover bowl and let set at room temperature.

Lay sausage on sheet tray.

Turn the pan upside down, rolling the omelet onto the plate.

Grill on flat top until golden brown and cheese is melted.

Preheat oven for at least 1 hour before using.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 81. directly a. immediately b. directly c. later d. first 82. ends a. bruises b. mold c. stems d. ends 83. to form a. to form b. to grill c. to bake d. to separate 84. to order a. to order b. to be able to c. to put d. to ask 85. quickly a. carefully b. generously c. slowly d. quickly 86. size a. height b. distance c. length d. size 87. speed a. heat b. degree c. level d. speed Roast directly on oven racks.

50

Cut off ends.

Form into 1.75 oz patties.

Assemble and serve when ordered.

Cook quickly until just done.

Medium Time: 17-25 minutes (depending on batch size.)

Beat for 2-3 more minutes on low speed.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL 88. between a. next to b. in front of c. between d. behind 89. center a. center b. bottom c. side d. top 90. to continue a. to continue b. to begin c. to finish d. to stop Place wings on pan with space between them.

51

Cake should be just firm in the center when done.

Continue baking for 5-10 minutes until cheese is melted.

ASSESSING SECOND LANGUAGE SKILLS: TEST PROPOSAL Appendix D Administrative Score Report Form The following score report is from the Workplace English Training Placement Test taken by ___________________________________________ on ________/ ________ / ________. Employee Name / CSU Employee ID # month day year

52

The employee was evaluated according to the correctness of his/her selections. Each item answered correctly was worth 1 point. There were a total of 90 points.

Number of items correct involving: Adjectives: ______ / 27 Verbs: _____ / 22 Nouns: ______ / 20 Adverbs: ______ / 11 Prepositions: ______ / 10

Total Raw Score: ______ / 90 Percentage Score: ________

Raw score required to test out of Workplace English Training: 68/90 Percentage score required to test out of Workplace English Training: 75% Does this employee test out of Workplace English Training? Yes _____ No______