ASSESSING STUDENTS ABILITY TO USE NEW VOCABULARY IN WRITTEN SENTENCES Patrisius Istiarto Djiwandono Universitas Ma Chung

As the title suggests, the paper aims to discuss the assessment of abilities to use new vocabulary. The focus is on two techniques for teaching new vocabulary, defined here as collocations, that are viewed from construct validity framework. The paper presents a report on the comparison between two different techniques of teaching collocates, one of them providing model sentences, and the other does not. The focus was on the learners abilities to create natural collocates after they were exposed to the two techniques. A quasi experimental design was used in the research. The result is then discussed within the framework of construct validity.

Method Twenty two sophomores in the English Letters Department of Universitas Ma Chung who were taking Vocabulary class were taken as the subjects. In the first technique, the subjects were asked to translate these following ten target words: 1. Proceed 2. Conduct 3. Layers 4. Confer 5. Sustain 6. Sole 7. Bond 8. Tense 9. Albeit 10. Reluctance

CamTESOL 2012

Page 1

They were then prompted to check the answers in some model sentences given by the lecturer. They were asked to notice how the new words were used in these model sentences. Then they were instructed to use these target words in sentences of their own. In the subsequent week, they were again instructed to use the same words in their own sentences, but this time they did that without noticing preceding model sentences. The objective of this sentence production was to see the naturalness of their collocations with the new words they just learned. It was hypothesized that students who are instructed to notice model collocates will be able to produce more natural or appropriate collocations than when they write their sentences without noticing the model. Their sentences from the first week and the second week were then scored for the naturalness of the collocations of the target words. The analysis showed that the first technique helped them better in producing natural collocates, which hints at effective teaching of collocations. The following table shows the result of the analysis using t-test for repeated measures: Table 1. Scores after Scores after

Treatment Treatment 1 Average SD Total 6.227273 2.022064 137 2 4.590909 2.085302 101

P value: 0.0006 t = 4.0059 df = 21 standard error of difference = 0.408

CamTESOL 2012

Page 2

Discussion A fundamental concept of test that is relevant to the two techniques here is construct validity, defined as the quality of a test in terms of the theoretical model on which the test is based (Davies et al, 1999: 222). The concept is applicable to each technique insofar as the theory of ability to produce collocation is represented adequately by the tasks. In the first task where the students were allowed to see some models of collocates, the validity of this kind of test hinges on whether the ability to produce collocates can be perceived as depending upon the noticing of the models. In other words, if it is believed that forming collocations is the result of noticing some model collocates before the act of producing collocates itself takes place, then the first test can be considered valid. In the second technique where the students were left without any model collocates, the validity depends on whether the ability can be defined as the act of producing collocates without the assistance of a previous noticing of some models. The question of which theory seems more plausible is then an open question for which further research can be conducted. Mongkolchais study (2008), for instance, focuses on the collocational ability of Thai students and implies subconscious acquisition of collocates. It is argued that collocational ability grows together with the learning of vocabulary. Howvere, later in the report, it is implied that some kind of cosncious attention to explicit instruction is needed. The latter hints at the importance of providing models of collocates and making students notice them before they can be asked to to produce correct collocations on their own. Thus, Mongkolchai (2008:24) states that Focus needs to be placed on building students consciousness of how words work in combination with one another, so that they can continue developing their collocational competence after they leave the ESL class. Clear and explicit introduction is needed to provide guidance in building learners awareness.

The finding itself can be discussed in the light of some relevant studies. Nakata (n.d., http://www.paaljapan.org/resources/proceedings/PAAL11/pdfs/13.pdf, ) suggests that there are two different kinds of collocations, namely, non-congruent, and congruent. The former includes those English collocations which do not have equivalents in the learners mother tongue; the latter covers those collocates which have equivalent expressions in their mother tongue. Nakata concludes that non-congruent collocates calls for a learning approach which intentionally focuses the learners attention to them. A cursory analysis of the target words in
CamTESOL 2012 Page 3

this current study suggested that the following words are of non-congruent types: confer, conduct, proceed and sustain. These following words, in contrast, are potentially of congruent type: bond, albeit, layers and reluctance. It was noticed that the subjects rarely made mistakes in making sentences using the latter words. In general, the finding espouses a previous finding by Rezvani (2011), who found that production activities and noticing grammatical collocations are more effective than just reading an English text. The subjects failure in producing correct collocates may be attributed to overdependence on dictionaries and thesaurus. Regarding this, Lee (2010:1) argues that the interaction between lexicon and pragmatics is often overlooked by most L2 learners because they rely on dictionaries and thesaurus to provide denotational meaning of a lexical item which lack ample collocational information. Thus, it is believed that the subjects habit of looking up words in dictionaries has made them oblivious of the collocates that typically appear with those words. This might have explained why most of them failed to produce appropriate collocates when they were no longer given model sentences in the second treatment. Some of them who succeeded may have been helped by their relatively higher English proficiency. Hanaokas study (2007) lends support to the result of this present study. He found that most learners notice the lexical items but fail to notice their concomitant collocates. Consequently, when they write on their own, they are able to use the words but with inappropriate collocates. This is apparently the case with some of the sentences produced by the subjects below: He tries to sustain his teacher that he didnt cheat. She is not good at her conduct, so we have to reject it We introduce our culture to foreign people in order to sustain Indonesian culture. Hanaokas study leads to the distinction between hole and gap, which is of much relevance to the present study. The hole occurs when learners feel that there is an empty slot which they cannot work out because of their lack of knowledge and skill. In contrast, the gap occurs when they notice a model sentence and realize there is a difference (hence gap) between what they can do and what the correct model shows them. This is the critical
CamTESOL 2012 Page 4

juncture at which the learners can be prompted to produce better, more appropriate output. In the context of this study, the cycle of hole and gap simply failed to take place completely, which may have accounted for the subjects inability to produce correct collocates in the second technique. In the first technique, however, the cycle was more or less complete as they had the chance for hole before being shown the model sentences, and then led to notice the gap when the models were displayed. An implication that can be immediately drawn from the above discussion is the necessity of providing learners with exposure to natural sentences that are rich with collocations. The learners should be made to notice the collocates and later to incorporate them in their mind through intensive writing practices. It follows from here an argument that is based on trait theory and new behaviorism by Chapelle (cited in Fulcher and Davidson, 2007:16). The argument maintains that the construct of collocational ability is comprised of the noticing act of model collocations. In other words, in order to interpret the result of the test as described above, one has to accept the notion of noticing that precedes the production of collocations.

It can be concluded then from the finding that one of the ways to assess learners mastery of collocates and obtain as valid result as possible is to engage them in a series of natural learning sequence that starts off from noticing and culminating in their own production of the collocates in question.

References

Davies, A., Brown, A., Edler, C., Hill, K., Lumley, T., and McNamara, T. (1999). Dictionary of language testing. Studies in Language Testing 7. Cambridge: Cambridge University Press. Fulcher, G., and Davidson, F. (2007). Language testing and assessment: an advanced resource book.. London: Routledge. Hanaoka, O. (2007). Output, noticing, and learning: an investigation into the role of spontaneous attention to form in a four-stage writing task. Language Teaching Research, vol. 11, 4, pp. 459-479.
CamTESOL 2012 Page 5

Lee, C. (2010). A study of collocation behaviors on lexical pragmatics. Asian EFL Journal, vol. 12, 4, hal 1 12. Mongkolchai, A.( 2008). A study of university students ability in using English collocations. Unpublished Master thesis. Srinakharinwirot University. Nakata, T. (n.d). English collocation learning through form-focused and meaning-focused: interaction of activity types and L2 L1 congruence. Downloaded 28 October 2011 from http://www.paaljapan.org/resources/proceedings/PAAL11/pdfs/13.pdf Rezvani, E. (2011). The effect of output requierement on the acquisition of grammatical collocations by Iranian EFL learners. Journal of Language Teaching and Research, vol. 2, no. 3, hal 674-682.

CamTESOL 2012

Page 6