Journal of Test Construction 1

Short-Term Memory Test:

An Assessment Method for Future Counselors
Marlin Rusli, Edwin Sutamto, Lena Febriana, and Alice Ade Faculty of Psychology Undergraduate Program Atma Jaya Catholic University Jakarta, Indonesia

Abstract This paper discusses about a test construction based on the construct of short-term memory. Specifically, this test was constructed as a reference to a university for selecting the student candidates in the Guidance and Counseling Program from the Teacher Training and Education Faculty. The construction of the test comes up from the awareness of the lacking basic skills the students have. This situation may cause the low performance of the students in counseling practice. Its not the matter of how well they speak, but mainly on how they receive their future clients information and use it as a treatment reference. Introduction Nowadays, counselor has been a popular and promising profession for some people. The reason why is life challenges has become harder each day. Those problems are not just common in adults life, which most people think as a cause of their work problems, family, and even love life. Life problems can occur in childrens daily life. Most schools in the large cities have their own school counselors. Usually they are teachers who took counseling or psychology major as a focus on their further studies. A study in one of Jakartas leading Catholic University shows that school counselors are created from its counseling major. Subjects that refer to the skills of the future counselors are

basic counseling technic, approaches in counseling, counseling on social and individual aspects, and a lot more. Most of the subject requires field activities. Students are challenge to learn not only from textbooks, but also in real life practices. Carl Rogers theory talks about the basic skills that are needed as a good counselor, empathy, congruence, and unconditional positive regard. As for the Head of Guidance and Counseling Program from the Teacher Training and Education Faculty, Yoseph Pedhu, CP, said that its not only the matter of theory but also the real life practice itself. Students must have good listening and memory skills. Short-term memory (primary memory) is the capacity to hold information in the brain in a short time and under conditions that can be directly used (Miller, 1956). Short term memory stores information for roughly 18-30 seconds, although the information can be stored longer if a conscious effort is made to keep it there by rehearsing it. According to several experiments conducted by Miller (1956), the capacity of shortterm memory itself is limited to about seven units, plus or minus two. Each unit can be defined differently, depending on the types of input and also whether the listener has any previous knowledge on the particular type of input or not. Counselors way of doing their job is by listening to their clients stories, whatever it will be, and try to lead their client onto a solution. A study about listening comprehension shows that short-term memory in auditory is the most important component in listening comprehension. Therefore, counselors who mainly do their job by listening to peoples stories for the purpose of designing a treatment that suits the clients need to have a good auditory short-term memory. An auditory short-term memory or also called verbal memory functions to assist individuals in understanding a certain language. Individuals use the memory to store, process, understand, and assimilate a conversation (Alton, 2001). Auditory short-term memory is associated with speed and articulation of words. The main function of auditory short-term memory is important when individuals are listening. The Short-Term Memory Test is an ability/aptitude test that is made as tool for early prognosis on future counselors. Implying from the definition of short-term memory, there are three domains that can be break down: the sum of number elements in mind, the sum of letter elements in mind, and the number of word elements in mind. The design of Short-Term Memory Test is to consist of 21 made from numbers, letters, and word that have the range of difficulty from 3 to 9. These items are given in a form of an audio recording and to be tested individually. The main purpose of this test is to measure one of the important components in counseling, which is memory skill. This tests outcome will be a score on the students level of

Journal of Test Construction 3 auditory short-term memory. As said before that the auditory short-term memory play an important role in listening comprehension, counselors should have this skill as their basic foundation. The score from this test will act as a reference for the university in accepting the new admissions. Method Attempting Try Out Implementing from the definitions of short-term memory, we develop a set of numbers, letters, and words that are used as stimulus to measure the subjects short-term memory. Each element varies in the number of items in it. These items also act as the difficulty level of the Short-Term Memory Test. This tests format is a recording that contains the items spoken in Bahasa Indonesia. The sequence of the items was made from the lowest range (which is 3) to the highest (which is 9) in an order from the numbers, letters, and the words element. We conducted an individual try out to sort out which items that are acceptable for this test. Participants of the try out were 72 students from different majors in the university. For the try out, we prepared 63 items, which consist of 21 items formed in numbers, 21 items formed in letters, and 21 items formed in words. Each participant was given a headset that is attached to an audio player device. Participants were to re-write the items they heard from the recording in the right order as given. Scoring is based on the correct answers written by the participants. The value for each correct answer is given a score of 1 and 0 for every incorrect answer. Results Item Analysis The function of the try out is to decide which of the 63 items that will qualify in the Short-Term Memory Test. In the item analysis, we used item difficulty and item discrimination as the parameter of the qualified items. The item difficulty of each item should be in the range of 0.30 p 0.80 or else, it must be eliminated. If an item has a proportion below 0.30, it indicates that the item is very difficult. And also if the proportion of the item is above 0.80, then it is very easy. Point-biserial correlation was used to examine the item discrimination. The pointbiserial shows the correlation between each items score and the total score. The higher it correlates, it shows that the item manage to discriminate the participants who have high short-

term memory and who dont. Each items point-biserial correlation value should be higher than 0.20 to be qualified. Based on the analysis of the items that have been carried out, there are 23 out of 63 items that were accepted and can be used. From the calculation of item difficulty, there are 27 items that qualify. Items that qualify are items that have a moderate level of difficulty (0.30 p 0.80). The rest, 11 items that are rejected for being too difficult (p <0.3) and 25 items were rejected for being too easy (p> 0.8). In the point-biserial calculation, there are 32 items accepted and 31 rejected. Items that are denied have point-biserial correlation value which is lower than 0.2. It means that these items do not represent the measurements of short-term memory. As a result, these items couldnt discriminate participants who have high or low short-term memory capabilities. Validity The validation methods used for this test are construct validation and criterion prediction validation. Construct validation method is used in order to see whether this test has fulfilled its function to measure the construct in the form of short-term memory theoretically. While the criterion prediction validation is used to see whether this test is able to fulfill its function to predict individual performance in the practice of counseling in the future. For the construct validation, we used internal consistency as the internal validation of this test. We determined the limits of internal consistency by using the limits used in the calculation of item-total correlation, which is greater than 0.2 (Field, 2013). Another method used for computing the construct validation is convergent-discriminant method. The convergent test used by the group is one of the sub-tests in Wechsler-Bellevue test: the digit span. Meanwhile the discriminant test used is the sub-test also from Wechsler-Bellevue test: the information processing. When the correlation with convergent method has a value of 0.50.7, then this test can be said to be valid. Meanwhile, when the correlation with discriminant method has a negative value then this test can be said to be valid (Gronlund, 1982).

Journal of Test Construction 5 Table 1 Construct Validation

Domain Item No. 1 4 The sum of number elements in mind 7 10 13 16 19 2 5 The sum of letter elements in mind 8 11 14 17 20 3 6 The sum of word elements in mind 9 12 15 18 21 Pearson STMT - Digit Span 0.521 Domain Correlation 0.000 0.000 0.309 0.440 0.772 0.673 0.567 0.403 0.417 0.398 0.447 0.476 0.489 0.113 0.000 0.239 0.346 0.38 0.456 0.368 0.468 Pearson STMT Information Processing 0.371 Construct Correlation 0.000 0.000 0.468 0.595 0.786 0.706 0.528 0.380 0.218 0.339 0.478 0.772 0.518 0.244 0.000 0.113 0.645 0.356 0.444 0.212 0.243 Internal Consistency 0.383

The calculation result of discriminant validity, the total scores of STMT are correlated with the information processing sub-test of Wechsler-Bellevue test is equals to 0.371. Meanwhile the results of the convergent validity, which is by correlating the total score of STMT with digit span sub-test of Wechsler-Bellevue test is equals to 0.521. Aside from that, the calculation result for the internal consistency is equals to 0.383. Referring to the restrictions provided by Gronlund, the Short-Term Memory Test is not valid by the discriminant validity but said to be valid by the convergent validity and internal consistency.

Based on calculations performed using SPSS, the correlation value obtained for the criterion prediction validity is equals to 0.322. The score indicates a significant relationship between the Short-Term Memory Test and the Counseling subject scores in the 3rd semester students. So it can be concluded that the Short-Term Memory Test can be used to predict students performance in the future. Reliability The type of reliability testing used for the Short-Term Memory Test is one-time administrative procedures by the method of Kuder Richardson 20. This method is used because of the items score in this test is dichotomous and it consists of various degrees of difficulty on each item. Table 2 Reliability Testing

Method KR-20

Reliability Coefficient 0.827

Error 0.173

The results obtained when using the method of KR-20 is equals to 0.827. Due to the limit of the stated value on a reliable measuring instrument is 0.75, the Short-Term Memory Test can be said to be reliable for measuring the short-term memory of the Guidance and Counseling Program from the Teacher Training and Education Department student candidates. Setting Norms The standardization technique used for this test is setting standard by using norm. We used the norms of the group to see the position of individuals within the group itself. We computed the normality test to figure out which method will be used to define the norm. Then, computing a standard score (z-score) of the raw data will be required when the score of the distributions is generated in normal form.

Journal of Test Construction 7 Table 3 Normality Testing

One-Sample Kolmogorov-Smirnov Test TOTAL N Normal Parametersa,b Mean Std. Deviation Absolute Most Extreme Differences Kolmogorov-Smirnov Z Asymp. Sig. (2-tailed) a. Test distribution is Normal. b. Calculated from data. Positive Negative 42 11,60 3,683 ,132 ,096 -,132 ,855 ,458

Chart 1 Raw Score Distribution

Table 4 Z-score and T-scale Calculation

Z-Score -2,06229 -1,79076 -1,51924 -1,51924 -1,24772 -1,24772 -1,24772 -,97619 -,97619 -,70467 -,70467 -,70467 -,70467 -,43315 T-Scale 29,38 32,09 34,81 34,81 37,52 37,52 37,52 40,24 40,24 42,95 42,95 42,95 42,95 45,67 Z-Score -,43315 -,43315 -,43315 -,43315 -,16162 -,16162 -,16162 ,10990 ,10990 ,10990 ,38143 ,38143 ,65295 ,65295 T-Scale 45,67 45,67 45,67 45,67 48,38 48,38 48,38 51,10 51,10 51,10 53,81 53,81 56,53 56,53 Z-Score ,65295 ,92447 ,92447 ,92447 ,92447 1,19600 1,19600 1,19600 1,19600 1,19600 1,19600 1,19600 1,46752 1,46752 T-Scale 56,53 59,24 59,24 59,24 59,24 61,96 61,96 61,96 61,96 61,96 61,96 61,96 64,68 64,68

The interpretation for this test is divided into two categories, which are pass and fail. This was made based on the purpose of the Short-Term Memory Test, as a reference on the qualified student candidates. This categorization was done by using a cut-off score of the digit span subtest, which are <7. Transforming the raw scores of the group total score into the T-scale, of the digit span subtest taken during field. From the calculation of the scores from the digit span sub-test to the t-scale, the cut-off score for the Short-Term Memory Test is 35.95. The score 6 on test scores has a smaller SS and the approaching 35.96, which is 34.81. Therefore, we set the cut-off score <7 as a break-out limit. Table 5 Standardization Table from the Participants Score
RS 4 5 6 7 8 9 10 11 SS 29,38 32,09 34,81 37,52 40,24 42,95 45,67 48,38 Category Fail Fail Fail Pass Pass Pass Pass Pass

Journal of Test Construction 9

RS 12 13 14 15 16 17

SS 51,10 53,81 56,53 59.24 61,96 64,68

Category Pass Pass Pass Pass Pass Pass

Discussion Constructing the Short-Term Memory Test has its own challenges. Aside from its various domains, the STMT lacks on objectivity in its items. For the word element items, there was no previous assessment or studied indicating, which words can represent the main construct of the test associating with the targeted participants. The words used as the items of this test may have certain meanings or association for the participants. This may be the cause of the discriminant validity to be said not valid. The subjective stimulus may cause the ShortTerm Memory Test to have an overlap to other test that measures different construct. Selection for the discriminant validity test should also be given more attention. The wrong test may cause the not valid result due to the overlapping main construct. Aside from that, participants that were used for the try out and field have different criteria. This may affect the qualified items is really not that appropriate for this test. Another factor that may cause defect to the try out and field is the place where the tests are held. This should be more carefully controlled when conducting a test. Furthermore, in constructing a test, researchers should pay more attention to every details of the process. A single small mistake may cause invalid and unreliable test, which means a waste of energy and time for nothing.

Reference Alton, S. (2001). Short-term Auditory Memory. Accessed at 21 Juni 2013 from downsyndromensw.org.au/data/Short_term_auditory_memory_-_Alton_UK.pdf Anastasi, A. & Urbina, S. (1997). Psychological testing (ed. 7). Upper Saddle River, NJ: Prentice Hall. Bhinnety, M. (2008). Struktur dan Proses Memori. Buletin Psikologi, vol 16 (2), page 74-88 Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Belmont, CA: Wadsworth Group/Thomson Learning. Gronlund, N. E. (1982). Constructing Achievement Tests. New Jersey: Prentice Hall Miller, G. A. (1956). Human memory and the storage of information. IRE Trans. Inform Theory, page 129-137. Miller, G. A. (1956). The magical number seven plus or minus two: some limitations or our capacity for processing information. Psychological Review, 63(2): 81-97. Ohata, K. (2006). Auditory Short-Term Memory in L2 Listening Comprehension Processes. Journal of Language and Learning, vol. 5 (1), page 21-28 Santrock, J. W. (2011). Life-Span Development. 13th ed. United States: McGraw-Hill. Shiffrin, R. & Nosofsky, R. (1984). Seven plus or minus two: A commentary on capacity limitations. Psychological Review, vol 2 (101), page 357-361 Wade, C. & Tavris, C. (2011). Psychology. 10th ed. United States: Pearson.