Assessment and Evaluation module

Teaching and learning plays a big part in the field of education as education is the process of acquisition – knowledge, skills, values and beliefs. Assessments and evaluation are utilized by stakeholders to keep track of the inputs and outputs of education on each individual learner.

Many times throughout this module, we are reminded that tests are created to gauge students’ understanding in order to provide recommendations and feedback for improvement. This made me ponder over my experiences during my schooling years as I (along with my classmates) were thoroughly convinced that our teachers set the test to fail us instead of facilitate our learning. The test definitely made us feel horrible about ourselves and the subject, although it did motivate us to study harder, not because we wanted to understand the subject, but rather, to maintain our results and self-dignity.

Knowledge about the principles of language assessment has definitely expanded my views and thoughts about assessing and being assessed. I find myself thinking critically about assessing and being assessed in this module as well as other modules.

For one, is it reliable if students are given the “choice” to do an extra assignment which is said to be able to earn them extra points? Or is that merely a clever way of phrasing homework/task to students? Not only that, in the scoring guide that was given to us later on in the module, we found out that a minimum of three pieces of reflection is expected of us. However, this task is muddy as no clear explanation were given as opposed to the first and second part of the portfolio. Short pieces could defer from person to person in terms of the word count, as well as to the depth of the content of the reflections. In this sense, there is also no consistency as there is limited guidance given to students in writing the reflections. The instruction could be narrowed to ensure students’ understanding and direction in responses so as to maintain reliability.

Another thought that pops into mind is that students can be assessed in groups on their knowledge of assessment and evaluation through designing of assessment. As this might be our only exposure on assessing and evaluating students, we could be given the task to actually prepare items and tests for students. Discussions and feedback will also help us as we encounter more problems and confusions about how to go about creating test items, regardless of whether it is formative or summative assessment. 

All in all, I have thoroughly enjoyed learning about assessment and evaluation as this is an important aspect in the teaching and learning process. It has been deeply instilled in me about the objectives of a test, the underlying principles of assessment, alternative in assessment as well as expanded my knowledge about formative assessment. I had plenty of deep thoughts and reflections on my past experience of learning and how I would like my lessons to be in the future.

Dictation

In order to assess students’ listening skills in a Form 3 class of low proficiency, a full dictation text about Air Pollution will be administered.

Air Pollution

Air pollution is the mixing of any harmful substances to the atmosphere fresh air causing huge level of damages, human health disorders, reducing the quality of life, etc. Air pollution is increasing day by day because of the increasing number of industries. Such polluted air never remains at one place however spread to whole environment and affecting the lives of the people all through the world. Death rates of the human beings have been increased because of increased variety of diseases. The polluted air we breathe every moment causing lungs disorders and even lung cancer thus affecting the health of other body organs.

Air pollution is continuously damaging the whole ecological system and affecting the lives of plants and animals as well. It has reached to the critical stage and affecting the whole atmosphere by allowing more harmful radiations from the sun to the earth. Again polluted air acts as a better insulator which prevents heat to get escaped back into the space.

Objectives of the test:

1.     Students will be able to audibly recognize –s sounds in plural nouns and spell them correctly.

2.     Students will be able to convert the spelling for words that end with –y to -ies.

I will read the text to the students in a clear and audible voice for a minimum of 3 times. I will first read the text at a normal pace, with appropriate pauses at the natural break points. I will then repeat myself in a slower speed, long enough for students to write down what they have heard. I will also repeat each sequence twice while the students are writing. At the end of my dictation, I will read the text again at a normal speed for students to proof-read their writings before collecting their papers. The learner response would be that students are able to write a text based on what they heard.

I will only score students based on the accuracy of the spelling of the 17 plural nouns (including the repetitions of some words) from the text. I will ignore the minor spelling errors which are unrelated to the plurality of the words as well as punctuations. The expected positive washback is that students will pay full attention and be alert in class. Students will also be able to apply their knowledge from previous lessons about changing singular words to plural words.

Thoughts on MUET

On MUET’s weighting for each section, what is the justification for the weightings as follows?

Listening – 15%

Speaking – 15%

Reading – 40%

Writing – 30%

In comparison with TOEFL that follows the score range of 0-30 for each section but has equal weightage across all 4 sections. The IELTS too, have the component scores weighted equally.

Why is it that the Reading section has the highest weightage, by 15% if each of the components were to be of equal weightage? Is it as if the test is made to help students obtain a better score through the reading section which is entirely MCQs, in which students can make lucky guesses or eliminate distractors. This definitely allows students to score, even without really understanding the text. The difference in weightage does not entirely correspond to the description of aggregated score. For example, Band 6 deemed as a highly proficient user would have very fluent communicative ability, has highly appropriate use of language, very good understanding of language and context as well as has very high ability to function in the language. How valid is the test if a test taker obtains an average score across all 4 sections but manages to score a Band 6 just because of the higher weightage that the reading section holds?

In regards to the Speaking section, MUET is rather unique in its approach of having group interactions. Although this imitates an authentic situation of a conversation with other speakers, the assessment also covers one’s management of a discussion such as initiating, turn taking, prompting and negotiating. This aspect seems to test students’ morale and civics qualities rather than from their speaking skills and language competency.

  

Alternative in assessment

For the purpose of assessing the reading competence of students in a lower form in a typical Malaysian secondary school, I would most likely use journals. As reading is a receptive skill that is invisible and inaudible, a written response will allow me to assess their understanding of the given text. Hence, a journal which is a written record of one’s thought, feelings, reactions or ideas towards a stimulus is fitting.

There is a low emphasis on grammatical errors or language structures, allowing students to write freely and frequently, using it as a form of “thinking”.  Journals can provide comfort for students as it allows them to articulate their thoughts in words before attempting to speak. As their insecurities to pronounce the words accurately inhibits students’ confidence in speaking, writing about them is an alternative method for students to get their message across. Besides that, students who may have a vague understanding of the reading text and are unable to answer the questions accordingly are able to write freely, allowing students to express their understanding in a way that they are more comfortable in. This could raise the content validity and impact if the entries are closely connected to the curriculum goals. 

 This form of assessment is low in practicality, although it could be countered with the usage of online platforms by providing convenience for both students and teachers alike, in terms of feedback. Reliability of journals is only moderate due to the differences in students’ written response.

Journals is an effective form of student-teacher communication as it allows teachers to understand and meet students’ individual needs better. This form of assessment accounts to positive washback as it allows teacher to offer feedback to students to encourage and celebrate their achievements, provide suggestions for further improvement as well as moderate students self-expectations.

Formative Assessment

           Formative assessment is one of the components of the term assessment for learning in which the main aims is to promote students’ learning through the adaptation of students’ current understandings to facilitate their learning from that point. This form of assessment has proven to create a supportive environment where students are not afraid to make mistakes, but rather is able to acknowledge and utilize them to develop understanding. Besides that, students become active participants and are responsible for their learning by creating their own understandings. In the perspective of the learner, formative assessment encourages students by providing explicit feedback on what is needed to improve. Through assessment and feedback in the form of comments, meaningless competition for a higher grade is forgone, and instead, the opportunity to enhance students’ learning is offered. Formative assessment also encourages multiple types of collaboration; teacher-teacher, student-student as well as teacher-student collaboration. Collaboration brings about more effective teaching and learning as it allows its users to share problems, solutions and ideas through interaction with each other.

            I fully support the statement in regards to the negative effects of scores/grades in the article which mentions how comments, in turn help parents focus on the learning issues. In our society where parents are the immediate stakeholders of their children’s education, comments on aspects to work on truly allow parents to address the shortcoming of the learner rather than on a summative grade, which does nothing to one’s improvement in the subject. Hence, I cannot entirely agree that summative tests can become a positive part of learning as the results of summative tests are numerical figures, and this creates competition amongst students even if the results are used to develop students’ learning. It also depends on the objectives of the test and what the test actually tests.

As a student, I have always experienced summative assessment in that my teachers have always assumed the roles of the “information transmitter” while the students passively accepts information often times without understanding or being able to apply the knowledge. Following that, tests and examinations are given out for gate-keeping and placement purposes with no follow-up to promote learning. Therefore, the idea of formative assessment appeals to me as what I strive for as a teacher is to truly educate my students. I think it is important to encourage students to constantly improve themselves, and for this to happen, I have to play my part in facilitating the process by providing constructive feedback.

It takes teacher collaboration and the support from schools to allow the concept of formative assessment to be implemented and for it to work (i.e. not giving grades, but instead targets and comments) in our schools. I am also dubious about how increasing think time will eventually result in the same way as it did, based on a teacher’s account in the article. Especially since our students here are taught to input information and answers without critical thinking, although this is a very interesting concept that I would like to see implemented in our shy and spoon-fed culture.

Washback and Impact

Alderson and Wall’s study, Does Washback Exist? in 1993 aims to investigate the nature of washback and the condition in which it operates as well as exploring the causes of the effects. The authors raised a number of good points whereby tests can be presumed to have beneficial washback in the event that it promotes motivation for students and teachers to place more emphasis on lessons. Alternatively, tests generally cause negative washback by inducing anxiety in students and teachers. Alderson and Wall ponders upon other factors that are often overlooked by researchers especially when it concerns specific competency tests such as IELTS and TOEFL. They criticized the lack of evidence in the existing literatures as well as the authenticity and the consequential validity of the test used in the researches. Hence, more extensive research with concrete evidence has to be done before the meaning of washback can be defined accurately. For example, direct observations of classrooms will provide a much more reliable data within the empirical study rather than to rely solely on the reports or claims of what happens during the teaching and learning sessions.

Alderson and Hamp-Lyons’s study, TOEFL preparation courses: a study of washback aims to explore and understand the effect of TOEFL on the content of classroom instruction or the methods teachers use. Although there are studies that involve questionnaires and tests to students and teachers, there is a lack of observation of the teaching process. Hence, this study provides actual classroom observational data in an EFL classroom to study the effects of washback. In their study, two teachers were observed when they taught in their TOEFL preparation and their non-TOEFL preparation classes. They wanted to understand teachers’ take on teaching these classes as well as how students perceive the preparation and taking the test. The study showed that TOEFL preparation classes does indeed have an effect on what and how teachers teach – it is much more rigid and teacher-centered compared to the non-TOEFL preparation class. However, it is observed that the varying teaching styles exhibited by the two teachers does contribute to the prominent difference of the nature of the classes. There is a need for wider surveys concerning test preparation activities and other instructions as well as curricular time and class size. The authors also suggested an addition to the Washback Hypothesis in regards to the amount and type of washback.

Lee’s study in 2004 aims to provide readers a viewpoint on why MUET seems to be not meeting its purpose of increasing the level of English proficiency amongst its takers, their reading competence in particular. The impact that MUET has lies on its status, the sociopolitical situation, the teachers’ perceptions of the test and the construct of the test. Although there is the awareness on the importance of English through a new English test, the SPM 1119 as well as the introduction of teaching Science and Mathematics in English, the Malaysian culture that place emphasis on the results rather than the process defeats the purpose of MUET which intends to increase students’ English proficiency. The author also challenges the validity of the test, pondering on the skill that is actually tested, instead of what it proclaims to test in regards to the choice of assessment, MCQs for instance. There are simply too much undue expectations on MUET, in the implications it brings to students – in terms of improvement in English language proficiency and in the way that reading is taught. Hence, the construct of reading of the test has to be revised in order for MUET to impact its takers as it aims to.

            Through the experience of taking examinations as a student, I personally noticed and was taught to the test across all subjects in school. Especially during the years where my cohort has to sit for a major national examination, lessons tend to be taught strictly to the syllabus and the scoring rubrics of the subject. Perhaps it is because teachers are expected to teach and guide students to perform well in the exams. As a result of that, students’ knowledge of the subjects are limited to the topics that will be tested. Memorization is also another negative effect of washback as students are able to regurgitate facts but are unable to apply their knowledge. This has been well proven recently when a simple question of why the car gets heated up under the hot sun was posed to my class. None of us were able to apply our Physics knowledge to explain the phenomenon even though we knew the concept of electromagneticwaves and the green house effect. Negative washback is noticed when a wrong focus of the administered test emerges, that is to achieve good scores rather than to fully understand the topics. With that mindset, holistic education can only be an ideal but unachievable concept within the education field.

The existence of the TOEFL preparation class alongside regular English classes is an example of negative washback as it shows that standardized language testing holds such high importance in certain cultures and groups of people. The culture of teaching to the test emerged as teachers are expected to guide students in achieving excellent scores in the test. The purpose of the test, which is to test an individual’s language proficiency, has been ignored in the name of scoring well in it. Hence, it is not abnormal for a person to have scored well in TOEFL but has low language competency and is unable to use English outside the examination and/or classroom context. However, when it comes to language acquisition, isn’t the application (to be able to converse) in the target language the main objective of learning a new language?

            The understanding of the concepts of washback and impact is important as a teacher because it creates awareness for us in the future as we design assessments. With that knowledge, only then can steps be taken to overcome and prevent the problem. In relation to our education system today, washback and the social impact of assessment will indeed exist. However, teachers can attempt to manage the negative effects of washback through thorough self-evaluation of conducting lessons and by avoiding teaching to the test. Besides that, teachers can opt to assess students by administering more formative assessments which aims to help students continue their learning process. In a nutshell, the objectives of a test should be sufficient in guiding a teacher’s consideration to the potential negative and positive washback as well as ways to manage them.

1.     Alderson, J.C. and Wall, D. 1993. Does washback exist? Applied Linguistics Vol 14:115-129

2.     Alderson, J.C. and Hamp-Lyons,L. 1996. TOEFL preparation courses: a study of washback. Language Testing 13 (3):280-297

3.     Lee, K.S. 2004. Exploring the connection between the testing of reading and literacy: the case of the MUET. GEMA online Journal of Language Studies 4(1)