The implementation of four-skill English examinations for university admission purposes is expected to lead to positive learning outcomes. At the same time, ensuring the reliability of the marking of these tests is essential for fairness considerations. We asked Dr Fumiyo Nakatsuhara, Reader at the Centre for Research in English Language Learning and Assessment (CRELLA) at the University of Bedfordshire, and Chair of the British Association for Applied Linguistics: Testing, Evaluation and Assessment Special Interest Group, about specific measures for that challenge.
- What positive washback* do you expect the implementation of the four skills in university entrance exams will have?
“On page 56 of Arthur Hughes' 2003 book Testing for Language Teachers, he states:
“Before we decide that we cannot afford to test in a way that will promote beneficial backwash, we have to ask ourselves a question: What will be the cost of not achieving beneficial backwash? When we compare the cost of the test with the waste of effort and time on the part of teachers and students in activities quite inappropriate to their true learning goals, we are likely to decide that we cannot afford not to introduce a test with a powerful beneficial backwash effect.”
I came across this book about 20 years ago. At that time, even though I had performed well in secondary school exams, I didn’t have the practical communication skills that the Course of Study aimed for. Realising the importance of having testing that could support such educational goals, I decided to become a researcher in language testing.
While research has shown that the introduction of adequate tests does not necessarily lead directly to positive washback, it is certainly possible to expect some amount of positive washback from the implementation of tests that include all four skills. The willingness and time that high school students devote to learning to speak and write for university entrance exams, and the effort and time that teachers devote to teaching these skills, would certainly increase. Appropriate teacher training and support are the key to improving students’ ability to speak and write in English.”
*Washback (or backwash) refers to the impact of testing and assessment ‘structure’ on curriculum design, teaching practices, and learning behaviours.
- What is most important when ensuring reliable testing of the four skills?
“With regards to speaking and writing tests, we believe the following are important:
Establishing assessment criteria based on evidence from linguistic analysis of target learners
Regular training and certification testing for examiners
Having individual assessment systems with two examiners for each test (if there is a large difference in scores, a third examiner will make the final decision)
Adjusting results using statistics from a Many-Facet Rasch Model analysis (this may not be practical for larger trials)
Monitoring the performance of the examiners (if not reliable the examiner stops until he/she retakes the training and certification exam)
Creating a system that allows for re-scoring and re-assessment upon request.”
- What role do you think tests that assess English language proficiency, such as IELTS, play for English learners in Japan?
“In order to accurately measure their ability to speak and write in English, it is important to, as much as possible, have them take the test in the same environment and conditions as the situation in which the skill will be used. It is also essential that the test-taker’s skill level and assessment criteria match.
For learners who want to improve their English skills, a test like IELTS is a tool to find out how close they are to their goal and what their weaknesses are. IELTS is also a prerequisite to study at universities in English-speaking countries such as the UK and Australia, so their future possibilities may expand depending on their test results.”