One of the most engaging and enlightening sessions at the recent New Directions Conference in Yokohama was the panel discussion about the issue of language assessment policy. The conference was held shortly after the decision to postpone the introduction of four-skills tests to University Entrance Exams in Japan had been announced. In a break from the more formal style of prepared mini-presentations, Professor Barry O’Sullivan chaired a lively discussion in which panellists responded to questions from the chair, the audience, and points made by each other. This discussion captured the spirit of the conference, and brought some of the key issues into focus.
The influence of testing on learning
One of the main aims of introducing four-skills testing is to drive educational reform, but Professor Yoshinori Watanabe from Sophia University warned that we should be wary of over-reliance on testing for this purpose. He noted that high stakes tests have to be practical, fair and very reliable, which may affect the scope of what can be tested, so there are limits to the potential positive influence of high-stakes tests. Professor Mitsuhara Ota of Nagoya University, on the other hand, felt that some existing tests were too focused on rote-learning, testing patience and endurance more than English ability, so the move to four-skills testing is necessary to monitor progress in students’ ability to use English to communicate.
The British Council’s Professor Barry O’Sullivan noted that one reason why the attempted reform in 1994 had little impact was that the test didn’t change. So, while the test alone may not be enough to change the system, if the test doesn’t change then the system will not change. Assessment reform is therefore necessary but not sufficient to drive educational reform.
What other factors are important?
Michael Connolly, the British Council’s global Head of English for Education Systems (EES) noted that successful programmes tend to be at the state or municipal level rather than national level. He gave the example of bilingual education in Madrid. One reason for its success is a whole system approach, looking at curriculum and teacher development alongside assessment. In addition, this programme includes research, monitoring and evaluation of what is happening. Professor O’Sullivan noted that this requires policy-makers to be brave, which is perhaps why it works better in smaller contexts.
Professor Joseph Lo Bianco from the University of Melbourne also said that education often has too little focus on students’ motivation to learn. Students need Capacity, Opportunity, and Desire (COD) to succeed. Desire is the least practised element, but it might make the biggest difference.
Professor Lo Bianco encouraged everyone to think of policy-making as a conversation between policy-makers, experts, and communities. Policy literacy involves understanding how everyone can be actively involved in shaping policy, because these policy conversations work best when everyone’s voice is heard. The concept of assessment literacy is well-established, but policy-literacy was a new idea to most people at the conference. Assessment experts often wish that policy makers understood assessment better, but Professor Lo Bianco challenged them to understand policy making better if they want to influence policy successfully.
Huu Nguyen, director of the National Foreign Languages Project in Vietnam, outlined an example of a policy conversation in Vietnam, where a new curriculum was launched in 2018. Officials identified a problem and commissioned a baseline study to find out the current situation, then invited experts to analyse the data and propose possible policy solutions. Policies were then piloted and communities asked to comment on them so that issues can be addressed. By the time the new policy was implemented, everyone has had an opportunity to contribute to the conversation, and it is a neat example of how both top-down and bottom-up processes can feed into decision-making.
Professor O’Sullivan concluded the discussion by highlighting three themes that had emerged: context, communication, and time. “If we don’t take our time and do things properly, if we don’t communicate with people, and if we don’t take the context into account, nothing is going to work.”