Creating theories and data for test development with language testing research

Takanori Sato
Associate Professor
Center for Language Education and Research

Associate Professor Takanori Sato of the Center for Language Education and Research is a specialist in English language testing research. He discusses the purpose of his research, which involves the development and evaluation of the validity of language tests, and the Think-Aloud method used in actual studies.

I specialize in English language testing research. My research is categorized as applied linguistics, and my objectives are to conduct a variety of research useful for the development of language tests, and to verify the effectiveness and validity of these tests based on the findings of this research.

Language testing research findings are incorporated into many of the language proficiency tests conducted in Japan. For example, when creating a test to measure communicative competence in English, language testing research can present specific definitions and evaluation indicators, such as what we mean by communicative competence in English, or by what method we can determine it. Test developers then use this information as a guide when considering the format, content, and structure of the test questions.

Defining communicative competence is an important task

Given the current emphasis on the ability to connect globally, defining communicative competence has become one of the key themes in language testing research. Communication through language can be oral or written, and having good speaking and writing skills means that you have the ability to be highly regarded by the listeners and readers with whom you are communicating. So what exactly is that ability? To properly define communication competence, we must first clarify this part of the question.

I use the Think-Aloud method, where thoughts are vocalized as they occur, as a means of understanding the other person’s assessment in the communication process. In this method, an evaluator reads a text or watches a pre-recorded speech, and simultaneously reports their impressions on contextual continuity, clarity of meaning, and other points. The advantage of this method is that it allows the researcher, through a subsequent analysis of the report, to exhaustively observe emotional dynamics that would normally go unspoken.

Non-verbal communication are also part of our speaking proficiency

An interesting finding from a previous speaking study was that many of the comments positively evaluated gestures and facial expressions during conversation. This result was somewhat surprising, as I had assumed that the focus of interest in English speaking would be on pronunciation and fluency.

The results of this study vary depending on the speaker and the evaluator, but at the very least, it is clear that native-like pronunciation and fluency are not the only factors in assessing speaking ability. These findings are useful not only for test development, but also for acquiring speaking skills, and this is why I present them in my university classes.

My current research focus is on defining critical thinking skills in writing. While critical thinking skills are gaining attention as a 21st century-style skill, their specific nature and assessment methods have yet to be established. While clarifying these issues in future research, I would like to work on further developmental research, such as the definition of critical thinking skills in writing and speaking, and the differences in assessment indices.

The book I recommend

“Measuring Second Language Performance”
by T.F. McNamara, Pearson Education

I read this book while I was a master’s student. The author is a linguist whom I studied under during my studies at the University of Melbourne. I do not believe there are many books that focus so much on the non-linguistic components of communicative competence in English.

Takanori Sato

  • Associate Professor
    Center for Language Education and Research

Graduated from the Department of School Education, Faculty of Education and Human Studies, Akita University, and received his M.A. from the Graduate School of Languages and Linguistics, Sophia University, and his Ph.D. in Linguistics from the Graduate School of Languages and Linguistics, The University of Melbourne. Assistant Professor at the Center for English as a Lingua Franca, Tamagawa University, Assistant Professor at Sophia University Center for Language Education and Research, before assuming his current position in 2021.

Center for Language Education and Research

Interviewed: September 2022

Sophia University

For Others, With Others