Predicting EFL writing ability from levels of mental representation measured by Coh-Metrix: A structural equation modeling study

This study investigates the interrelations between levels of mental representation and text quality. 163 Chinese EFL learners wrote sample expository and persuasive essays that were marked by four trained raters using a writing scale comprising Word Choice, Ideas, Organization, Voice, Conventions, and Sentence Fluency traits. The psychometric validity of the writing scores was investigated using many-facet Rasch measurement. Based on the construction-integration (CI) model of comprehension, three levels of mental representation were delineated for the essays: the surface level (lexicon and syntax), the textbase, and the situation model. Multiple proxies for each level were created using Coh-Metrix, a computational tool measuring various textual features. Using structural equation modeling (SEM), the interactions between the three levels of representation, text quality, and tasks were investigated. The SEM with the optimal fit comprised 23 observed Coh-Metrix variables measuring various latent variables. The results show that tasks affected the situation model and several surface level latent variables. Multiple interactions were identified between writing quality and levels of representation, such as the Syntactic Complexity latent variable predicting the situation model and the situation model latent variable predicting Conventions and Organization. Implications for assessment are discussed.