e-ΚΠγ

The effect of text and reader variables on reading comprehension: the case of the Greek State Certificate of English Language Proficiency Exams (KPG) - A New Text Difficulty Index for Automatic Text Classification

Jenny Liontou
Faculty of English Language and Literature
School of Philosophy National and Kapodistrian University of Athens Greece

Abstract
The aim of the present research has been twofold: a) to delineate, based on a specific theory of language, a range of linguistic features that characterize the reading texts used at the B2 (Independent User) and C1 (Proficient User) level of the KPG language exams in English in order to better define text difficulty per level of competence, and b) to examine whether specific reader variables influence KPG test-takers perceptions of reading comprehension difficulty. In other words, an attempt has been made to find the relationship between these predictor variables and the readability level of English texts included in the reading test papers of the KPG examinations. The ultimate purpose of such research has been to provide a Text Classification Profile per level of competence and create a formula for automatically estimating text difficulty and assigning levels to texts, consistently and reliably, in accordance with the purposes of the exam and the KPG candidature special characteristics.
The main outcomes of the research are a) the Text Classification Profile that includes the qualitative and quantitative description of linguistic characteristics pertinent in B2 and C1 reading texts and b) the L.A.S.T. Text Difficulty Index that makes possible the automatic classification of B2 and C1 English reading texts, based on four in-depth linguistic features, i.e. lexical density, syntactic structure similarity, tokens per word family and academic vocabulary. Given that the predictive accuracy of the formula has reached 95% on a new set of reading tests, it seems safe to argue that the practical usefulness of the proposed index could extend to EFL testers and materials writers, who are in constant need of texts calibrated to specific levels of language competence. Finally, the comparative analyses of 188,556 KPG test-takers' exam scores and 7,500 KPG questionnaires made possible firstly the identification of specific textual features that can affect test-takers' performance, and secondly the detection of variables which have an important impact on readers' perceptions of text difficulty –variables that the present study suggests need to be taken into account during the test development and validation process.

<Πίσω>

Task dependent interlinguistic mediation performance as translanguaging practice: The use to KPG data for an emperically based study

Maria Stathopoulou
Faculty of English Language and Literature
School of Philosophy, National and Kapodistrian University of Athens

Abstract
This research is concerned with interlingual mediation, which involves relaying in one language information purposefully extracted from a source text in another language so as to restore communication gaps between interlocutors. Strategies for effective interlingual performance are essential in today's multilingual contexts, and the ability to mediate across languages is considered important by the influential Common European Framework of Reference for Languages (CEFR). However, due to the monolingual paradigm still prevalent in mainstream foreign language teaching and testing, mediation has not received the attention it deserves.
Retrieving data from the KPG Task Repository and the KPG English Corpus, both of which have been compiled with information (tasks and scripts) from the Greek foreign language national exams known as the KPG exams –the only proficiency testing system which tests mediation performance, this project aims at exploring the complex nature of interlinguistic mediation in a testing context. The focus is on written mediation in English, produced by test takers on the basis of messages in Greek texts. By viewing mediation as inextricably linked to the tasks which instigate and thus affect performance, this work attempts a systematic linguistic description of KPG mediation test tasks by level, which leads to the construction of a task taxonomy, organised in terms of task characteristics. Task- dependent performance exploration follows. It is empirically investigated through the textual analysis of scripts, with a view to finding task dependent mediation strategies used for successful communication at different levels of proficiency. The KPG task and learner data are analysed using both top-down and bottom-up methodologies. Task analysis involving the systematic description of tasks on the basis of specified categories determined by the theory of language and the theory of writing which informs KPG task design, is a top-down method. Textual analysis of mediation scripts, which leads to an understanding of how task parameters affect written mediation strategies, is a bottom up method used to empirically develop a framework for the mediation strategy descriptors.
The methodology and results of this study may both be useful in that task and mediation strategy descriptors may facilitate the development of mediation levelled tasks for teaching and testing purposes, while they may also contribute to the creation of standardised measures and clear benchmarks for a more reliable assessment of mediation competence, thus providing a critical supplement to the CEFR and to the curricula of language courses interested in developing learners' interlinguistic competence.

<Πίσω>

World representations in language exam batteries: critical discourse analysis of texts used to test reading comprehension

Amalia Balourdi
Faculty of English Language and Literature
School of Philosophy, National and Kapodistrian University of Athens

Abstract
Adopting the view that “anything said or written about the world is articulated through language from a particular ideological position” (Fowler, 1991: 10), the basic assumption upon which this thesis rests is that the texts used by English language examination systems are invested with the particular ideologies that characterise the institutions which are responsible for the exams. The basic aim of the thesis is twofold: (a) to explore the ways in which the texts used to test reading comprehension in three popular English language proficiency exam batteries in Greece (i.e. the state certificate for language proficiency, known as the KPG Exams, the Cambridge ESOL Exams, and the Michigan Exams) linguistically construe or represent the world, and the subject positions they create for their readers, and (b) to develop a systematic and accurate model of analysis that will best serve this purpose.
Understanding that “the grammar of a language is its theory of reality” (Kress & Hodge, 1993: 7), the present study, which follows the traditions of Critical Discourse Analysis and Critical Testing and adheres to the principles of Halliday’s theory of language as social semiotic, combines the methodological tools of Systemic Functional Linguistics with corpus linguistics techniques and attempts a detailed lexicogrammatical analysis of a corpus of texts. This corpus of texts consists of three sub-corpora, each containing 31 texts from each exam battery. The analysis, which focuses on the investigation of the experiential and interpersonal meanings the texts make, indicates that the texts of each examination battery sub-corpus are characterised by particular linguistic and discursive choices, by means of which three different realities are construed. The method of analysis applied in this study has helped reveal the different ideological positions from which the texts of each exam battery are articulated, thus demonstrating the effectiveness of the SFL analytical tools, particularly in combination with those of corpus linguistics, in studies with critical aims.
The importance of this study lies in that, apart from providing a systematic method of lexicogrammatical analysis, by revealing the ideological implications of English language tests viewed as powerful social institutions which administer and introduce specific knowledge, it also introduces a critical and political angle from which to view texts for teaching and testing language(s). In fact, the ultimate goal of this thesis, and consequently the gap in the current literature it hopes to fill in is to generate a critical stance to the content of foreign language testing, by clearly demonstrating that there are particular linguistic constructions of reality in the texts used for testing reading comprehension.

<Πίσω>

Μεταπτυχιακες και Διδακτορικες Διατριβες

Διδακτορικές διατριβές που έχουν ήδη εκπονηθεί για τις εξετάσεις στην Αγγλική

• World representations in language exam batteries: critical discourse analysis of texts used to test reading comprehension. (Απρίλιος 2012)
• Task dependent interlinguistic mediation performance as translanguaging practice: The use to KPG data for an emperically based study (Φεβρουάριος 2013)
• The effect of text and reader variables on reading comprehension: the case of the Greek State Certificate of English Language Proficiency Exams (KPG) - A New Text Difficulty Index for Automatic Text Classification (Φεβρουάριος 2013)

Πλήθος διδακτορικών διατριβών με ποικίλα θέματα που βρίσκονται εν εξελίξει για τις εξετάσεις στην Αγγλική. Ενδεικτικά παρατίθενται οι παρακάτω:

• Listening comprehension in language proficiency tests: a study of EFL test-takers’ strategy based performance in listening testing situations
• Investigating literacy requirements of reading and listening comprehension test tasks: the case of the KPG English and French exams
• A corpus-based research of candidates’ scripts concerning text grammar
• Exploring listening comprehension difficulty in language proficiency testing: the case of the Greek State Certificate Examination
• Interlocutor performance variability in language proficiency testing: the case of the Greek State Certificate Examinations
• Towards the validation of the KPG activities of oral performance
• Factors affecting writing and written mediation task difficulty

Μεταπτυχιακές διατριβές στις οποίες το RCeL προσέφερε ερευνητική υποστήριξη παρατίθενται παρακάτω:

• Γραπτή διαμεσολάβηση στις Εθνικές Εξετάσεις Πιστοποίησης Γλωσσομάθειας: διερεύνηση γλωσσικών υβριδικών σχηματισμών λόγω της κανονιστικής επενέργειας του αρχικού κειμένου (2009).
• Αξιολογώντας την πραγματολογική ικανότητα των υποψηφίων του ΚΠΓ στις δραστηριότητες παραγωγής γραπτού λόγου: σχεδιάζοντας σκάλες αξιολόγησης συγκεκριμένων δραστηριοτήτων (2009).
• Σύνδεση του Κρατικού Πιστοποιητικού Γλωσσομάθειας (ΚΠΓ) με το Ολοήμερο Σχολείο: ένα προτεινόμενο πρόγραμμα διδασκαλίας για την ανάπτυξη στρατηγικών αντιμετώπισης της εξέτασης Επιπέδου Α' (2009).
• Investigating aspects of validity in mediation tasks: a case study from the Greek State Certificate of Language Proficiency.
• Relating language tests to the Common European Framework: a case study of a reading comprehension test.

Έρευνα για το ΚΠΓ

• Τo «προφίλ» του Έλληνα μαθητή των ξένων γλωσσών