Developing a useful measure of French-English cognate awareness

Presentation on theme: "Developing a useful measure of French-English cognate awareness"— Presentation transcript:

1 Developing a useful measure of French-English cognate awareness
Marlise Horst – Concordia University Tom Cobb – Université du Québec à Montréal Joanna White - Concordia University AAAL, March 2013 Joanna begins

2 Acknowledgements We thank our research associate
Randall Halter And our graduate and undergraduate research assistants, students at Concordia University and UQAM Juliane Martini Tayebeh Shalmani Victoria Dwight Jessica Bate David Bertrand Andrew Chevrier Karine Valliquette We are also grateful to ESL teachers and students in Montreal area secondary schools Social Sciences and Humanities Research Council of Canada

3 Overview of the presentation
Context of the research A new measure of French-English cognate awareness Findings Do Quebec ESL learners recognize helpful cognates? How does cognate density affect reading comprehension? A new measure of cognate density Conclusions

4 Background Why do we need a test of cognate awareness?
Awareness of L1-L2 cognates can offer learners a huge reading comprehension advantage There are thousands of helpful French-English cognates But we don’t know the extent to which Quebec secondary learners of English can Recognize cognates Deploy cognate knowledge in understanding texts

5 Background Our study of primary ESL learners shows that cognate instruction increases cognate awareness (White & Horst, 2012) A test might also show us whether instruction might benefit secondary learners But no cognate awareness measure for French- English cognates exists… To our knowledge

6 Developing a cognate test
Existing measures of cognate awareness? Cognate Awareness Test for Spanish-speaking learners of English August, Kenyon, Malabonga, Longuit, Caglarcan & Carlo, 2001 We looked for an alternative to their multiple-choice format to avoid Scope for guessing Answers requiring knowing English Part 1 of our test has 40 English words to translate directly into French 16 cognates, 16 non-cognates, 8 ‘easy’ words Marlise here

7 Part 1. Translate the underlined word into French
Part 1. Translate the underlined word into French. If you don’t know the word, check “I don’t know.” 1. That noise is loud. …………………………………………………. ___ I don’t know. 2. What is the consequence? …………………………………………………. ___ I don’t know. 3. We condemn them. …………………………………………………. ___ I don’t know. 4. We have a new president. …………………………………………………. ___ I don’t know. 5. We have the address. …………………………………………………. ___ I don’t know. 6. He can heal you. …………………………………………………. ___ I don’t know. 7. What is the origin of this? …………………………………………………. ___ I don’t know. 8. There was lightning. …………………………………………………. ___ I don’t know. Etc.

8 The 16 cognates and 16 non-cognates are comparable
As much as possible, each cognate item has a comparable non-cognate counterpart Same part of speech Same rough frequency in English according to frequency lists based on the BNC (Nation, 2006)

9 Test-takers also deploy cognate awareness
Part 2 of the test: reading comprehension task with guessing unknown words from context Task: Translate into French A cognate-rich text A cognate-impoverished text Each text contains 3 infrequent, non-cognate words (underlined) COGNATE HIGH The white colour of our apartment was totally uninteresting to me. My mom permitted me to paint it the hue I most preferred. I chose orange. But while I was painting the ceiling I suddenly lost my balance and fell. As a result, I created a giant orange scribble on the white wall. I thought the new decoration was fantastic but my mom loathed it. COGNATE LOW I thought my plain white bedroom walls looked boring. My mom said she would let me paint them any hue I liked. So I picked orange. But while I was painting the ceiling , I started to fall off the ladder. Paint splashed everywhere and I made a huge orange scribble on the white wall. I thought it looked rather lovely but my mom loathed it.

10 PAINTING -- COGNATE HIGH
The white colour of our apartment was totally uninteresting to me. My mom permitted me to paint it the hue I most preferred. I chose orange. But while I was painting the ceiling I suddenly lost my balance and fell. As a result, I created a giant orange scribble on the white wall. I thought the new decoration was fantastic but my mom loathed it. PAINTING -- COGNATE LOW I thought my plain white bedroom walls looked boring. My mom said she would let me paint them any hue I liked. So I picked orange. But while I was painting the ceiling , I started to fall off the ladder. Paint splashed everywhere and I made a huge orange scribble on the white wall. I though it looked rather lovely but my mom loathed it.

11 Texts were counterbalanced for topic
Version A Version B Painting (Cognate High) Gift (Cognate Low) (Cognate Low )

12 GIFT -- COGNATE HIGH I have a favourite aunt who adores me. When I finished secondary school, she presented me with a diamond bracelet. It was a valuable family treasure and my siblings were so jealous of me. The next day I was walking in a meadow, and it fell off my arm. I searched for many hours but the heirloom had disappeared. I am still inconsolable. GIFT -- COGNATE LOW I have an aunt who is close to me. When I finished high school she gave me a gold ring. It was an heirloom and worth a lot. My siblings had always loved it dearly. The next day I was walking in a meadow, and it fell off my hand. I looked for the ring for a long time but I never found it. I still feel the loss. B version had gift low

13 Testing the test Participants
343 secondary ESL students in French medium schools in the Montreal area 14 classes 5 levels: Secondary 1, 2, 3, 4, 5 Age ±13-18

14 Results of the cognate testing
Do Quebec ESL learners recognize helpful cognates?

15 Part 1 40 items to translate random mixture of 16 cognates
16 non-cognates 8 ‘easies’

16 Scoring Answers were rated right (1 point) or wrong (0 points)
Part 1. Translate the underlined word into French. If you don’t know the word, check “I don’t know.” 1. That noise is loud. …………………………………………… ___ I don’t know. 2. What is the consequence? …………………………………………… ___ I don’t know. 3. We condemn them. …………………………………………… ___ I don’t know. 4. We have a new president. …………………………………………… ___ I don’t know. 5. We have the address. …………………………………………… ___ I don’t know. 6. He can heal you. …………………………………………… ___ I don’t know. 7. What is the origin of this? …………………………………………… ___ I don’t know. 8. There was lightning. …………………………………………… ___ I don’t know. 9. He has never met her. …………………………………………… ___ I don’t know. 10. I saw his face. …………………………………………… ___ I don’t know. 11. The tub is over there. …………………………………………… ___ I don’t know. 12. He started to slump. …………………………………………… ___ I don’t know. 13. It is very spicy. …………………………………………… ___ I don’t know Etc. In non-informative context sentences. Guessing was encouraged Answers were rated right (1 point) or wrong (0 points) In consultation with a fluent bilingual, a native speaker of Quebec French “I don’t know” = 0 points

17 Results Means: 16 non-cognates vs. 16 cognates (N = 343) 75% 25%
Findings look too neat but they’re real SD in the cognates is lower (2.00) than for non-cognates (2.6) 25%

18 Results Means: 16 non-cognates vs. 16 cognates by year
SD’s are always much smaller in cognates – there’s more consistency Pattern of increase is what you’d expect Right. This looks good.

19 Results Means by word – 16 Cognates Correct = 1 Incorrect = 0
Maximum possible score = 1 Note drop-off at sponge

20 What do we see? Cognates with exact or close resemblance to French are easily recognized consequence, liberty, origin Recognition decreases with school/école pattern sponge, spicy, slave, strangled, stun, spouse Scope for awareness raising instruction here

21 Results Means by word – 16 Non-cognates

22 What do we see? The most known non-cognate is flight
A common experience for young Quebeckers? Some of the least known ones were confused with sound-alikes rod / road wealthy / healthy Students didn’t break up widespread Scope for raising awareness of compounding?

23 Results Means by word – 8 easy (1k) words Correct = 1 Incorrect = 0

24 Results of the cognate testing
How does cognate density affect reading comprehension?

25 Part 2 2 texts to translate One cognate high, one cognate low
One about painting, one about a gift Counterbalanced (Versions A & B)

26 Rating the translated texts
All translations were rated twice by two French-English bilinguals Disagreements were discussed, resolved Scoring 3 = perfect, near perfect with sensible guesses 2 = comprehensible but avoidance of 2 or 3 targets 1 = minimal, partially complete 0 = not attempted

27 Sample translation rated 3

28 Results Means - text translations (max score =3)
Difference in the expected direction on Test B only. Sd is lower in both cognate-highs: so performance was more consistent

29 Summary Performance on single-word cognates (Part 1) was substantially better than on comparable non-cognates Performance on cognates (12/16) allows scope for improvement (via classroom awareness raising) But comprehension of a cognate-rich text (Part 2) was facilitated by cognates in version B only In fact, the cognate-rich texts are harder than the cognate-low ones

30 Why was the expected advantage for cognate- rich texts not (consistently) found?

31 To consider… Q: What makes an English text easier to read?
Lots of French-English cognates – usually less frequent English words? I have a favourite aunt who adores me. OR Lots of very frequent English words – often Anglo- Saxon non-cognates? I have a dear aunt who is close to me. We need a way of assessing texts that takes both cognate richness and frequency into account (Easier for a francophone, I mean)

32 Ideal for this experiment would have been
Cognates: Text X > Text Y Frequent words: Text X = Text Y Q: Is there a simple, reliable way to judge texts for these two factors? One of the goals of this research is to develop a computer-based text assessment tool along the way Like Lextutor’s «Vocabprofile » What we have got so far…  Tom here

33 We know that more frequent words = more comprehensibility -- but frequency does only part of the job

34 Two spoken texts, both at K1>90%
A child speaking about baseball Adults speaking about writing assessment I like baseball, it is fun, when I play it I hit homeruns. I like baseball because you, you get to get people out. That's all, baseball is easy what you do is you, you, if there is a fly ball and you're running on the base you go halfway and if they catch it you are out and if they drop it you are not out, you just keep running. If you do an overthrow you only can go one base. If you were playing out in the outfield you must pay attention so the ball won't hit you in the head. DEMOS lextutor.ca/vp/bnc/ I can't recall when, but it was for the writing assessment. And one of the concerns at that point is what happens if students have more time? And so in that assessment, it was a comparison of 20 minutes and 15 minutes to see if in fact the additional time made a difference. I'm mentioning this here not because it's writing, but people might have been concerned about how much time was given to the students to read and then write the extended passage.

35 Speech (both kids and adults) is almost entirely high frequency (1k) words

36 List carve-up ( 1k) SO WHAT’S THE DIFFERENCE? The kid and adult speech both use lots of frequent words BUT the adult uses more cognates (GL items). Frequency profiles alone don’t capture this. We need a cognate /non-cognate carve-up of the frequency lists

37 Building a new cognateness-of-texts measure  VPCOGNATES
All families on BNC frequency lists 1k-10k were judged as either GL: More “Greco-Latin” (cognate) AS: More “Angl0-Saxon” (non-cognate) Easy cases English video / French vidéo = cognate English horse / French cheval = non-cognate But how to classify actual /actuel ? English actual = real French actuel = 1. current, ongoing 2. real Added by M: There was a step missing

38 Building a new cognateness-of-texts measure  VPCOGNATES
Many previous attempts to harness the reality of cognates pedagogically Séguin, H. & Tréville, M.-C. (1992); Tréville (1993) Friel & Kennison (2001) “… methodological issues and descriptive norms” … have foundered or become too complex for use on the problem of classifying words as cognate Our approach: make principled decisions Build a large scale testing environment for these decisions Then test and revise And test and revise… Added by M: There was a step missing

39 Our principles for classing words of varying degrees of cognateness – GL or AS
FORM (ortho) M E A N I N G SIMILAR DIFFERENT SIM- ILAR (1) video (vidéo) (2) school (école) DIFF- ERENT (3) actual (actuel) (4) impeach (empêcher) 0-1 difference in form, 0-1 difference in meaning = cognate (with training)

40 GRECO-LATIN FAMILIES - 1k a able absolute accept account act active actual address admit affect age agent air amount and another apart apparent appear apply appoint approach appropriate argue arrange art associate assume at attend authority baby balance ball bank bar base basis beat beauty benefit blue boat bottle box brief brilliant budget bus buss by card case cat cause cent centre certain chair chance change character charge choice choose claim class clear client club coffee colleague collect college colour comment commit committee common community company compare complete concern condition confer consider consult contact continue contract control converse copy correct cost council count county couple course court cover create current cut danger date debate decide decision definite degree department depend describe design detail develop difference difficult dinner direct discuss district divide doctor document double doubt due during each east economy educate effect elect electric employ encourage enjoy enter environment equal especial exact example except excuse exercise exist expect expense experience explain express extra eye face fact family farm favour figure film final finance fine finish force form fortune function fund future garden gas general govern grand group guy hate have he history honest hospital hour hullo idea identify imagine important include individual industry inform insure interest introduce invest issue item join judge just key labour language large law letter limit line link list local long machine major mark market marry matter measure member mention mile million minister minute mister moment motion music … ` Use this one

41 ` ALL 1k + 2k INCORPORATED INTO VOCABPROFILE  Use this one
ANGLO-SAX FAMILIES - 1k about achieve across add advertise afford after afternoon again against ago agree all allow almost along already alright also although always answer any area arm around as ask available aware away awful back bad bag be bear because become bed before begin behind believe best bet between big bill birth bit black bloke blood blow board body book both bother bottom boy break bring brother build business busy but buy cake call can car care carry catch chairman chap cheap check child church city clean clock close clothe cold come compute cook corner could country cross cup dad day dead deal dear deep die do dog door down draw dress drink drive drop dry early easy eat egg eight either eleven else end engine enough even evening ever every evidence fair fall far fast father feed feel few field fight file fill find fire first fish fit five flat floor fly follow food foot for forget forward four free Friday friend from front full fun further game get girl give glass go god good goodbye grant great green ground grow guess hair half hall hand hang happen happy hard head health hear heart heat heavy hell help here high hit hold holiday home hope horse hot house how however hundred husband if improve in income increase indeed inside instead into involve it job jump keep kid kill kind king kitchen knock know lad lady land last late laugh lay lead learn leave left leg less let level lie life light like likely listen little live load lock look lord lose lot love low luck lunch main make man manage many match may maybe mean meaning meet middle might milk mind minus miss Monday money month more morning most mother move mrs much must … ALL 1k + 2k INCORPORATED INTO VOCABPROFILE  ` Use this one

42 The difference between the kids’ and professors’ same-k texts becomes clear

43 Can we test the combined effect of Frequency x Cognateness?
MINI-EXPERIMENT Class task, TESL course in teaching of reading 40 teachers in training, 2011 and 2012 Each student prepared ~ Two sequential texts from a Bookworm graded reader Starting from a very high A-Sax component Text 1 = original text (98% of lexical words = 1k+2k) Text 2 = cognate text (GL-level 10% increased) Sequence varied by choice (roughly 50-50) Cognate text made using Edit-to-Profile feature of VPCOGNATES

44 MINI-EXPERIMENT (cont’d)
Student teacher finds low-intermediate ESL learner to read and retell texts in L1 cognate-low and cognate-high texts Provenance of measure: Bernhard, 1991 Comprehension measure 10 pre-determined key elements Score is number of key elements included or alluded to in re-tell

45 Is there proof it makes any difference
Is there proof it makes any difference? From the annals of the class Moodle… Heredera Plamondon

46 Is there proof it makes any difference?
Valiquette Results Anglo-Saxon text results 6/11 = 55% Greco-Latin text results 8/11 = 73% Etc. SUMMARY > 40 informal case studies With a range of low-intermediate French ESL learners Show consistent advantage for higher-cognate texts

47 Ramp up VPCOGNATES to 10 k-levels
SO now… Ramp up VPCOGNATES to 10 k-levels To explore our AS and GL classifications in a full range of authentic texts And investigate learner response to cognate-low and cognate-high texts This job has in itself has already led to some interesting observations Which both inform this work and shed light on its importance

48 List carve-up ( 1k, 2k … – 10k)

49 An aside on methodology…

50 Observation 1 We can see the rough proportions of each of the four degrees of cognateness Against the backdrop of a near-exclusive focus on ‘faux- amis’ in previous research AT ALL 10 FREQUENCY LEVELS, 97 % OF ITEMS ARE IN BOX 1, 2 OR 3 4’s ARE QUITE RARE FORM (ortho) M E A N I N G SIMILAR DIFFERENT SIM- ILAR (1) video (vidéo) (2) school (école) DIFF- ERENT (3) actual (actuel) (4) impeach (empêcher)

51 And further - As the analysis proceeded from 1k  5k  10k ~
The proportion of Box 3 items gradually decreases Leaving the vast majority in Box 2 Very similar meaning but with one form difference Highly promising for a form-recognition training program FORM (ortho) M E A N I N G SIMILAR DIFFERENT SIM- ILAR (1) video (vidéo) (2) school (école) DIFF- ERENT (3) actual (actuel) (4) impeach (empêcher)

52 Example: 30 cognates at different k-levels
Example: 30 cognates at different k-levels (Words with meaning issues italicized) 1k COGNATES able absolute accept account act active actual add address admit affect age agent air amount apart apparent appear apply appoint approach appropriate area argue arrange art associate assume attend … 8k COGNATES abhor accentuate accordion acetate acoustic acronym acrylic adrenaline aggressor aide allude altar altitude altruistic amnesty anaesthetist ancillary angina anthem antiseptic apartheid apostrophe arboretum archaeological arctic armistice armoury aromatherapy asbestos assimilate …

53 Observation 2 all the way up to 10k as has been commonly assumed
QUIZ FOR AUDIENCE : What happens to the proportion of ASax items as we head toward 10k ? ANSWER : Almost half the common lexicon of English is ASax all the way up to 10k ASax does not dwindle after 2k as has been commonly assumed

54

55 Observation 3 The full 10k cognate analysis enables us to…
Evaluate the GL-ASax components of full-size, natural English texts (beyond graded readers + course books) And assess the true importance of building an Asax lexicon for French (and other Romance) learners Depending on their reading goals 

56 QUIZ FOR AUDIENCE : What are the proportions of Asax to GL items across text types?

57

58

59

60

61 Answer to Quiz Some text-types can go up to 45% GL composition Despite recurrence of high-frequency ASax items like prepositions and helping verbs Or down as low as 8% Factor of 5 And patterns are remarkably consistent across text types The 10k-analysis allows us to see this

62 This has huge implications for L1 Romance learners reading English
Reading in some domains could be quite easy and proceed on largely L1+cognates basis Law, medicine, sciences generally GL 40% by tokens, mainly 3k-10k Applied linguistics articles? Others could be quite difficult and require major lexical expansion Fiction GL 11% by tokens

63 But before that ~ Are our classifications valid on the level of practice? We must use VPCOGNATES to create texts on numerous topics combining Frequency x Cognateness in known ratios Then test these texts for comprehensibility As previously but With a broader range of texts (Not just graded readers) With a broader range of learners (Not just low intermediates) With a broader range of measures (Not just translation)

64 Ultimate goals For Francophones learning English
Develop recognition-training for Box 2 cognates (école) Interpretation training for HFreq Box 2 cognates (actuel) ? Define the cognates that should just be learned as new words (impeach) Incorporate (a) frequency and (b) cognateness into reading programs Presumably proceeding from high-cognate to low cognate texts In line with learners’ goals Whether with regard to found or adapted texts And eventually work a similar program English French

65 Questions?

66 ??? Quite amazing that ~ Learning design for ESL reading typically does not incorporate the GOALS of learner, in coordination with COGNATES or lack of them in the type of English texts the learner is targeting Possibly because of the size of the task of doing anything with cognates Which we here have cut down in complexity somewhat – still a ton of work, and many classifications are debatable No excuse not to have a plan for cognates!

67 Sample translation rated 2