Monday 4 October 2021

It's over 9000!!

(This post includes video clips that may not show up on mobile devices)

The recent hit Netflix show Squid Game has caused controversy online after reports that the translation is not correct. 

Podcast host Youngmi Mayer caused a stir when she tweeted out that the English translations of the Netflix show were "botched" and that "if you don’t understand Korean you didn’t really watch the same show". She noted that "zero" of the original dialogue was preserved. So, what happened? How could a huge company like Netflix make such a mess of translating a show like this? 

I decided to investigate. 

The first thing to understand about translations are that there are different types. One is literal and another is localised. For an example of the problems that translating can cause think about the phrase "bless you". In Japanese no one says "bless you" after someone sneezes so if you were translating a "bless you" in a movie into Japanese, how would you do it? 

A literal translation would have you translate the phrase "bless you" into Japanese. That would be weird because it would imply some kind of religious experience. So, you might opt instead to "localize" the dialogue and have someone say something like "caught a cold?" which might express the same level of concern as "bless you". 

Localisation explains why the 90s show Pokemon rendered Rice Balls into "donuts". Presumably Western 90s kids had never seen rice balls before and the translator opted to go for a more familiar food. 

Another issue with translation is that it often differs depending on whether you are translating for a dub or for subtitles. With dubbing, getting a translation that matches the mouth movements of the actor may take precedence over literal accuracy. If someone says "Get out!" in English, you would look for a similar phrase in the other language which had two syllables (Check this video out for an interesting look at this process). 

Rumour has it that the infamous "over 9000" meme (original "over 8000" in Japanese) was dubbed as 9000 because it better fit the mouth movements (I can't find any evidence to support this claim though it would be a curious change to make for no reason). 

For subtitles, the main concern is space. Time reading is time not watching the show. Some people are also much slower readers than others and so brevity is key. If you are interested in the rules for subtitles you might want to peruse this Channel 4 guide. For dramas, they allow 38 characters per line. That's about the length of the previous sentence. 

All of this means, if you watch a dubbed foreign show with the subtitles on you will find that they don't match at all. Both are good translations but the subs and the dub are working with different constraints. 

With further ado, let's take a look at the complaints about Squid Game. 

The first one is that one of the female players, Han Mi-Nyeoswears a lot and the swear words get removed in the translation. The example she gives is when the player says "what are you looking at" and the subtitles say "go away". 

The problem with this criticism, as a few twitter users pointed out to her, is that rather than watching the subbed version she seemed to be watching the [CC] closed caption version. As this account points out, the closed caption is a transliteration of the dub. The line is actually correctly translated in the subtitles as "what are you looking at" but in the CC version it's just "go away". But why the difference?  As Yuri7 notes, the original Korean phrase is two syllables so the dub opted for the two syllables "go away" presumably to fit the mouth movements better. 

The next example Mayer uses is a character who says "I'm not a genius but I can work it out" rather than the correct, "I'm really smart but I never got a chance to study". As it turns out though, Yuri yet again notes that Mayer was watching the closed captions and not the subtitles, which did accurately render that line as "I never bothered to study but I'm unbelievably smart". The Gganbu scene can also be explained as an issues with space and reading the CC instead of the subs. 

So this all seems to have been just a simple misunderstanding. No harm, no foul, right?

Perhaps, but something that makes me a little uncomfortable about all of this is the speed at which this spread and was reprinted in several newspapers (for example, in the BBC, the NY post and the Independent.) without really anyone bothering to check the accuracy of it.  

I also feel a little sorry for the translator(s) who worked on the show. To her credit, Mayer wrote that it wasn't the fault of translators but the industry which overworks and underpays them. Presumably though, Mayer has no idea if the translating agencies Netflix employ, overwork or underpay employees, but this tweet has 33k likes and so most people will be left with that impression. 

Mayer's original tweet has 116k likes and her tiktok video has 2 millionMost people who have seen this story are now left thinking that Netflix subtitles are rubbish and this is because translators are exploited. All of this is because of one person's misunderstanding. Also, rather disappointingly, after having the mistake pointed out to her and admitting she got it wrong, Mayer still has not deleted the original post or published a correction. 


*notes: I don't speak Korean and I'm not a translator so if any of the information in this post is wrong let me know and I'll correct it. 

Saturday 25 September 2021

Everything you ever wanted to know about massing, blocking, spacing and interleaving (Interview with John Rogers)

While writing our recent book, an introduction to evidence based ELT, we contemplated writing a chapter on grammar teaching using massed and spaced conditions. Quickly it became clear that this was a rather complicated area and not one with clear findings, and so the chapter didn't make it to the book. This was a shame because there was a lot of interesting research we came across when planning it. 

While reading around the subject, one name that kept coming up in the SLA research was John Rogers. So I was delighted when I discovered John's account on twitter and when he agreed to talk to me about this area of research. 


If you would like to read more of John's work then check out his profile here or if you have any questions then follow him on twitter



Wednesday 7 July 2021

Interview with Sue Leather and Jez Uden (Extensive Reading)

Sue Leather and Jez Uden have a new book out on the topic of Extensive Reading and Motivation. I was keen to talk to them after looking into research on extensive reading and finding it to be one of the most effective interventions available to teachers. Here we chat about their new book, and some of the benefits of extensive reading. 


If you would like to read the book then you can get it from Amazon or directly from the Routledge site.


Thursday 1 July 2021

An introduction to evidence-based teaching


So, rather unbelievably, I have a book coming out

In February 2019 I gave a plenary at Ireland ELT. Afterwards, Kirsten Holt of Pavilion said she enjoyed the talk and that someone should really write a book about this kind of thing. Later, at a Pavilion event Kirsten mention the same idea to Carol Lethaby. As it turned out, Carol and I had previously worked together on a couple of papers and Carol and Patti Harries were already working on proposal for a book. 

Two years later and the book is finally out to buy! 

I think this book is quite different to other books on teaching and research (but then again, I would say that). A lot of books about research into teaching are written by researchers attempting to bring research findings to a more general audience. What we have attempted to do is something similar but as all of us are teachers, not researchers, we hope that this will give the work more of a teacher's perspective on things. 

An interesting aspect of the book for me was that it touches on topics that aren't discussed that frequently in ELT. for instance, the information on Cognitive Load Theory, retrieval practice and the importance of background knowledge was largely new to me (though I had seen it mentioned in non ELT books). 

We have tried to provide clear sources for all the claims and pack as many useful ideas into the text as possible. I feel pretty confident that even those who know quite a lot about research will find some interesting takeaways in here. 

And while we do mention learning myths and things that don't have good evidence to support them it isn't 200 pages of slamming learning styles. There is as much "what works" as "what doesn't". The book also features advice for teachers who want to look at research themselves. 

The books is not a "how to" or a "100 great games" type of book. There are no photocopiable practical activities for teachers to use in their next class. That said, it does have a lot of practical suggestions on what teaching techniques might be effective and how materials can be adapted or replaced. The second half is also practical and contains activities for teachers or trainee teachers, so the book could be used on teacher training courses like the DELTA, CELTA or even on MA courses. 

Despite bearing the title "evidence based" we are not making any claims that the book is flawless or the definitive word on best practice in ELT. The word "introduction" is used to indicate that we do not go into minute amounts of detail on every topic. Rather, the book gives and overview on a number of topics. No doubt there will be some who disagree with our conclusions on some topics as it's hard to find 100% agreement on much. There are even parts that the authors don't entirely agree on. That being said we believe we have gone with reasonable positions on the issues we have dealt with. 

Thanks to everyone who has offered advice or constructive feedback over the years! 

I hope you enjoy the book 

Tuesday 1 June 2021

A list of lists

While reading for my forthcoming book I came across a lot of lists that authors had put together based on what we could say about teaching from the available evidence. I thought they were interesting but they didn't make it into the book. So here is a list of those lists. 

(There are some other interesting lists which are too long to include here. They include Swan's list of things he believes about language teaching, Thonrbury's 12 observations, and Hattie's list of principles.)

Mike Long (2014

  1. Use task, not text, as the unit of analysis 
  2. Promote learning by doing
  3. Elaborate input (do not simplify; do not rely solely on “authentic” texts). 
  4. Provide rich (not impoverished) input.
  5. Encourage inductive (“chunk”) learning.
  6. Focus on form (TBLT)
  7. Provide negative feedback
  8. Respect “learner syllabuses”/ developmental processes.
  9. Promote cooperative/ collaborative learning.
  10. Individualize instruction

Rod Ellis (2005) principles of instructed language learning 

  1. Instruction needs to ensure that learners develop both a rich repertoire of formulaic expressions and a rule-based competence
  2. Instruction needs to ensure that learners focus predominantly on meaning
  3. Instruction needs to ensure that learners also focus on form
  4. Instruction needs to be predominantly directed at developing implicit knowledge of the L2 while not neglecting explicit knowledge
  5. Instruction needs to take into account the learner’s ‘built-in syllabus’
  6. Successful instructed language learning requires extensive L2 input
  7. Successful instructed language learning also requires opportunities for Output
  8. The opportunity to interact in the L2 is central to developing L2 Proficiency
  9. Instruction needs to take account of individual differences in learners
  10. In assessing learners’ L2 proficiency it is important to examine free as well as controlled production

Nation's principles of vocab learning (2013)

1. Provide clear, simple, and brief explanations of meaning
2. Draw attention to the generalisable underlying meaning of a word
3. Give repeated attention to words
4. Help learners recognize definitions
5. Prioritise what should be explained about particular words
6. Help learners remember what is explained
7. Avoid interference from related words

Hunt and Beglar (1998) principles of vocab learning

1. Provide opportunities for the incidental learning of vocabulary.
2. Diagnose which of the 3000 most common words learners need to study.
3. Provide opportunities for the intentional learning of vocabulary.
4. Provide opportunities for elaborating word knowledge.
5. Provide opportunities for developing fluency with known vocabulary
6. Experiment with guessing from context.
7. Examine different types of dictionaries and teach students how to use them.

Lightbown generalizations drawn from research (1985)

  1. “Adults and adolescents can ‘acquire’ a second language.”
  2. “The learner creates a systematic interlanguage which is often characterized by the same systematic errors as the child learning the same language as the first language, as well as others which appear to be based on the learner's own native language”
  3. “There are predictable sequences in acquisition such that certain structures have to be acquired before others can be integrated”
  4. “Practice does not make perfect”
  5. “Knowing a language rule does not mean one will be able to use it in communicative interaction”
  6. “Isolated explicit error correction is usually ineffective in changing language behaviour”
  7. “For most adult learners, acquisition stops—'fossilizes'—before the learner has achieved native-like mastery of the target language”
  8. One cannot achieve native-like (or near native-like) command of a second language in one hour a day
  9. “The learner's task is enormous because language is enormously complex”
  10. “A learner's ability to understand language in a meaningful context exceeds his/her ability to comprehend decontextualized language and to produce language of comparable complexity and accuracy”

Palmer's principles of language teaching 

(1) The initial preparation of the student by the training of his spontaneous capacities for assimilating spoken language.
(2) The forming of new and appropriate habits and the utilization of previously formed habits. 
 (3) Accuracy in work in order to prevent the acquiring of bad habits. 
 (4) Gradation of the work in such a way as to ensure an ever-increasing rate of progress. 
 (5) Due proportion in the treatment of the various aspects and branches of the subject. 
 (6) The presentation of language-material in a concrete rather than in an abstract way. 
 (7) The securing and maintaining of the student’s interest in order to accelerate his progress. 
 (8) A logical order of progression in accordance with principles of speech-psychology. 
 (9) The approaching of the subject simultaneously from different sides by means of different and appropriate devices

Van Patten 5 implications of research

1. The more input the better
2. The more interaction the better
3. All learner production should be meaning based or communicative
4. Focus on Form should be meaning based or tied to input or communication
5. We should watch out for what we expect from our learners

Ortega 5 generalizations of interlanguage research (2009)

1. Instruction affect the route of L2 development in any fundamental way
2. Instruction can have some effect on processes, fostering some and inhibiting others
3. Instruction can be ineffective and even counterproductive when it ignores developmental readiness
4. Not all sequences present equal challenges for instruction
5. Instruction has a large positive effect on rate of development and ultimate attainment

Ferris’s principles of preparing teachers for written error correction

1. Teachers of L2 writers need to study aspects of grammar that are particularly problematic for non-native speakers of English.
2. Teachers of L2 writers need practice in recognizing and identifying errors in student writing.
3. Teachers of L2 writers need practice in developing lessons and teaching grammar points and editing strategies to their writing students.
4. Teachers of L2 writers need to understand the principles of second language acquisition and of composition theory.
5. Teachers of L2 writers should become familiar with language structures needed for different task types and academic disciplines.

Dornyei's 10 commandments of motivation 

1. Set a personal example with your own behaviour
2. Develop a good relationship with the learners
3. Increase the learners’ linguistic self-confidence
4. Make the language classes interesting
5. Promote learner autonomy
6. Personalise the learning process
7. Increase the learners’ goal-orientedness
8. Familiarize learners with the target culture
9. Create a pleasant relaxed atmosphere in the classroom
10. Present the tasks properly

Grabe and Stoller reading syllabus principles (2020)


1. Ask students to read a lot and often for well-defined purposes.
2. Incorporating deliberate practice into reading curricula supports reading skill development
3. Promote discussion about the text
4. Build students motivation to read
5. Teach vocabulary as the foundation to reading
6. Make students aware of textual features
7. Work on fluency
8. Teach learners to be strategic readers
9. Teach rather than test for main idea comprehension
10. Reading lessons should be pre- during post style
11. Texts should be selected based on students needs and ability
12. Digital literacy needs should be considered
13. Connect reading to writing
14. Assess their progress

Wednesday 5 May 2021

EBEFL asks: should we use translation software?

 I was recently presented with an almost flawless piece of writing from a students whose English level precluded her producing such an almost flawless piece of writing. Initially I thought, “oh no…we have to have *that* conversation”…

In her tutorial the student guilty confessed to using translation software. I told her I was surprised because google translate famously produces awful translations from Japanese to English. “ah” she said, “I didn’t use google”.

She directed me to a site called DeepL. I threw a bit of Japanese in from Wikipedia and this is what I got out.

DeepL

Now this isn’t perfect but it’s pretty damn good. For good measure I threw it into google translate and got a pretty good rendering too.

google translate

I was quite surprised at how good the Google Translate version was. But I shouldn’t have been . Sure, it was an endless source of comedy in 2004 when it produced weird and wacky sentences, but that was 15 years ago and technology moves on (in 2004 no one thought computers would beat humans at Go any time soon, that happened in 2015. There is an excellent documentary about it online). Google translate switched to using “Neural Machine Translation” around 2017 and this has reportedly led to much better quality translations.

So, is there any point in banning students from using translation software to write their essays anymore, particularly in EAP contexts? We wouldn’t mind them using dictionaries to translate words, and rather than just banning them, perhaps we could focus on getting them to use this tool more effectively? It certainly beats receiving a paid for or plagiarised submission.

Let me know your thoughts.

Retraction in ELT

 I am currently reading the new book, Science Fictions by Stuart Ritchie. It deals with meta research (research about research) and outlines all the ways in which science is currently going wrong. In one section dealing with retraction, Ritchie notes that “1.97 per cent of scientists admit to faking their data at least once” and suggests that that number is probably an underestimate as people are unwilling to admit to things like this even when asked in anonymous surveys.

This number means that for every 50 papers published in ELT one is likely to contain faked data. Some of these cases have come to light in biology and psychology and this led me to wonder if there were many retractions in ELT. So I asked twitter.

Marc Jones instantly found two (here and here) both of which were retracted due to plagiarism. One was plagiarism of another scholars work and the other was self plagiarism (submitting the same paper for double credit). I also found one from The journal of Computer Assisted Language Learning retracted for (self) plagiarismSo far no fraud.

Stuart Ritchie pointed me to the retraction watch searchable database. I tried searching by journal and found a RELC paper published by Ivan Chong which was withdrawn for “significant data errors“. There was also The journal, System which had the misfortune to publish one article twice –in the same issue!

The journals Applied Linguistics, ELTJ and TESOL Quarterly have apparently had 0 retractions to date. The database is not complete though and I was also sent this piece which was retracted from the prestigious “language learning” journal. It’s not that clear what went on here but it seems like one author noticed errors in the data and requested a retraction.

As an interesting aside, Richie notes that people continue to cite retracted articles even after they have been retracted. I was curious about this so I used google scholar to check citations of the 2003 Language Learning article. There were hits from 2018 and even from 2019. I don’t know the date of the retraction but I feel pretty confident it was well before these dates.

Out of thousands of papers in ELT I could find only two that were withdrawn due to data issues. So either ELT is a paragon of virtue or we haven’t got very good at sniffing out fraud yet.

But how easy is it to get a journal to retract a paper? While researching an article on learning styles, I came across a couple of very curious articles. The first was Hyland 1993, the second was Hyland 1994. While reading the 1994 article I got a strange sense of Deja Vu:

In essence, learning style research suggests that people make sense of the world in different ways and these ways are partly created by cultural experiences (Hyland 1994)

Learning style research therefore suggests that people make sense of the world in different ways, more importantly however, these ways are partly created by cultural experiences (Hyland 1993)

and

Eight Japanese universities participated in the survey with 265 undergraduate students responding. The questionnaire was also administered to 140 Japanese students at various English proficiency levels at a tertiary college in New Zealand to determine whether overseas study influenced modality and group preferences. (Hyland 1993)

Eight Japanese universities participated in the survey with 265 undergraduates responding. The questionnaire was also administered to 140 Japanese students at different English proficiency levels at a tertiary college in New Zealand. (Hyland 1994)

and

Essentially the concept expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikesbut studies have addressed an enormously wide range of factors. (1993)

Learning style research expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikesbut people differ in their learning styles in a number of ways and studies have addressed a huge range of factors. (1994)

Most of the later (1994) article is a verbatim copy of the earlier one with minor phrasing adjustments such as those shown above. This type of thing is usually known as either self-plagiarism (see the examples above) or, on a smaller scale, text recycling and is considered unacceptable in academic publishing. Many journals have rules against it, such as JALT itself:

Papers sent to JALT journal should not have been previously published, nor should they be under consideration for publication elsewhere.

I thought this was a bit strange and so I contacted JALT, the 1994 publisher, to make them aware of the issue. They told me that they take such matters very seriously and would investigate. After an investigation they informed me that this was all just an honest mistake, a bit of a mix up. The paper had been submitted to two journals by accident and and as a previous editor had dealt with the matter, it would be wrong to retract the article now.

How one accidentally submits the same article to two journals is, I must confess, a mystery to me. More mysterious was JALT’s reasoning. Regardless of what a previous editor had decided, a repeat publication is still in the literature with no indication that it is a repeat.

I wrote back suggesting that since I was unaware that it had all been resolved, as was the current editor and presumably future readers, it might be worth retracting the article, or at least adding a note to explain what had happened. They told me they were very grateful for my suggestion but no, they weren’t going to do anything. And so both articles remain in the literature.

It is also odd that Hyland himself, a incredibly respected editor and prolific author would not want the article to be retracted. In fact, until recently he continued to list both papers among his publications (his new blog, however, only lists papers from 2003). Not retracting the paper may be less embarrassing in the short term but it means that there is always the chance for some annoying blogger to bring attention at some point, to what was a mix up .

In truth, I’m not ‘that’ surprised that nothing happened here. Brendan O’Connor a student at the University of Leicester discovered that a well respected psychologist was “recycling” parts of papers into new publications to an alarming degree. Although O’Connor has documented this to an impressive degree, some journals were reluctant to do anything at all when confronted with these findings.

As whistleblowers, data sleuths and anyone else who’s contacted a scientific journal or university with allegations of impropriety will tell you, getting even a demonstrably fraudulent paper retracted is a glacial process – and that’s if you aren’t simply ignored or fobbed off by the authorities in the first place.

Science Fictions

Well quite.

Non evidence based teaching tips

 Here are a random collection of things which I think are really important but which are not really evidence based.

5. Deal with behavioural issues as soon as possible

If you’re like me, then having a student on their phone or basically doing something distracting will put you off your stride. I think most of the time, with adult learners there’s no malice, they are just thinking about their own problems.

If something is bothering you in class then deal with it as soon as possible. If a student is bothering you, then it’s likely they are bothering other students in the class. They suffer and you suffer. I usually ask to see anyone who is doing something they shouldn’t after the class. I don’t think this has to be a difficult conversation, just “don’t do X”. Setting clear rules on day 1 is a big help.

4. use coloured paper with wide margins and light ink when making cards

It might seem a bit “TEFL” for an EAP teacher but it really saves time. If you are producing cut-ups then pre-make a document which has side margins wide enough that you don’t have cut them at all. Make the spaces between the sentences as large as possible because there is nothing worse than chopping a few words off one set. Finally, print each set on different coloured paper to make them easy to sort.

3. Everything usually takes longer than you think

I have a compulsive fear of running out of materials before the end of the lesson and I know that I am not alone. I did have a tendency to over plan for a long time and end up not getting through the material. I’m better at this now, and there are some easy ways to control the time in the lesson. These usually relate to how you give the answers (just write them all up and let the students check themselves or nominate people and have others confirm the answers). You can also, – shock horror – do activities twice if you find you have too much time. It might seem lazy, but the students will probably benefit from the practice.

(I only actually ran out of material twice in 20 years, which isn’t bad.)

2. Learn student names

I’m not great with names but I make sure that I always do this in the first class. It’s really cringy by the third lesson to be pointing and saying “you” and I think it makes the classes less effective. If I can’t learn them because there are too many students, or because I will only see the group a few times then I give them a name plate of some kind.

  1. Do the damn worksheet

This is a cardinal rule for me and yet the one that I observe the least, much to my detriment. Do the damn worksheet yourself. It’s only once you are in class that you realise that Q3 isn’t actually possible in passive, or that you can’t explain something about the grammar point you are supposed to be focusing on, or you don’t actually know why “pain” is countable in “what a pain”, or how to explain the word “innovative”, or why “try” takes infinitives and gerunds while”trying” only takes infinitives, and on and on for 20 years. It is only when you try to do the worksheet that you realise the planned activity is not physically possible or that it would take 2 hours to complete. Do the damn worksheet!

What are your basic teaching tips?

Evidence isn’t enough

 

Why do some people continue to believe that climate change is not a threat despite the overwhelming scientific evidence that it is?

One theory is that the people who reject the evidence for climate change are doing so because they are uninformed. The idea is that if they were only a bit more educated on science they would suddenly realise the error of their ways. This is called the knowledge deficit model.

Researchers have sought to investigate how accurate this model is. They tested the hypothesis that a lack of knowledge about climate change predicted thinking it was less of a risk. They tested subjects scientific literacy and compared that with views on climate change.

If their hypothesis were true we would expect to see that risk perception would rise with more knowledge. It would look a bit like this.

source here

In fact they found this. 

source here

There was no correlation between scientific literacy (as measured by a test) and belief that climate change was a risk. Some of those who believed climate change was a big problem knew a lot about science and conversely some who knew little about science thought the risk was great. In short, the knowledge deficit model was a bust.

However, when the researcher compared political affiliation with belief in climate change they found the correlation they were looking for. It seems that being left wing was a much bigger indicator of believing climate change was a risk and vice versa for being right wing and this was regardless of scientific literacy.

The researchers propose that when a subject, like climate change, becomes politically partisan, facts will do very little to alter opinions. David McRaney has dubbed this “tribal psychology“. When something becomes a matter of identity for your in-group, then the evidence ceases to matter very much.

Take something as simple as wearing face masks during a pandemic. A year ago it would have been hard to image people physically attacking each other over an issue like this, but here we areMcRany has an excellent Podcast on this subject. He notes that masks may now be a badge of group loyalty, like putting pronouns in your bio. These findings are especially interesting when thinking about research and teacher beliefs.

In education most people are already left wing, yet the battle between trad and prog teacher “tribes” rages on with almost daily outrages during which members can prove their group loyalty. Today, it’s language use in schools, before that it was exclusion and before that it was something else. Topics change, but the dance remains the same.

It’s very “natural” for people to pick a side and to stick to that side regardless. As McRaney notes, our evolutionary history as social animals means that group membership could have meant the literal difference between life and death. It’s also a very frustrating part of human psychology leading to irrationality, partisanship, class divisions, religious intolerance, and so on.

A large body of psychological research shows how when we form in and out groups we start to view those two sides very differently. Our side is hard done to, sensible, upstanding and decent. The other side is getting away with murder, idiotic, immoral and craven. If we make a mistake, it was an accident, if they do it was calculated. worst of all, McRaney notes that people would rather be wrong than out of good standing with their “tribe”.

If people’s identities are wrapped up in a certain sets of beliefs and if those beliefs form an in-group/out-group dynamic it is likely that everything will be viewed through that lens. it is also likely that, in this situation, no amount of evidence will change minds. A good example of this is phonics research, the results of which have been resisted for decades.

So here are some suggestions to try to avoid our natural tendencies towards “tribal psychology”. This advice is aimed mostly at myself, but if you find it useful, then great. I suggest these in the full knowledge I will probably be unlikely to follow them very well.

  • Try not to become too invested in any one side 

This is very difficult, but the more we see the other side as the enemy, the easier it is to start seeing everything through a partisan lens. Next, we may stop looking at the evidence all together and just side with our ‘tribe’.

  • Try to find things you can agree with  

Whenever possible and no matter how small it is, try to find points of agreement between you and people in your out group.

  • Try to stay agnostic where possible 

We would like the research to say “X” but it doesn’t exactly say “X”. Well, the best thing we can do is just not have an opinion. “I don’t know” is a valid opinion.

  • Be dubious of research that supports your world view

Paper finds thing I want to be true is wrong *scoff* “what was the sample size?” “Let me check the methodology section!” “This author is right winger!”

Paper finds thing I want to be true is true! RETWEET without reading! “OK the sample size isn’t ideal but it’s a promising piece of research!” “who cares what the author’s politics are?!”

  • Make appeals to people on their terms

Learning styles advocates never seem to get particularly fazed by the charge that the practice lacks evidence. They do, however, get upset by the idea that it pigeon holes students. My guess is learning styles advocates like the theory because it represents individualism and student centric learning. Questioning that claim resonates more than pointing out the lack of evidence.

Facts aren’t enough

Research tell us that the knowledge deficit model is flawed. If we can’t get people to believe in the threat of climate change despite the evidence, there is little hope of convincing them about anything else. A research based approach has to consider not only the facts that can be gleaned from research, but people’s feelings too. Bashing teachers over the head with research is unlikely to change minds but it may cause them to resent you. It’s then fairly easy for teaching experts to dismiss research as “irrelevant for teachers” and find a receptive audience.

What is acquisition and how is it measured?

 In SLA research, one finding seems beyond reproach is that there is a set order in which students acquire grammar. This “internal syllabus” cannot be overridden and thus textbooks that present grammar unit by unit are pointless and worse ‘unnatural’, because students are unable to learn what is taught until they are developmentally ready.

The research that underpins these claims comes from three main sources. The first are the morpheme studies which attempt to emulate L1 research showing native speakers learn English morphemes in a fixed order. The second is Pienemann’s work which unlike the morpheme studies does not look at the order of acquisition of several forms but instead looks at the stages learners go through in acquiring a single form (questions for instance). The third are interlanguage studies.

Although this research is often discussed, I have found the details are rarely forthcoming. I was curious to know two things about these landmark studies. Firstly, how was ‘acquisition’ measured, and secondly what do they consider to be ‘grammar’? In this post I will be looking at the morpheme studies.

1. what falls under ‘grammar’ and what does not?

In the morpheme studies, a set of roughly 10 morphemes are usually researched. These vary slightly such as when researchers separate articles into ‘the’ and ‘a’, or look at long and short plural sounds but in general they don’t differ much between researchers. The list of morphemes include such things as plural forms (dogs), Copula (is) (He is happy), auxiliary be (he’s coming), irregular pastregular pastarticles and possessive -s (John’s cat).

There is a lot that teachers would consider ‘grammar’ that is not included. For instance:

  • I should play tennis. (Modals)
  • If you like it, then buy it. (Conditionals)
  • I’ve told you already (perfect forms)
  • What are you doing? (Questions)

2. How is acquisition measured?

In the morpheme studies, a test subject is said to have acquired a grammatical form if they can produce it correctly in a test. The test, called a Bilingual Syntax Measure (BSM), is usually carried out on children and involves showing cartoon like pictures and eliciting language from the subjects. A researcher will, for instance, say ‘here is a girl, now there are two of them. So there are two _____?’ this is known as an ‘obligatory context’ as students have to use the correct form to answer.

The next stage is that researchers score the learners depending on whether they produce the correct form or not. For instance (Dulay and Burt 1974):

  • totally correct ie. “she’s dancing” (2 points)
  • half right, ie. “She’s dances” (1 point)
  • wrong ie. “She’s dance” (0 points)

The scores of the entire group are then added up and plotted on a chart. The equation used was the sum of the whole group / the number of possible points x 100. If more learners correctly produce plural -s than produce possessive -s, then the researchers claim that plural -s is acquired before possessive -s.* In the morpheme studies a form was said to be ‘acquired’ if subjects produced it accurately when elicited 90% of the time.

What did they find?

Researchers seemed to find that all students acquired language in the ‘roughly’ the same order regardless of their L1. For instance Mitchell and Myles (2004: 43) argue that these results suggest ‘second language learners are guided by internal principles that are largely independent of their first language’.

source oxfordenglishtesting.com

The interesting thing to notice when looking at this table is that the orders found were not actually the same between researchers, which is a little surprising for a ‘universal’ order. That said some researchers seemed not to mind and grouped the morphemes into ‘sets’ which are acquired in order.

source Krashen in Larsen-Freeman and Long 1991: 90

The eagle-eyed among you will perhaps spot that there are still some outliers here such as “articles” appearing in stage 2 yet 1st in Dulay and Burt and 11th in Hakuta.

Issues with this research

I was interested to discover, that despite the ‘Holy Grail of SLA research‘ status that the morpheme studies have achieved, they have been under scrutiny for almost as long as the have been around. Some of the criticisms levelled at this research is as follows (apologies for not being able to properly source the origin of these).

  • morphemes with different meanings (a/the) were grouped together in some studies
  • What was classed as ‘grammar’ was a very limited number of morphemes
  • most of the early research was carried out on ESL learners, not EFL students
  • The orders vary in different papers, notably Hakuta 1974 (n-1)
  • students were all grouped together to obtain results, hiding individuals or national groups who may not have followed the “natural” order.
  • the studies did not look at acquisition over time but rather just took a snapshot
  • accuracy order does not necessarily mean acquisition order
  • Students’ overuse of the target morphemes was not counted
  • The “universal order” is more accurately thought of as the “Spanish student order”

(Note some of these criticism have merit and others less. Check Larsen-Freeman and Long 1991 for a more detailed explanation.)

Notably, the claim that the order is universal has started to look suspect as L1 does indeed seem to have some influence on L2 (something that will not surprise most teachers). Luk & Shirai (2009) have argued that researchers continue to promote the order as ‘universal’ ignoring the evidence that it seems to be affected by a students’ L1. Corpus research, for instance shows that students seem to acquire morphemes in a different order. For instance, Japanese has a possessive particle ‘の’ but no plural particle and Japanese students seem to learn possessive -s before they learn plural -S (Anecdotally, this chimes with my experience). Hakuta’s study had a similar results and interestingly, Hakuta found that articles, which do not exist in Japanese, were late acquired by the Japanese student he studied.

Luk & Shirai (2009) found that not only Japanese but Korean and Chinese learners (all of who lack plurals) generally acquired possessive -s earlier and both plurals and articles ‘later than is predicted’ by the ‘natural order’ hypothesis. Other authors have noted that salience (how easy it is to hear the morpheme in input) could also play a role in explaining the order. And another possible factor is frequency, which is ‘the second most popular of the suggested causes of the L2 functor acquisition order (after L1 transfer)’ (Goldschneider and DeKeyser 2002: 29)

So the ‘holy Grail’ seems to have a few cracks in it. One author who believes that the morpheme studies have been used to make claims that they could not support is Mike Swan, who notes:

We have no reason at all to believe that the learning of most grammatical items is constrained in this way: that for yet-to-be uncovered developmental reasons, students might need to learn comparatives before relative pronouns, dativizing verbs before quantifiers or infinitives of purpose before possessive ’s. To claim that learnability findings preclude the operation of a grammatical syllabus is a large and unjustified leap across a wide logical gap.

(Swan 2018: 254)

*the research methods are actually a bit more complex than this and differed between researchers but I have simplified it for the purposes of the this post.