In a 2012 post I talked about the famous Pygmalion effect in education which shows that teachers expectations for students alone were enough to influence actual outcomes. It’s a truly amazing finding and minimizes the influence of home life, peer group and IQ. Rosenthal and Jacobson showed that, as I noted, "teachers' attitudes towards their students can affect students in quite remarkable ways". The authors were lauded and their paper has been cited over 12,000 times but there was just one problem, it probably wasn't true.
Tuesday, 18 January 2022
Monday, 4 October 2021
(This post includes video clips that may not show up on mobile devices)
The recent hit Netflix show Squid Game has caused controversy online after reports that the translation is not correct.
Podcast host Youngmi Mayer caused a stir when she tweeted out that the English translations of the Netflix show were "botched" and that "if you don’t understand Korean you didn’t really watch the same show". She noted that "zero" of the original dialogue was preserved. So, what happened? How could a huge company like Netflix make such a mess of translating a show like this?
I decided to investigate.
The first thing to understand about translations are that there are different types. One is literal and another is localised. For an example of the problems that translating can cause think about the phrase "bless you". In Japanese no one says "bless you" after someone sneezes so if you were translating a "bless you" in a movie into Japanese, how would you do it?
A literal translation would have you translate the phrase "bless you" into Japanese. That would be weird because it would imply some kind of religious experience. So, you might opt instead to "localize" the dialogue and have someone say something like "caught a cold?" which might express the same level of concern as "bless you".
Localisation explains why the 90s show Pokemon rendered Rice Balls into "donuts". Presumably Western 90s kids had never seen rice balls before and the translator opted to go for a more familiar food.
Another issue with translation is that it often differs depending on whether you are translating for a dub or for subtitles. With dubbing, getting a translation that matches the mouth movements of the actor may take precedence over literal accuracy. If someone says "Get out!" in English, you would look for a similar phrase in the other language which had two syllables (Check this video out for an interesting look at this process).
Rumour has it that the infamous "over 9000" meme (original "over 8000" in Japanese) was dubbed as 9000 because it better fit the mouth movements (I can't find any evidence to support this claim though it would be a curious change to make for no reason).
For subtitles, the main concern is space. Time reading is time not watching the show. Some people are also much slower readers than others and so brevity is key. If you are interested in the rules for subtitles you might want to peruse this Channel 4 guide. For dramas, they allow 38 characters per line. That's about the length of the previous sentence.
All of this means, if you watch a dubbed foreign show with the subtitles on you will find that they don't match at all. Both are good translations but the subs and the dub are working with different constraints.
With further ado, let's take a look at the complaints about Squid Game.
The first one is that one of the female players, Han Mi-Nyeo, swears a lot and the swear words get removed in the translation. The example she gives is when the player says "what are you looking at" and the subtitles say "go away".
The problem with this criticism, as a few twitter users pointed out to her, is that rather than watching the subbed version she seemed to be watching the [CC] closed caption version. As this account points out, the closed caption is a transliteration of the dub. The line is actually correctly translated in the subtitles as "what are you looking at" but in the CC version it's just "go away". But why the difference? As Yuri7 notes, the original Korean phrase is two syllables so the dub opted for the two syllables "go away" presumably to fit the mouth movements better.
The next example Mayer uses is a character who says "I'm not a genius but I can work it out" rather than the correct, "I'm really smart but I never got a chance to study". As it turns out though, Yuri yet again notes that Mayer was watching the closed captions and not the subtitles, which did accurately render that line as "I never bothered to study but I'm unbelievably smart". The Gganbu scene can also be explained as an issues with space and reading the CC instead of the subs.
So this all seems to have been just a simple misunderstanding. No harm, no foul, right?
Perhaps, but something that makes me a little uncomfortable about all of this is the speed at which this spread and was reprinted in several newspapers (for example, in the BBC, the NY post and the Independent.) without really anyone bothering to check the accuracy of it.
I also feel a little sorry for the translator(s) who worked on the show. To her credit, Mayer wrote that it wasn't the fault of translators but the industry which overworks and underpays them. Presumably though, Mayer has no idea if the translating agencies Netflix employ, overwork or underpay employees, but this tweet has 33k likes and so most people will be left with that impression.
Mayer's original tweet has 116k likes and her tiktok video has 2 million. Most people who have seen this story are now left thinking that Netflix subtitles are rubbish and this is because translators are exploited. All of this is because of one person's misunderstanding. Also, rather disappointingly, after having the mistake pointed out to her and admitting she got it wrong, Mayer still has not deleted the original post or published a correction.
*notes: I don't speak Korean and I'm not a translator so if any of the information in this post is wrong let me know and I'll correct it.
Saturday, 25 September 2021
Everything you ever wanted to know about massing, blocking, spacing and interleaving (Interview with John Rogers)
While writing our recent book, an introduction to evidence based ELT, we contemplated writing a chapter on grammar teaching using massed and spaced conditions. Quickly it became clear that this was a rather complicated area and not one with clear findings, and so the chapter didn't make it to the book. This was a shame because there was a lot of interesting research we came across when planning it.
While reading around the subject, one name that kept coming up in the SLA research was John Rogers. So I was delighted when I discovered John's account on twitter and when he agreed to talk to me about this area of research.
Wednesday, 7 July 2021
Sue Leather and Jez Uden have a new book out on the topic of Extensive Reading and Motivation. I was keen to talk to them after looking into research on extensive reading and finding it to be one of the most effective interventions available to teachers. Here we chat about their new book, and some of the benefits of extensive reading.
Thursday, 1 July 2021
Tuesday, 1 June 2021
While reading for my forthcoming book I came across a lot of lists that authors had put together based on what we could say about teaching from the available evidence. I thought they were interesting but they didn't make it into the book. So here is a list of those lists.
(There are some other interesting lists which are too long to include here. They include Swan's list of things he believes about language teaching, Thonrbury's 12 observations, and Hattie's list of principles.)
Mike Long (2014)
- Use task, not text, as the unit of analysis
- Promote learning by doing
- Elaborate input (do not simplify; do not rely solely on “authentic” texts).
- Provide rich (not impoverished) input.
- Encourage inductive (“chunk”) learning.
- Focus on form (TBLT)
- Provide negative feedback
- Respect “learner syllabuses”/ developmental processes.
- Promote cooperative/ collaborative learning.
- Individualize instruction
Rod Ellis (2005) principles of instructed language learning
- Instruction needs to ensure that learners develop both a rich repertoire of formulaic expressions and a rule-based competence
- Instruction needs to ensure that learners focus predominantly on meaning
- Instruction needs to ensure that learners also focus on form
- Instruction needs to be predominantly directed at developing implicit knowledge of the L2 while not neglecting explicit knowledge
- Instruction needs to take into account the learner’s ‘built-in syllabus’
- Successful instructed language learning requires extensive L2 input
- Successful instructed language learning also requires opportunities for Output
- The opportunity to interact in the L2 is central to developing L2 Proficiency
- Instruction needs to take account of individual differences in learners
- In assessing learners’ L2 proficiency it is important to examine free as well as controlled production
Nation's principles of vocab learning (2013)
1. Provide clear, simple, and brief explanations of meaning
2. Draw attention to the generalisable underlying meaning of a word
3. Give repeated attention to words
4. Help learners recognize definitions
5. Prioritise what should be explained about particular words
6. Help learners remember what is explained
7. Avoid interference from related words
Hunt and Beglar (1998) principles of vocab learning1. Provide opportunities for the incidental learning of vocabulary.
2. Diagnose which of the 3000 most common words learners need to study.
3. Provide opportunities for the intentional learning of vocabulary.
4. Provide opportunities for elaborating word knowledge.
5. Provide opportunities for developing fluency with known vocabulary
6. Experiment with guessing from context.
7. Examine different types of dictionaries and teach students how to use them.
Lightbown generalizations drawn from research (1985)
- “Adults and adolescents can ‘acquire’ a second language.”
- “The learner creates a systematic interlanguage which is often characterized by the same systematic errors as the child learning the same language as the first language, as well as others which appear to be based on the learner's own native language”
- “There are predictable sequences in acquisition such that certain structures have to be acquired before others can be integrated”
- “Practice does not make perfect”
- “Knowing a language rule does not mean one will be able to use it in communicative interaction”
- “Isolated explicit error correction is usually ineffective in changing language behaviour”
- “For most adult learners, acquisition stops—'fossilizes'—before the learner has achieved native-like mastery of the target language”
- One cannot achieve native-like (or near native-like) command of a second language in one hour a day
- “The learner's task is enormous because language is enormously complex”
- “A learner's ability to understand language in a meaningful context exceeds his/her ability to comprehend decontextualized language and to produce language of comparable complexity and accuracy”
Palmer's principles of language teaching
Van Patten 5 implications of research1. The more input the better
2. The more interaction the better
3. All learner production should be meaning based or communicative
4. Focus on Form should be meaning based or tied to input or communication
5. We should watch out for what we expect from our learners
Ortega 5 generalizations of interlanguage research (2009)1. Instruction affect the route of L2 development in any fundamental way
2. Instruction can have some effect on processes, fostering some and inhibiting others
3. Instruction can be ineffective and even counterproductive when it ignores developmental readiness
4. Not all sequences present equal challenges for instruction
5. Instruction has a large positive effect on rate of development and ultimate attainment
Ferris’s principles of preparing teachers for written error correction1. Teachers of L2 writers need to study aspects of grammar that are particularly problematic for non-native speakers of English.
2. Teachers of L2 writers need practice in recognizing and identifying errors in student writing.
3. Teachers of L2 writers need practice in developing lessons and teaching grammar points and editing strategies to their writing students.
4. Teachers of L2 writers need to understand the principles of second language acquisition and of composition theory.
5. Teachers of L2 writers should become familiar with language structures needed for different task types and academic disciplines.
Dornyei's 10 commandments of motivation1. Set a personal example with your own behaviour
2. Develop a good relationship with the learners
3. Increase the learners’ linguistic self-confidence
4. Make the language classes interesting
5. Promote learner autonomy
6. Personalise the learning process
7. Increase the learners’ goal-orientedness
8. Familiarize learners with the target culture
9. Create a pleasant relaxed atmosphere in the classroom
10. Present the tasks properly
Grabe and Stoller reading syllabus principles (2020)
1. Ask students to read a lot and often for well-defined purposes.
2. Incorporating deliberate practice into reading curricula supports reading skill development
3. Promote discussion about the text
4. Build students motivation to read
5. Teach vocabulary as the foundation to reading
6. Make students aware of textual features
7. Work on fluency
8. Teach learners to be strategic readers
9. Teach rather than test for main idea comprehension
10. Reading lessons should be pre- during post style
11. Texts should be selected based on students needs and ability
12. Digital literacy needs should be considered
13. Connect reading to writing
14. Assess their progress
Wednesday, 5 May 2021
I was recently presented with an almost flawless piece of writing from a students whose English level precluded her producing such an almost flawless piece of writing. Initially I thought, “oh no…we have to have *that* conversation”…
In her tutorial the student guilty confessed to using translation software. I told her I was surprised because google translate famously produces awful translations from Japanese to English. “ah” she said, “I didn’t use google”.
She directed me to a site called DeepL. I threw a bit of Japanese in from Wikipedia and this is what I got out.
Now this isn’t perfect but it’s pretty damn good. For good measure I threw it into google translate and got a pretty good rendering too.
I was quite surprised at how good the Google Translate version was. But I shouldn’t have been . Sure, it was an endless source of comedy in 2004 when it produced weird and wacky sentences, but that was 15 years ago and technology moves on (in 2004 no one thought computers would beat humans at Go any time soon, that happened in 2015. There is an excellent documentary about it online). Google translate switched to using “Neural Machine Translation” around 2017 and this has reportedly led to much better quality translations.
So, is there any point in banning students from using translation software to write their essays anymore, particularly in EAP contexts? We wouldn’t mind them using dictionaries to translate words, and rather than just banning them, perhaps we could focus on getting them to use this tool more effectively? It certainly beats receiving a paid for or plagiarised submission.
Let me know your thoughts.
I am currently reading the new book, Science Fictions by Stuart Ritchie. It deals with meta research (research about research) and outlines all the ways in which science is currently going wrong. In one section dealing with retraction, Ritchie notes that “1.97 per cent of scientists admit to faking their data at least once” and suggests that that number is probably an underestimate as people are unwilling to admit to things like this even when asked in anonymous surveys.
This number means that for every 50 papers published in ELT one is likely to contain faked data. Some of these cases have come to light in biology and psychology and this led me to wonder if there were many retractions in ELT. So I asked twitter.
Marc Jones instantly found two (here and here) both of which were retracted due to plagiarism. One was plagiarism of another scholars work and the other was self plagiarism (submitting the same paper for double credit). I also found one from The journal of Computer Assisted Language Learning retracted for (self) plagiarism. So far no fraud.
Stuart Ritchie pointed me to the retraction watch searchable database. I tried searching by journal and found a RELC paper published by Ivan Chong which was withdrawn for “significant data errors“. There was also The journal, System which had the misfortune to publish one article twice –in the same issue!
The journals Applied Linguistics, ELTJ and TESOL Quarterly have apparently had 0 retractions to date. The database is not complete though and I was also sent this piece which was retracted from the prestigious “language learning” journal. It’s not that clear what went on here but it seems like one author noticed errors in the data and requested a retraction.
As an interesting aside, Richie notes that people continue to cite retracted articles even after they have been retracted. I was curious about this so I used google scholar to check citations of the 2003 Language Learning article. There were hits from 2018 and even from 2019. I don’t know the date of the retraction but I feel pretty confident it was well before these dates.
Out of thousands of papers in ELT I could find only two that were withdrawn due to data issues. So either ELT is a paragon of virtue or we haven’t got very good at sniffing out fraud yet.
But how easy is it to get a journal to retract a paper? While researching an article on learning styles, I came across a couple of very curious articles. The first was Hyland 1993, the second was Hyland 1994. While reading the 1994 article I got a strange sense of Deja Vu:
In essence, learning style research suggests that people make sense of the world in different ways and these ways are partly created by cultural experiences (Hyland 1994)
Learning style research therefore suggests that people make sense of the world in different ways, more importantly however, these ways are partly created by cultural experiences (Hyland 1993)
Eight Japanese universities participated in the survey with 265 undergraduate students responding. The questionnaire was also administered to 140 Japanese students at various English proficiency levels at a tertiary college in New Zealand to determine whether overseas study influenced modality and group preferences. (Hyland 1993)
Eight Japanese universities participated in the survey with 265 undergraduates responding. The questionnaire was also administered to 140 Japanese students at different English proficiency levels at a tertiary college in New Zealand. (Hyland 1994)
Essentially the concept expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikes, but studies have addressed an enormously wide range of factors. (1993)
Learning style research expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikes, but people differ in their learning styles in a number of ways and studies have addressed a huge range of factors. (1994)
Most of the later (1994) article is a verbatim copy of the earlier one with minor phrasing adjustments such as those shown above. This type of thing is usually known as either self-plagiarism (see the examples above) or, on a smaller scale, text recycling and is considered unacceptable in academic publishing. Many journals have rules against it, such as JALT itself:
Papers sent to JALT journal should not have been previously published, nor should they be under consideration for publication elsewhere.
I thought this was a bit strange and so I contacted JALT, the 1994 publisher, to make them aware of the issue. They told me that they take such matters very seriously and would investigate. After an investigation they informed me that this was all just an honest mistake, a bit of a mix up. The paper had been submitted to two journals by accident and and as a previous editor had dealt with the matter, it would be wrong to retract the article now.
How one accidentally submits the same article to two journals is, I must confess, a mystery to me. More mysterious was JALT’s reasoning. Regardless of what a previous editor had decided, a repeat publication is still in the literature with no indication that it is a repeat.
I wrote back suggesting that since I was unaware that it had all been resolved, as was the current editor and presumably future readers, it might be worth retracting the article, or at least adding a note to explain what had happened. They told me they were very grateful for my suggestion but no, they weren’t going to do anything. And so both articles remain in the literature.
It is also odd that Hyland himself, a incredibly respected editor and prolific author would not want the article to be retracted. In fact, until recently he continued to list both papers among his publications (his new blog, however, only lists papers from 2003). Not retracting the paper may be less embarrassing in the short term but it means that there is always the chance for some annoying blogger to bring attention at some point, to what was a mix up .
In truth, I’m not ‘that’ surprised that nothing happened here. Brendan O’Connor a student at the University of Leicester discovered that a well respected psychologist was “recycling” parts of papers into new publications to an alarming degree. Although O’Connor has documented this to an impressive degree, some journals were reluctant to do anything at all when confronted with these findings.
As whistleblowers, data sleuths and anyone else who’s contacted a scientific journal or university with allegations of impropriety will tell you, getting even a demonstrably fraudulent paper retracted is a glacial process – and that’s if you aren’t simply ignored or fobbed off by the authorities in the first place.Science Fictions
Here are a random collection of things which I think are really important but which are not really evidence based.
5. Deal with behavioural issues as soon as possible
If you’re like me, then having a student on their phone or basically doing something distracting will put you off your stride. I think most of the time, with adult learners there’s no malice, they are just thinking about their own problems.
If something is bothering you in class then deal with it as soon as possible. If a student is bothering you, then it’s likely they are bothering other students in the class. They suffer and you suffer. I usually ask to see anyone who is doing something they shouldn’t after the class. I don’t think this has to be a difficult conversation, just “don’t do X”. Setting clear rules on day 1 is a big help.
4. use coloured paper with wide margins and light ink when making cards
It might seem a bit “TEFL” for an EAP teacher but it really saves time. If you are producing cut-ups then pre-make a document which has side margins wide enough that you don’t have cut them at all. Make the spaces between the sentences as large as possible because there is nothing worse than chopping a few words off one set. Finally, print each set on different coloured paper to make them easy to sort.
3. Everything usually takes longer than you think
I have a compulsive fear of running out of materials before the end of the lesson and I know that I am not alone. I did have a tendency to over plan for a long time and end up not getting through the material. I’m better at this now, and there are some easy ways to control the time in the lesson. These usually relate to how you give the answers (just write them all up and let the students check themselves or nominate people and have others confirm the answers). You can also, – shock horror – do activities twice if you find you have too much time. It might seem lazy, but the students will probably benefit from the practice.
(I only actually ran out of material twice in 20 years, which isn’t bad.)
2. Learn student names
I’m not great with names but I make sure that I always do this in the first class. It’s really cringy by the third lesson to be pointing and saying “you” and I think it makes the classes less effective. If I can’t learn them because there are too many students, or because I will only see the group a few times then I give them a name plate of some kind.
- Do the damn worksheet
This is a cardinal rule for me and yet the one that I observe the least, much to my detriment. Do the damn worksheet yourself. It’s only once you are in class that you realise that Q3 isn’t actually possible in passive, or that you can’t explain something about the grammar point you are supposed to be focusing on, or you don’t actually know why “pain” is countable in “what a pain”, or how to explain the word “innovative”, or why “try” takes infinitives and gerunds while”trying” only takes infinitives, and on and on for 20 years. It is only when you try to do the worksheet that you realise the planned activity is not physically possible or that it would take 2 hours to complete. Do the damn worksheet!
What are your basic teaching tips?