So long and thanks for all the fish!

10 years ago I posted my first post on this blog. 

It was a criticism of people taking the word "literally" too literally. Now, 10 years later, this is my final post. 

A lot has changed in ELT in the last 10 years. Learning styles and some of the other weird practices I criticised seem to be on their way out. This would be cause for celebration but I am worried that new whacky ideas have rushed in to fill the gap. I'm not sure that ELT is any more evidence-based now than it was 10 years ago. 

A lot has changed for me too. Since starting this blog I have written a book which was nominated for an ELTon, a number of papers and I have been invited to speak in a number of countries. I couldn't have imagined when I started the blog back in 2012 that anything like that would have happened but it's been a (mostly) fun experience and I have got to meet some really great people.

  

10 years in numbers

In 2012 I wrote a post celebrating the fact the blog hit 1,000 views. Since then it has been viewed more than half a million times. The most popular nation for views was the US followed, rather curiously, by Russia. The most common search term that led to the blog is xxxxxx (no doubt related to this). 

This post will be the 152nd post on this blog. 

So after 10 years what were the most popular posts of all time? 

10. Chris Smith's guest blog on oral error correction 

9. Leo Selivan on Misapplied linguistics 

8. The Myth of neat histories (one of my favourite posts) 

7. 2018 wrap up post (a rather odd one for the top 10) 

6. Is Korea the worst place to teach English

5. Learning Styles: Facts and Fictions

4. Philip Kerr on left and right brains in ELT

3. Skimming and Scanning 

2. Swearing shows a lack of intelligence? 

1. MA or DELTA: which to do? 

If you want to explore other posts on this blog then there is an incomplete index here. 

Unwritten blog posts 

I have a bad habit of starting blog posts but never finishing them. At present there are about 60 half written blog posts that seemed doomed to stay half written. So here are some of the delights I never got round to finishing. 

1. A post defending learning styles, as an intellectual exercise. 

2. A post on language and religion, with specific reference to Mormons. 

3. Something about the number of rejections I have had when submitting articles to journals and some things you could do to increase your chances of getting an article accepted. 

4. A long post on Krashen 

Note: I did a poll and many people replied that Krashen has been proved "generally right" about most things. The post was looking at how wrong he is about so many things, like teaching reading (he is seriously anti-science in his views), input only approaches, rejection of error correction as useful, and some of the weirder things he has claimed such as his views on bilingual education and how he treats suggestopedia as a serious approach. Krashen is charismatic, funny and persuasive and the closest thing in ELT to a real cult-of-personality Guru and I'm a little surprised academics give him as much credit as some do. 

5. A post on cancel culture in ELT

6. A post on creativity and if it can be taught

7. A whole series of posts on testing

8. A whole series of posts on Chomsky 

Note: I have so many notes but no complete posts. Chomsky is endlessly interesting but if you want to know more I suggest reading "The Linguistics Wars" edition 2 which is excellent on the subject OR Christina Behme's book on the subject which is an exceptional piece of writing.  

9. A post on bad science for good causes

Note: This will probably appear somewhere in some form. 

Finally! 

I want to say thank you to the few people who have approached me over the years at conferences and other events and said they enjoyed reading the blog. It really meant a lot and I appreciate it. 

I've had a strange relationship with twitter over the years. I have met some great people there and it's a great resource for academics to find each other and share work. It can also be incredibly toxic and has some bad incentives programmed into it. 

I will likely keep my twitter account but these days I find myself checking it less and less. I will lurk so feel free to send me a DM but don't be offended if I don't get back to you very quickly. 

Anyway, I am now off to work on a much bigger and more personal project that I expect to take up almost all of my time. 

Thanks for reading!

Russ Mayne

2023/1/1

Is there a replication crisis in ELT?

In a 2012 post I talked about the famous Pygmalion effect in education which shows that teachers expectations for students alone were enough to influence actual outcomes. It’s a truly amazing finding and minimizes the influence of home life, peer group and IQ. Rosenthal and Jacobson showed that, as I noted, "teachers' attitudes towards their students can affect students in quite remarkable ways". The authors were lauded and their paper has been cited over 12,000 times but there was just one problem, it probably wasn't true. 

As I noted in that original post, there were some failed attempts to replicate the results. What I didn't realise at the time is just how shaky the original research was (Thorndike wrote that the study “is so defective technically that one can only regret that it ever got beyond the eyes of the original investigators!”) and just what a controversial subject it has become (there is a nice overview here). 

The Pygmalion effect is not, however, an anomaly. In fact a host of famous psychological studies have come under scrutiny in recent years. The Stanford prison guard experiment, for example, has, to date, not been successfully replicated. Nor has the claim that holding a pencil in your teeth will make you feel happier by creating a "smile".  Did you hear the one about how priming people with adjectives relating to old age can make them walk like old people? That didn't replicate. Neither did the one in which thinking about professors makes you smarter. How about the one about how standing in a power pose makes you actually more powerful? You guessed it. 

In one incredible article with over 270 authors that attempts to replicate 100 studies in top ranking psychology journals, researchers found that only around one third to a half replicated. This finding is part of the replication crisis in science, which has involved fields from medicine to cognitive psychology. One paper notes that "Data on how much of the scientific literature is reproducible are rare and generally bleak." 

Bad news for science, but what's the situation like in ELT? 

The simple answer is that no one really knows as replication is vanishingly rare in SLA and applied linguistics. One of the people pushing for replication in ELT is Emma Marsden, co founder of the oasis database. In a comprehensive 2018 paper (which stretches to 70 pages) Marsden and colleagues investigate the state of replication in the field. They note that at present "little is known about replication in second language (L2) research". 

In the paper Marsden et al surveyed the literature for replications, finding 67, a number they describe as "very low". The authors estimate that for every 400 papers published, 1 gets replicated. Of the 67 papers they found around a third of the replications they looked at did not produce the same result as the original study. At a rough estimate then 30% of the research out there is wrong.  (Edit: Dan Isbell correctly points out that failure to replicate does not mean the original study was "wrong". As he notes "a single replication is not a final verdict.")

Replication can provide important insights for teachers. A replication of nations work on 432 for instance, showed that merely repeating the task, without shortening the time, led to the same increases in fluency as practice + time reduction. That means 4/4/4 is as effective as 4/3/2. Replication can also help by testing claims in non-"weird" populations (particularly useful for ELT). The lack of replication may explain the surprisingly low level of retraction in ELT. 

There are efforts to change the situation and increase the amount of replication. For instance, Language Learning is publishing a special "replication" edition.  The publication of this project is set of for June of this year and will make interesting reading. But as things stand the lack of replication in SLA should seriously worry those trying to argue for the importance of research in supporting teaching.