Wednesday 5 May 2021

Retraction in ELT

 I am currently reading the new book, Science Fictions by Stuart Ritchie. It deals with meta research (research about research) and outlines all the ways in which science is currently going wrong. In one section dealing with retraction, Ritchie notes that “1.97 per cent of scientists admit to faking their data at least once” and suggests that that number is probably an underestimate as people are unwilling to admit to things like this even when asked in anonymous surveys.

This number means that for every 50 papers published in ELT one is likely to contain faked data. Some of these cases have come to light in biology and psychology and this led me to wonder if there were many retractions in ELT. So I asked twitter.

Marc Jones instantly found two (here and here) both of which were retracted due to plagiarism. One was plagiarism of another scholars work and the other was self plagiarism (submitting the same paper for double credit). I also found one from The journal of Computer Assisted Language Learning retracted for (self) plagiarismSo far no fraud.

Stuart Ritchie pointed me to the retraction watch searchable database. I tried searching by journal and found a RELC paper published by Ivan Chong which was withdrawn for “significant data errors“. There was also The journal, System which had the misfortune to publish one article twice –in the same issue!

The journals Applied Linguistics, ELTJ and TESOL Quarterly have apparently had 0 retractions to date. The database is not complete though and I was also sent this piece which was retracted from the prestigious “language learning” journal. It’s not that clear what went on here but it seems like one author noticed errors in the data and requested a retraction.

As an interesting aside, Richie notes that people continue to cite retracted articles even after they have been retracted. I was curious about this so I used google scholar to check citations of the 2003 Language Learning article. There were hits from 2018 and even from 2019. I don’t know the date of the retraction but I feel pretty confident it was well before these dates.

Out of thousands of papers in ELT I could find only two that were withdrawn due to data issues. So either ELT is a paragon of virtue or we haven’t got very good at sniffing out fraud yet.

But how easy is it to get a journal to retract a paper? While researching an article on learning styles, I came across a couple of very curious articles. The first was Hyland 1993, the second was Hyland 1994. While reading the 1994 article I got a strange sense of Deja Vu:

In essence, learning style research suggests that people make sense of the world in different ways and these ways are partly created by cultural experiences (Hyland 1994)

Learning style research therefore suggests that people make sense of the world in different ways, more importantly however, these ways are partly created by cultural experiences (Hyland 1993)

and

Eight Japanese universities participated in the survey with 265 undergraduate students responding. The questionnaire was also administered to 140 Japanese students at various English proficiency levels at a tertiary college in New Zealand to determine whether overseas study influenced modality and group preferences. (Hyland 1993)

Eight Japanese universities participated in the survey with 265 undergraduates responding. The questionnaire was also administered to 140 Japanese students at different English proficiency levels at a tertiary college in New Zealand. (Hyland 1994)

and

Essentially the concept expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikesbut studies have addressed an enormously wide range of factors. (1993)

Learning style research expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikesbut people differ in their learning styles in a number of ways and studies have addressed a huge range of factors. (1994)

Most of the later (1994) article is a verbatim copy of the earlier one with minor phrasing adjustments such as those shown above. This type of thing is usually known as either self-plagiarism (see the examples above) or, on a smaller scale, text recycling and is considered unacceptable in academic publishing. Many journals have rules against it, such as JALT itself:

Papers sent to JALT journal should not have been previously published, nor should they be under consideration for publication elsewhere.

I thought this was a bit strange and so I contacted JALT, the 1994 publisher, to make them aware of the issue. They told me that they take such matters very seriously and would investigate. After an investigation they informed me that this was all just an honest mistake, a bit of a mix up. The paper had been submitted to two journals by accident and and as a previous editor had dealt with the matter, it would be wrong to retract the article now.

How one accidentally submits the same article to two journals is, I must confess, a mystery to me. More mysterious was JALT’s reasoning. Regardless of what a previous editor had decided, a repeat publication is still in the literature with no indication that it is a repeat.

I wrote back suggesting that since I was unaware that it had all been resolved, as was the current editor and presumably future readers, it might be worth retracting the article, or at least adding a note to explain what had happened. They told me they were very grateful for my suggestion but no, they weren’t going to do anything. And so both articles remain in the literature.

It is also odd that Hyland himself, a incredibly respected editor and prolific author would not want the article to be retracted. In fact, until recently he continued to list both papers among his publications (his new blog, however, only lists papers from 2003). Not retracting the paper may be less embarrassing in the short term but it means that there is always the chance for some annoying blogger to bring attention at some point, to what was a mix up .

In truth, I’m not ‘that’ surprised that nothing happened here. Brendan O’Connor a student at the University of Leicester discovered that a well respected psychologist was “recycling” parts of papers into new publications to an alarming degree. Although O’Connor has documented this to an impressive degree, some journals were reluctant to do anything at all when confronted with these findings.

As whistleblowers, data sleuths and anyone else who’s contacted a scientific journal or university with allegations of impropriety will tell you, getting even a demonstrably fraudulent paper retracted is a glacial process – and that’s if you aren’t simply ignored or fobbed off by the authorities in the first place.

Science Fictions

Well quite.

No comments:

Post a Comment