Friday, 29 March 2019

evidence based resources

So you want to be evidence based but don't know where to start! Here are a list of sites and resources which promote evidence in education for free! 

Summaries of research

Research bites is an excellent site which offers summaries of ELT and SLA research. THe site offers summaries of single papers in clear and accessible terms. There are a range of author and I believe the summary writers write to the article authors to check that they are happy with the summary. Anthony Schmidt runs the site and his own blog is worth a look too. 


The OASIS summaries page offers something very similar to research bites but is run by academics rather than teachers. They also offer advice about how to cite the summaries in your research. The summaries are in pdf form and can be download. The IRIS database also includes summaries of research and in addition to that offers research tools . The NCELP is another site which offers resources but for modern language teachers. 


Free access journals 

Should you want to read academic articles directly there are a few things you can do. There is increasingly a move towards open access in all kinds of publications and ELT is no different. This article on open access in ELT, is open access. It's written by Emma Marsden who is a big advocate for transparency in research. 


ELTjam featured a really nice article showing you which journals have free access and limited free access and these days most journals have something you can view for free. The article has a lot of great tips on getting hold of articles (legally) for free. Another thing you can try is writing to the author. With academics I've had a pretty good success rate when just emailing them and asking for a copy of papers. I think most of them are just overjoyed that someone wants to read their stuff. Disclaimer: I wouldn't try this with someone who makes their living selling reference books and the like. I very much doubt Scott Thornbury will email you a copy of 'the A-Z of ELT'. 

The British council and Cambridge (CUP) both offer some of their own research for free. You can get hold of quite a lo of good quality stuff just by browsing their sites. It should be noted that nothing in this post represents an endorsement of any of the research you find on these sites. For instance, the British Council site has a section on the dubious '21st century skills


Websites 

The education endowment Foundation also offers some summaries of research (though it is general education not ELT). The site also has reports on various areas of teaching. The site is very accessible and lays out information in a very accessible way


A couple more useful sites are 3 star learning and the learning scientist (the latter of which has an accompanying podcast). They both offer interesting articles on research in Education however the former seems to have no way of navigating the site. The learning scientist has some nice, clear downloadable resources. (Thanks to Anthony Schmidt for directing me to these two websites.)

blogs

There are a number of blogs which seek to present evidence in education. This blog, for instance has a 'try this it works' section which attempts to summarise research. Philip Kerr has some good stuff on translation and adaptive learningIn addition to this Greg Ashman's blog on teaching is usually well researched as is David Didau's 'learning spy' site. These last two are general education though. 


If I missed anything out please let me know and I'll update this page. 

Tuesday, 12 February 2019

Woo Watch: EdyouFest

I recently came across an education festival for TEFLers called 'EDUyoufest'. Plenary speaker Lonny Gold is presenting a talk on "Teaching WITH the brain instead of AGAINST it". Now whenever 'the brain' gets mentioned I do start to get a bit worried. Brain based approaches generally tend to come from the Romantic Humanist wing of the ELT world so I decided to investigate further and Gold did not disappoint. 


Gold, a Suggestopedia Master Trainer, has appeared in videos promoting the whacky teachings of Georgi Lozanov. Regular readers of this site will know that I have something of an obsession with all things Lozanov and so Gold instantly grabbed my attention. 

I managed to find a few of his articles and one of them in particular really impressed me. Most of lozanov's acolytes are cool with his claims of accelerated learning and suggestive states of mind. Yet there aren't many who are willing to follow the good doctor when he starts talking about using telepathy as a communication method. Gold is fearless though. He writes about a workshop presentation he held for the Liverpool SEAL conference:
The third and final segment of the workshop dealt with telepathy. In any open and nurturing environment, the telepathic connections between people are countless and to pretend they do not exist is silly and even irresponsible. In the case of teachers, a belief that what happens in class is not largely determined by the telepathic links within the group is either a dereliction of duty or - far worse - an admission that all form of human life has been successfully extinguished.
So there it is folks. Telepathy as a teaching tool. You heard it here first! 



Tuesday, 15 January 2019

The authenticity trap

Whenever I go to Japanese restaurants outside Japan, I'm always a bit annoyed to find they're entirely staffed by Chinese people. After spending years in Japan, I'd quite like authentic sushi lovingly crafted by authentic Japanese hands. Anything else is cultural appropriation, right?(I kid, but only a bit.)


Authenticity in an age of mass production is a valuable commodity. It also forms part of the communicative ideal of language teaching. We all like authenticity, but isn't it just discrimination to want Mexicans staffing my Mexican restaurant? Can't the Taiwanese or Japanese produce Whiskey just as well as Scots and Irish?




I'm sure a Mexican woman could make sushi just as well as any Japanese man. There's no logical reason this couldn't be the case. But there is something undeniably attractive about the idea that you're getting the 'real deal' -as fictional as that authenticity might turn out to be. 

This is one of the reasons I'm a little wary of shouting ‘racist’ at someone who expresses a preference for native speaker teachers. I can't help but wonder about my Chinese students. Arriving in Leicester, the first UK city to have a white minority, after being brought up on images of a fictional white Britain often leads to ignorant questions like 'where are all the British people?' For these students having possibly their first foreign experience, 'white' possibly equates to 'authentic'. 

This notion is backed up by research. Kiczkowiak points out, that often 'non-white native speaker teachers are rated less favourably on their pronunciation and teaching skills' than white NS teachers. What isn't always mentioned as much, is the flip side of this. I've noticed that white NNS may be given a pass by students.Their whiteness seemingly being enough to be deemed ‘authentic’.

'Native looking'
Kind hearted and loving



The notion that 'white' means 'authentic' is as silly as thinking someone Japanese will somehow be better at making sushi or Chinese people will be Kung Fu experts. Yet, the authenticity illusion is seductive. Even Kiczkowiak, known for his admirable work on NNS discrimination doesn't challenge it when it is presented in the right way. In a podcast episode his co-host and he, discussed the merits of retaining one’s accent when speaking a foreign language:
R: if I pick up the phone and someone's trying to sell me wine and they have a French accent immediately they're going to be much more credible, I'm going to believe that they know their wine, the wine that they're selling me is of a good quality because of our association between good wine and France...if you have a student in your class is going to be doing a job like that you're actually having a negative effect on their sales figures...and if you're a professor of ancient Rome and do you have an Italian accent that's going to be a really positive thing people are going to saying oh this person is from Rome...it's more believable that they know about this particular subject. (source
As I noted earlier, I think these associations are real, and part of the shortcuts brains employ to save energy. Want to know about wine? Ask a French person! Want to get great Sushi? Find someone from Japan. Want to Learn English? -fill in the mental shortcut students have for English speakers. That these stereotypes exist is undeniable. But as with all kinds of mental biases, we get smarter when we learn to recognise and discard them.

Monday, 31 December 2018

2018 wrap up and some thoughts on twitter

So farewell 2018! 

This year I managed to write 10 posts (including this one). I haven't been that prolific since 2015!


That said, not all the posts were written by me. January started with a guest post from Michael Griffin about the joys of Korea. I also introduced a new section to the blog called 'letters to the editor' and hope to have more of these. I unfortunately got a bit distract by Carol Black's defence of learning styles and ended up dedicating two whole posts on the subject. I could write two more but I think I've probably spent more than enough time on that. 

I also wrote a couple of posts on politics in the classroom the first of which got over 2,500 views at time of writing. I'm quite proud of that one. I also wrote a second blog post on suggestopedia and clearly, after the woeful number of views I must now accept that I am the only person who is interested in this topic at all

My most popular blog posts remain pretty much unchanged:


A rather exciting bit of news is that I got an article published with Carol Lethaby* which we started writing in 2015. Also I will be speaking in Ireland in 2019 which is exciting and nerve wracking in equal measures. That's pretty much it for me but I'd like to spend a bit of time in this post thinking out loud about twitter. 


some thoughts on twitter


I've been a big fan of twitter since I joined in 2012. It was fun and I liked the community aspect of it, particularly when I was living alone in Japan. Being able to talk directly to people who may have influenced you in various ways is great. Twitter also has the potential for massive impact. You could start an account tomorrow and assuming you had something interesting and novel to say, be talking to and possibly changing the minds of thousands of teachers by the end of the week. That's way more effective than academic articles or conference talks. Twitter is not all fun and games though. One ill-judged tweet, or even a comment taken the wrong way could mean career suicide or jail time without even leaving your house. 

This year I haven't enjoy twitter that much. In fact I think my enjoyment of it has been decreasing for a few years now. Particularly noticeably (to me anyway) is the amount of argumentativeness and snark. I thought perhaps this was just me being overly sensitive until I heard Mike Griffin make a similar point on a podcast recently



I think Mike is correct that we should perhaps view the period of niceness as the anomaly. If regular EduTwitter is anything to go by he's probably right. Hana Ticha has written about the "hostility" that people encounter on twitter and how it has led some to quit or think about quitting. There are, for me at least, just a handful of people who make it a less than pleasant experience but twitter has lots of tools, like 'mute', 'turn off retweets' and (in one case) 'block' which can remedy a lot of what is wrong with the site.

I mustn't exclude myself from this either. I probably (unconsciously) make other people's twitter experience unpleasant. 

I don't particularly like arguments online, and twitter has something of a multiplying factor in that you may feel in the 'spotlight' when discussing something on a 'public' platform and this can make people feel more defensive and aggressive. One writer notes that "tweeting is one of the most emotionally arousing activities you likely engage in on most days....studies show that tweeting raises your pulse, makes you sweat and enlarges your pupils -all indicators of arousal."



But more than the quality of some twitter interactions, IOS new 'screentime' function which tells you how long you spend on your phone has been quite eye opening for me. I didn't realise how much time I spent on twitter. Some days it is as much as 5 hours a day. Even if it's only an hour a day (and it rarely is) it's  hours which could be spent doing other things, like writing papers, reading books or just going outside. I can't claim to have 'no time' to get things done when I spend hours on twitter every day. 

I also find it harder and harder these days to concentrate enough to even read a book. As soon as I start I want to reach for my phone. After reading that other people seem to have the same issue, I've decided to take a break from social media. I'm not quitting and plan on still using twitter to post links to blog posts but I'm going to try to get out of the habit of daily interactions. why post this here and not just do it? Well, I'm hoping that posting it here will help to keep me honest. I don't know how long I'll last (3 months is my goal) but we'll see. 

Anyway, thanks for reading and I hope you all have a great 2019! 






Sunday, 16 December 2018

Woo watch: Science / fiction

I wrote about Carol Black's attempts to discredit opponents of learning styles as racists and sexists hereBlack's piece troubled me not just for it's unpleasant accusations but also because a number of sensible people told me they thought her argument was compelling. I think I understand why they think this. Black was a writer for The Ellen Show and 'The Wonder Years', both very successful US TV shows. She is clearly a very talented writer. I think that some commentators are possible confusing 'well written' with 'well argued'. These are very different things. 

There are at times some interesting observations in the piece but what there is is obscured by the poor reasoning Black employs to make her case. In fact Black's piece can be used to illustrate a number of well-known cognitive fallacies and techniques which are commonly used to make persuasive looking arguments in lieu of evidence. I will examine these in detail below. 

Teach the controversy


A popular technique of creationists who want to force their views on school kids despite not having any empirical support is to demand that teachers 'teach the controversy'. Unreasonable demands to include religion in science class are presented as merely teaching kids the 'full range of scientific views' and who could argue with that? The problem is that presenting things like climate change or evolution as a two sided debate is to seriously misrepresent the weight of evidence on each side. 


to be clear, I don't think what Black advocates is anything like creationism. The point is merely that the tactics are the same. Black wants us to teach the learning styles controversy. She accepts that there are "are extremely reputable scientists on both sides of this debate". She's just asking for a fair hearing -and who could object to that?

The problem is that Black has already made up her mind and no amount of evidence is going to convince her otherwise. So what we see in her post is the pulling on of any strand, no matter how unpleasant, in order to bolster her preconceived beliefs. In short, this is a masterclass in motivated reasoning

Despite claiming there are reputable scientists on both sides, she is happy to make the argument that 'debunkers' are mostly men talking down to women. She ignores the fact that her 1/3 of the panel of "respected scientists and education researchers" who agree with her, are also male. She also ignores female researchers, like Lethaby and Harris or Rogowsky, Calhoun and Tallalall women and all 'debunkers' of learning styles. This omission is particularity ironic in a section in which she is complaining about Willingham "failing to mention the existence of legitimate competing views."

So does the existence of researchers who still carry out research into learning styles mean it's wrong to say to say LS shouldn't be dismissed? There are two problems:


1) There are lots of researchers in education who continue to promote the idea of learning styles. In my field, for instance, some of the most famous in fact; Hyland, Oxford, Nunan, Brown, Richards and so on. There had been no articles in the ELT research literature which were critical of the idea of learning styles, until 2015 when researchers published a critical piece. since then pieces continue to be published promoting learning styles. The pro v anti count is, I would guess, about 100:1 at this point. This tells us exactly nothing about the veracity of the claims of these writers. 

2) Black implies that as some researchers have published on topic X then topic X is still up for debate. To try to show the problem with this idea, here are a few examples of recently published papers. 

  • Another paper suggesting graphology might help with depression. Again this was published in 2018 in a journal with an actual impact factor. Does this mean the debate about handwriting analysis is still ongoing? No, it does not. 
The homeopathy example is especially interesting in light of this tweet. 



Black bristles against the idea that LS should be considered on a par with  something like homeopathy yet the identical arguments she makes for LS could be made for homeopathy.  As noted above, legitimate researchers in good institutions continue to research homeopathy and publish their results in fairly reputable journals. 

So why is Black so dismissive? Why is she "failing to mention the existence of legitimate competing views" about homeopathy? The answer is that Black has an ideological investment in the idea of learning styles that she does not have in homeopathy. Science and evidence only matter to her when they can be usefully marshalled to defend things that align with her worldview. 

This is a frequent feature of Black's work. In another article she dismisses all the research evidence about phonics teaching because her home-schooled daughter didn't seem to like the approach. It should go without saying that anecdotal evidence is not good evidence. Black writes:  
The “scientific consensus” about phonics, generated by a panel convened by the Bush administration and used to justify billions of dollars in government contracts awarded to Bush supporters in the textbook and testing industries...
Science is "science" when Black disagrees with it. When in produces results she agrees with it becomes plain old science again. Note too in this quote that Black attempts to poison the well by linking phonics to the educational boo words of 'textbooks' and 'testing'. That Bush convened a panel and that billions of dollars were awarded to various companies tells us very little about whether the conclusions of the research were valid or not. I do not know very much about phonics research but if Black wanted to persuade me she was right, a few links to good research would do far more than innuendo and smear. Learning that Einstein was a racist does not mean E no longer equals Mc2. 

Argument from popularity 

Black claims that because a lot of people believe in learning styles there must be something to them. She writes that 'Most people believe they exist, of course (including the vast majority of teachers)' but does not provide any evidence for this claim. She is, as it turns out, correct as Dekker et al (2012) and others have shownBlack is not the first to make the claim that popularity indicates validity. Hatami and Stobart have both made that argument in ELT literature. 

The problem with this argument, as I've noted several times on this blog, is that believing something doesn't make it true and increasing the number of people who believe in a thing does nothing to increase the possibility that it is true. A lot of (the same) people who believe in learning styles, believe in left brain and right brain learning. They also believe that humans only use 10% of their brains. A huge number of people believe in the Christian God and an equal amount believe in Allah. They cannot both be right. Should we look into penis theft or Korean fan death if enough people believe in these phenomena? 

The retort to this is usually to claim that those things 'are different' somehow and learning styles is more credible. Those wishing to make that argument should therefore tell us exactly which popular views should be taken serious and which ones should not and what criteria we are to use to know the difference. Black could start by telling us why, despite its popularity, she is so dismissive of homeopathy? 


Argument from authority 

Another technique that Black employs is the argument from authority. Authors who agree with her are "respected scientists and education researchers" with "legitimate competing views". Whereas most of those who criticise learning styles are mentioned only by name, her favoured researchers are presented in their full academic pomp:
Li-Fang Zhang, editor-in-chief of the Oxford Encyclopedia of Education Psychology...
...And, as it happens, the Oxford Handbook of Cognitive Psychology chapter on "Cognitive Styles" by Harvard researcher Maria Kozhevnikov says the same thing. Researchers Carol Evans (University of Southampton), Elena Grigorenko (Yale), Stephen Kosslyn (Keck Graduate Institute), and Robert Sternberg (Cornell), agree.
No mention of  'Daniel Willingham (Professor of Psychology at the University of Virginia). Instead his ilk are "male researchers and pundits telling female educators that their views on learning are cognitively childish and irrational", "patronising" and speaking with a "paternal [sic] tone". The attempt to discredit learning style pundits by reference to their gender is of course also an ad hominem as their maleness (or whiteness or whatever) has no bearing on the truth of learning styles. 


Burden of proof 

Black attempts to dismiss Willingham and Pashler's work as being too simplistic and thus not capable of showing the reality of complex creativity classrooms. The issue with this position is the following:

  • Person A claims that learning styles are real and can help with students learning. 
  • Person B tests this claim and finds it false 
  • Person A says the tests were not sensitive enough to find the results 
  • Person B tests again with more sensitive tests and still finds it false
  • Person A says the tests were STILL not sensitive enough to find the results
Can you prove there isn't a teapot orbiting the sun? It's too small to be seen by telescopes but I'm sure it's there. Prove me wrong! This analogy is known as Russell's teapot and Bertrand Russell proposed it to show that the burden of proof rests with the person making the claim. At what point should person A accept the responsibility to provide evidence for the claim they are making? If no tool is sensitive enough to measure the effects of learning styles in the classroom on what basis can they be said to be useful?

Conclusion 

Black claims she wants to DISMANTLE arguments that learning styles are a myth. If she really wanted to do this she could have simply done, or linked to, good experimental research showing the effectiveness of the approach. Instead she has marshalled arguments from authority, popularity, and has attempted to discredit opponents with accusations of racism and sexism. What's clear is that research and academia are tools and props for Black to further promote her worldview. For her, citations are so much cargo cult academic decoration.  


When Black is building her case against 'debunkers' she writes that people like me argue that "cognitive biases, emotions, denial, irrationality, etc., are what prevent untrained people from accepting this conclusive body of scientific data." Ironically, in this case, she's absolutely right. 

Sunday, 18 November 2018

EBEFL asks 4: desperately seeking theory

I'm looking for your help with theory. Here are a few language learning incidences that have happened to me and I'm curious if anyone can associate any theory with what happened or can explain what was going on, or let me know if you had any similar experiences.

situation 1
I was in a shop in Taiwan and the woman serving me told me there was a 'zhekou'. I'd never heard this word before but I guessed (from context!!!) it meant discount. A few minutes later a friend confirmed that Zhekou was in fact a 'discount'. I don't think I ever encountered that word before and yet I learnt it instantly and never forgot it. 

Situation 2
I was one taking a Chinese class and said the word 'duo' meaning 'a lot'. I got the tone wrong and the tutor correct it with a recast. I nodded a thanks. I never got the tone of that word wrong again and almost every time I say it I can picture the tutor saying it. Is there an explanation for this 'insta-correction' and how effective it was? 

Situation 3
I kept hearing the word 'kichin to' on Japanese TV. I could use it in a sentence and knew that it generally came before a verb, but had no idea what it meant. I asked someone and it means something like 'properly' (do something properly/completely).  I could use this word correctly but didn't know what it meant. So what's going on here? 

Situation 4
I learnt the phrase "n ja nakatta" (東京に行くんじゃなかった) after living in Japan for a while and being able to speak Japanese fairly well. I don't remember hearing it before that but it seemed like absolutely everyone was using it after I learnt it. Is this just a case of recency illusion or is it a specific known thing in language learning? 


Wednesday, 26 September 2018

Woo watch: Baba Vanga

Baba Vanga
Baba Vanga was a blind Bulgarian mystic. She is quite well-known among people who are into the weird and wonderful world of 'parapsychology'. She's famous for her Nostradamus like predictions which had a '80% accuracy rate'. She is said to have predicted, among other things the 9/11 attacks, the election of a Black president and the 2004 boxing day Tsunami


Impressive stuff. Of course, like all good psychics there is quite a bit of artistic licence. And more importantly while the hits are counted, the misses are quietly forgotten. 2016 is over so we can say with some confidence that her prediction* that 'Europe will cease to exist' didn't come true (a good woo-master would somehow link this prediction to Brexit ). It's perhaps not surprising since she also predicted that 2010 would be when World War 3 started and that it would end in 2014, and that Muslims would wage war against Europe in 2013. So all in all, I don't think I'm being unfair when I say that we shouldn't take Baba Vanga very seriously. 


So what does Baba Vanga have to do with TEFL? The figure of 80% accuracy (60-70% here) in her predictions was reported by a group of scientists who worked for the Bulgarian institute of Suggestology and Parapsychology. The head of the institute was Georgi Lozanov who was the inventor of the TEFL method known as Suggestopedia

Lozanov said of Baba Vanga (source) "The stories about Vanga Dimitrova are not fantasies...She is extraordinarily talented....Vanga does read the future for those who go to her personally...she has psychic capabilities..." (p. 275). Lozanov also reveals that he has psychic power and was able to 'block' Vanga to some extent. (p. 276)

The genetic fallacy means we shouldn't write off an idea, just because of where it came from. However, in Lozanov's case I think we have been a bit too generous. The same research group which produced the amazing results on the effectiveness of suggestopedia also took a psychic seriously and produced 'scientific research' showing how effective a psychic she was. It isn't therefore that Lozanov had some whacky ideas but his research was solid. We have evidence that his research was extremely unreliable. 

SEAL - a lozanov inspired org.
All of this information was available in the 70's and yet Suggestopedia was generally treated fairly credulously. It receives serious coverage in works by Krashen, Larsen-Freeman, and many, many othersIn Tomlinson's 'Materials development in language teaching' a whole chapter is devoted to writing and grammar presentation in 'the Lozanov method.' The author Hansen, tells us that these days (1998) it's easier to understand here Lozanov was coming from since "quantum science has become more familiar" meaning we can perceive in "multidimensional" ways. Even today you can find published papers (here, here and here for instance) examining the effectiveness of the method and even the ELTJ recently had an article citing Lozanov

Baba Vanga died in 1996 but almost every year an article appears talking about one or more of her predictions and trying to link it to some current eventLozanov died in 2012 but his influence lives on in suggestopedia courses, books and in articles. Usually defenders of Suggestopedia say we should take the 'good stuff' and leave the rest. I suppose we could do that with Vanga too. I don't believe in seeing into the future or magic powers but suggestopedia does seems to have something of a charmed life and I don't predict that changing any time soon. 



*difficult to find reputable sources for these claims. Webpages tend to vanish when things don't come true

Sunday, 9 September 2018

When critical thinking is not critical thinking

Science and social justice
The strange case of Lindsay Shepherd and Laurier University hit the news in 2017. During one class in order to illustrate how gender pronouns have caused controversyShepherd, a 23 year old teaching assistant, showed a clip of Canadian Psychology professor Jordan Peterson. The clip was of a TV show in which he discussed his opposition to legally enforced gender pronoun use. 

After the class, a student (allegedly) complained about the video and the university launched an enquiry. Shepherd was asked to attend a meeting and was castigated by her employer for showing the video. The conversation, which Shepherd recorded, included this exchange: 

Rambukkana: So bringing something like that up in class, not critically, and I understand that you're trying to-
Shepherd: It was critical. I introduced it critically.
Rambukkana: Howso?
Shepherd: Like I said, it was in the spirit of debate.
Rabukkana: Okay, "In the spirit of debate" is slightly different than "This is a problematic idea that maybe we want to unpack"
Shepherd: But that's taking sides.

This conversation shows two competing version of the term 'critical' crashing into each other in real time. So how do these two version of 'critical' differ?

The 'critical' schools 
From the 1960's there was a flourishing of academic subjects using the term 'critical' in the title. These include but are not limited to such things as:
These subjects often seem to be concerned with similar things. For example, Critical Discourse Analysis focuses on:
the role of discourse in the (re)production and challenge of dominance. Dominance is defined here as the exercise of social power by elites, institutions or groups, that results in social inequality, including political, cultural, class, ethnic, racial and gender inequality.
Critical pedagogy is defined as
an approach to language teaching and learning which, according to Kincheloe (2005), is concerned with transforming relations of power which are oppressive and which lead to the oppression of people. It tries to humanize and empower learners...The major goal of CP, as Vandrick (1994) claims, is to emancipate and educate all people regardless of their gender, class, race, etc
Critical EAP similarly seeks to take account of factors previously ignored in EAP, like "gender, class, race and power relations..." (Benesch) The key themes, then of 'critical' fields are 1) power and oppression, 2) 'social justice' and 3) the notion of using academia to transform society. This is quite different from the usual sense of 'critical' in phrases like 'critical thinking'. Burbules and Berk suggest that the traditional sense of being critical:
...basically means to be more discerning in recognizing faulty arguments, hasty generalizations, assertions lacking evidence, truth claims based on unreliable authority, ambiguous or obscure concepts, and so forth.
I recently had a couple of papers published. One was titled 'a critical look at NLP in ELT' and the other 'A critical examination of perceptual learning styles in ELT'. Both of these papers use 'critical' in the sense of something akin to scientific skepticism. Questioning the veracity of claims, asking for evidence to support arguments and evaluating claims. I would guess this is what most people understand 'critical' to mean. 


The other 'critical' thinking 

In contrast, the 'critical' in Critical Pedagogy means something akin to 'Marxist'. Proponents can be a bit coy about this, but Scholem (in Hammersley) notes that after the Nazi takeover of Germany, Marxists of the Frankfurt school fled to the US, a country not particularly welcoming to Marxism. There they adopted the term 'critical' to describe the kind of research they were interested in. Freire's critical pedagogy is an example of this:
Freire’s philosophy was continuous with what has been euphemistically termed “western” Marxism, which embraces the quest for a sufficient theory of subjectivity identified in the post-war periods with the Critical Theory of the Frankfurt school, psychoanalysis, and phenomenology.” (Aronowitz)
Freire was a Marxist with a fondness for approvingly quoting Mao Tse Dong*. The Marxist roots are important to note because they represent the underpinnings or tenants of 'critical' subjects and include such things as: 
Both types of 'critical' would describe what they are doing as 'critical thinking' but this seems to be, in the critical theory sense a case of humpty-dumptying (after the character's insistence that 'When I use a word it means just what I choose it to mean'). Freire's definition of critical thinking, namely "thinking which discerns an indivisible solidarity between the world and the people and admits of no dichotomy between them" (92) is not one most people would recognise as 'critical thinking'. 

It's worth noting too, that those who advocate for critical approaches don't necessarily see a difference between the two forms of critical thinking. One is merely the logical conclusion of the other. If your analysis identifies a problem in the world, naturally you would work to fix it. That is to say, "The philosophers have only interpreted the world, in various ways. The point, however, is to change it"(Marx). 


What's the difference?

So how would a 'critical' article differ from a traditionally critical one? Recently a useful example popped up in my twitter feed. It's a critical look at the book 'Visible Learning' called 'Seven reasons to question the hegemony of Visible Learning'.  Those not aware of critical approaches might take this to be an examination of Hattie's arguments and the evidence supporting them, but the authors are very clear that that is not the case:
Critique of this program [...] has tended to centre on the mechanisms of meta-analysis. We consider what Visible Learning puts to work in relation to cultural politics and find it closely aligned with agendas of neoliberalism, sexism and ableism...
That is, they are not going to criticise Hattie for factual errors but rather for having the wrong ideology. The journal in which it is published, 'discourse studies in the cultural politics of education' may just sound like any other journal name but if we examine its scope we note that it:
adopts a broadly critical orientation, but is not tied to any particular ideological, disciplinary or methodological position. It encourages interdisciplinary approaches to the analysis of educational theory, policy and practice
Recent articles include 'Key elements in the naturalisation of neoliberal discourse in higher education in Chile' and 'Christmas in U.S. K-12 schools: categorizing and explaining teacher awareness of Christo-normativity'. 


There are a many critiques of Hattie's work, based on issues with his statistical analysis (see for example hereherehere and here) but as this is a critical paper, the focus is elsewhere. The authors are concerned that "colonising metaphors" like Visible Learning, with its focus on the "dominance of the visual" and "seductive neoliberal style" are in danger of becoming tools of "fascistic education". Visible learning is "sexist and masculinist" in it's pornographic "preoccupation with the visual" that forces a "feminized profession" (teaching) to submit to a "heteronormative, sexist and ableist" vision of education which revolves around “ejaculatory outcomes”. 

Being critical of 'critical' 

Science, when it works, is a self-correcting system (see for instance the recent replication crisis and trial registration). So we can ask, 'is this the right way to go about thinking about a problem?', 'is this the right problem to be thinking about?' 'Are these criticisms valid?', 'how can we tell?', and so on.  The critical academic subjects generally do not:  
it is characteristic of CDA, and of much 'critical' work in the social sciences, that its philosophical foundations are simply taken for granted, as if they were unproblematic. This reflects the fact that, in many ways, the term 'critical' has become little more than a rallying cry demanding that researchers consider 'whose side they are on'.”(1997:244)
The ideas central to the critical subjects cannot be challenged. We cannot, for example, ask if Freire is right that people are not currently 'fully human' and that praxis and inquiry would make them 'fully human'. Nor can we ask if it's useful to divide the world into oppressors and oppressed. In short, critical subjects are not, themselves, subject to criticism. 

When we do approach them critically we notice problems. For instance, the seemingly simplistic division of people into either oppressor or oppressed class. It's never exactly clear how a person finds themselves in one of these groups. 
Freire deals only in vague generalities. Oppression is never clearly defined. Freire concentrates on the oppression of the poor and fails to deal realistically with oppression as it is found at all levels of society. It is a mistake to see only the poor as oppressed and all others as oppressors. (Elias 1976)
Among Radical Feminists a woman would be a member of the oppressed class 'woman' and a victim of the 'patriarchy' system. However, the same woman, if she is white would, in critical race studies be a privileged member of the oppressor class in the system of 'white supremacy'.


If we start from the position that women are part of an oppressed class, then our research will tend to look for examples that support that narrative whereas a fact based approach may tend to throw up problematic data. For instance, a recent trend on twitter was for female PhD holders to affix 'dr' in front of their names. This was in response to a viral tweet from 'Sci Curious' about how male colleagues were far less likely than female colleagues to correctly address a female colleague. When the researcher actually checked her emails she found no difference. 

There is also an unfortunate tendency to characterise opponents as fascist or at least unwitting agents of fascism. For instance, in the meeting with Lindsay Shepherd, Professor Rambukkana (who's written on topics like 'From #RaceFail to #Ferguson: Digital Intimacies, Racism and the Politics of Hashtag Publics.' and 'Taking the Leather out of Leathersex: BDSM Identity and the Implications of an Internet-Mediated Sadomasochistic Public Sphere.') thought showing a clip of Peterson's was comparable with showing a clip of Hitler (a position for which he later apologised). Tying opponents' opinions to unsavoury movements like fascism can in some cases, be a substitute for refutation.  

Widdowson, responding to a critical paper, characterises such approaches as having an 'epistemological intolerance' noting that:
There is here a sort of fundamentalism: a zealous adherence to a way of conceiving of the world based on an unthinking trust in the wisdom of the pronouncements of some guru, sage, or prophet, whether this be Karl Marx or Thomas Aquinas or Ron Hubbard.
Finally it's not at all clear that critical approaches actually deliver on the promise of empowerment and liberation. One reporter noted that "for years I have been searching for an instance in which peasants have broken out of their oppression, but have found none. When I asked Freire he admitted that neither has he."

The spread of a critical approach

slides from RadicalKent EAP conference
Over the last couple of years I've noticed that this critical approach seems to be gaining more popularity in ELT and applied linguistics circles. Perhaps this is just a frequency illusion or perhaps these approaches are really starting to resonate with people due to the particular political situation we find ourselves in. 

Recently, The University of Kent hosted a 'RadicalEAP' event, which included talks on subjects such as 'Learning and teaching for the post-capitalist economy', 'How can I increase my impact as a teacher upon WP and BME students?' and 'Critical Race Theory (CRT): A framework for liberating, learning, teaching, assessment and the curriculum in higher education (HE)’'. 



'White knowledge' 
similarly, the AAAL conference this year seemed to have quite a 'critical' focus. For instance, echoing the 'OscarsSoWhite' trend of 2015 the hashtag AAALsowhite was promoted by Ryuko Kubota who spoke against 'white Eurocentric knowledge' and criticised the conference for not having more PoC speakers. Another speaker dealt with the question of whether or not applied linguistics is a 'tool of white supremacy'. 

not the same

Adopting a critical perspective can mean viewing the world through a restrictive lens. Teaching English becomes enforcing 'linguistic imperialism', which in turn is pushing Western values on oppressed people and is thus a tool of white supremacy (even when 'the oppressed' don't necessarily agree). 

There is also a real danger that as critical approaches becomes influential, research which discovers uncomfortable truths will be censored or suppressed. There is evidence that this is already happening (see here and here). Alice Dredger's book Galileo's Middle Finger documents a number of cases of this kind. She argues that Good research has "to put the search for truth first and the quest for social justice second”. 

I think it's possible to worry that women or PoC often suffer discrimination without believing that there is a systematic 'neoliberal' conspiracy at work to keep them under the boot. It's also possible to want to improve the world without assigning yourself either oppressor or oppressed status. As Widdowson puts it"you do not have to be a critical linguist to have a social conscience". 



*It has been pointed out to me that the wording of this is not quite accurate. Freire does seem to talk approvingly of Mao's China up to 1985 and never walks those comments back, but he doesn't actually quote Mao in the main body of Ped of Opp.