Sunday, 16 December 2018

Woo watch: Science / fiction

I wrote about Carol Black's attempts to discredit opponents of learning styles as racists and sexists hereBlack's piece troubled me not just for it's unpleasant accusations but also because a number of sensible people told me they thought her argument was compelling. I think I understand why they think this. Black was a writer for The Ellen Show and 'The Wonder Years', both very successful US TV shows. She is clearly a very talented writer. I think that some commentators are possible confusing 'well written' with 'well argued'. These are very different things. 

There are at times some interesting observations in the piece but what there is is obscured by the poor reasoning Black employs to make her case. In fact Black's piece can be used to illustrate a number of well-known cognitive fallacies and techniques which are commonly used to make persuasive looking arguments in lieu of evidence. I will examine these in detail below. 

Teach the controversy


A popular technique of creationists who want to force their views on school kids despite not having any empirical support is to demand that teachers 'teach the controversy'. Unreasonable demands to include religion in science class are presented as merely teaching kids the 'full range of scientific views' and who could argue with that? The problem is that presenting things like climate change or evolution as a two sided debate is to seriously misrepresent the weight of evidence on each side. 


to be clear, I don't think what Black advocates is anything like creationism. The point is merely that the tactics are the same. Black wants us to teach the learning styles controversy. She accepts that there are "are extremely reputable scientists on both sides of this debate". She's just asking for a fair hearing -and who could object to that?

The problem is that Black has already made up her mind and no amount of evidence is going to convince her otherwise. So what we see in her post is the pulling on of any strand, no matter how unpleasant, in order to bolster her preconceived beliefs. In short, this is a masterclass in motivated reasoning

Despite claiming there are reputable scientists on both sides, she is happy to make the argument that 'debunkers' are mostly men talking down to women. She ignores the fact that her 1/3 of the panel of "respected scientists and education researchers" who agree with her, are also male. She also ignores female researchers, like Lethaby and Harris or Rogowsky, Calhoun and Tallalall women and all 'debunkers' of learning styles. This omission is particularity ironic in a section in which she is complaining about Willingham "failing to mention the existence of legitimate competing views."

So does the existence of researchers who still carry out research into learning styles mean it's wrong to say to say LS shouldn't be dismissed? There are two problems:


1) There are lots of researchers in education who continue to promote the idea of learning styles. In my field, for instance, some of the most famous in fact; Hyland, Oxford, Nunan, Brown, Richards and so on. There had been no articles in the ELT research literature which were critical of the idea of learning styles, until 2015 when researchers published a critical piece. since then pieces continue to be published promoting learning styles. The pro v anti count is, I would guess, about 100:1 at this point. This tells us exactly nothing about the veracity of the claims of these writers. 

2) Black implies that as some researchers have published on topic X then topic X is still up for debate. To try to show the problem with this idea, here are a few examples of recently published papers. 

  • Another paper suggesting graphology might help with depression. Again this was published in 2018 in a journal with an actual impact factor. Does this mean the debate about handwriting analysis is still ongoing? No, it does not. 
The homeopathy example is especially interesting in light of this tweet. 



Black bristles against the idea that LS should be considered on a par with  something like homeopathy yet the identical arguments she makes for LS could be made for homeopathy.  As noted above, legitimate researchers in good institutions continue to research homeopathy and publish their results in fairly reputable journals. 

So why is Black so dismissive? Why is she "failing to mention the existence of legitimate competing views" about homeopathy? The answer is that Black has an ideological investment in the idea of learning styles that she does not have in homeopathy. Science and evidence only matter to her when they can be usefully marshalled to defend things that align with her worldview. 

This is a frequent feature of Black's work. In another article she dismisses all the research evidence about phonics teaching because her home-schooled daughter didn't seem to like the approach. It should go without saying that anecdotal evidence is not good evidence. Black writes:  
The “scientific consensus” about phonics, generated by a panel convened by the Bush administration and used to justify billions of dollars in government contracts awarded to Bush supporters in the textbook and testing industries...
Science is "science" when Black disagrees with it. When in produces results she agrees with it becomes plain old science again. Note too in this quote that Black attempts to poison the well by linking phonics to the educational boo words of 'textbooks' and 'testing'. That Bush convened a panel and that billions of dollars were awarded to various companies tells us very little about whether the conclusions of the research were valid or not. I do not know very much about phonics research but if Black wanted to persuade me she was right, a few links to good research would do far more than innuendo and smear. Learning that Einstein was a racist does not mean E no longer equals Mc2. 

Argument from popularity 

Black claims that because a lot of people believe in learning styles there must be something to them. She writes that 'Most people believe they exist, of course (including the vast majority of teachers)' but does not provide any evidence for this claim. She is, as it turns out, correct as Dekker et al (2012) and others have shownBlack is not the first to make the claim that popularity indicates validity. Hatami and Stobart have both made that argument in ELT literature. 

The problem with this argument, as I've noted several times on this blog, is that believing something doesn't make it true and increasing the number of people who believe in a thing does nothing to increase the possibility that it is true. A lot of (the same) people who believe in learning styles, believe in left brain and right brain learning. They also believe that humans only use 10% of their brains. A huge number of people believe in the Christian God and an equal amount believe in Allah. They cannot both be right. Should we look into penis theft or Korean fan death if enough people believe in these phenomena? 

The retort to this is usually to claim that those things 'are different' somehow and learning styles is more credible. Those wishing to make that argument should therefore tell us exactly which popular views should be taken serious and which ones should not and what criteria we are to use to know the difference. Black could start by telling us why, despite its popularity, she is so dismissive of homeopathy? 


Argument from authority 

Another technique that Black employs is the argument from authority. Authors who agree with her are "respected scientists and education researchers" with "legitimate competing views". Whereas most of those who criticise learning styles are mentioned only by name, her favoured researchers are presented in their full academic pomp:
Li-Fang Zhang, editor-in-chief of the Oxford Encyclopedia of Education Psychology...
...And, as it happens, the Oxford Handbook of Cognitive Psychology chapter on "Cognitive Styles" by Harvard researcher Maria Kozhevnikov says the same thing. Researchers Carol Evans (University of Southampton), Elena Grigorenko (Yale), Stephen Kosslyn (Keck Graduate Institute), and Robert Sternberg (Cornell), agree.
No mention of  'Daniel Willingham (Professor of Psychology at the University of Virginia). Instead his ilk are "male researchers and pundits telling female educators that their views on learning are cognitively childish and irrational", "patronising" and speaking with a "paternal [sic] tone". The attempt to discredit learning style pundits by reference to their gender is of course also an ad hominem as their maleness (or whiteness or whatever) has no bearing on the truth of learning styles. 


Burden of proof 

Black attempts to dismiss Willingham and Pashler's work as being too simplistic and thus not capable of showing the reality of complex creativity classrooms. The issue with this position is the following:

  • Person A claims that learning styles are real and can help with students learning. 
  • Person B tests this claim and finds it false 
  • Person A says the tests were not sensitive enough to find the results 
  • Person B tests again with more sensitive tests and still finds it false
  • Person A says the tests were STILL not sensitive enough to find the results
Can you prove there isn't a teapot orbiting the sun? It's too small to be seen by telescopes but I'm sure it's there. Prove me wrong! This analogy is known as Russell's teapot and Bertrand Russell proposed it to show that the burden of proof rests with the person making the claim. At what point should person A accept the responsibility to provide evidence for the claim they are making? If no tool is sensitive enough to measure the effects of learning styles in the classroom on what basis can they be said to be useful?

Conclusion 

Black claims she wants to DISMANTLE arguments that learning styles are a myth. If she really wanted to do this she could have simply done, or linked to, good experimental research showing the effectiveness of the approach. Instead she has marshalled arguments from authority, popularity, and has attempted to discredit opponents with accusations of racism and sexism. What's clear is that research and academia are tools and props for Black to further promote her worldview. For her, citations are so much cargo cult academic decoration.  


When Black is building her case against 'debunkers' she writes that people like me argue that "cognitive biases, emotions, denial, irrationality, etc., are what prevent untrained people from accepting this conclusive body of scientific data." Ironically, in this case, she's absolutely right. 

Sunday, 18 November 2018

EBEFL asks 4: desperately seeking theory

I'm looking for your help with theory. Here are a few language learning incidences that have happened to me and I'm curious if anyone can associate any theory with what happened or can explain what was going on, or let me know if you had any similar experiences.

situation 1
I was in a shop in Taiwan and the woman serving me told me there was a 'zhekou'. I'd never heard this word before but I guessed (from context!!!) it meant discount. A few minutes later a friend confirmed that Zhekou was in fact a 'discount'. I don't think I ever encountered that word before and yet I learnt it instantly and never forgot it. 

Situation 2
I was one taking a Chinese class and said the word 'duo' meaning 'a lot'. I got the tone wrong and the tutor correct it with a recast. I nodded a thanks. I never got the tone of that word wrong again and almost every time I say it I can picture the tutor saying it. Is there an explanation for this 'insta-correction' and how effective it was? 

Situation 3
I kept hearing the word 'kichin to' on Japanese TV. I could use it in a sentence and knew that it generally came before a verb, but had no idea what it meant. I asked someone and it means something like 'properly' (do something properly/completely).  I could use this word correctly but didn't know what it meant. So what's going on here? 

Situation 4
I learnt the phrase "n ja nakatta" (東京に行くんじゃなかった) after living in Japan for a while and being able to speak Japanese fairly well. I don't remember hearing it before that but it seemed like absolutely everyone was using it after I learnt it. Is this just a case of recency illusion or is it a specific known thing in language learning? 


Wednesday, 26 September 2018

Woo watch: Baba Vanga

Baba Vanga
Baba Vanga was a blind Bulgarian mystic. She is quite well-known among people who are into the weird and wonderful world of 'parapsychology'. She's famous for her Nostradamus like predictions which had a '80% accuracy rate'. She is said to have predicted, among other things the 9/11 attacks, the election of a Black president and the 2004 boxing day Tsunami


Impressive stuff. Of course, like all good psychics there is quite a bit of artistic licence. And more importantly while the hits are counted, the misses are quietly forgotten. 2016 is over so we can say with some confidence that her prediction* that 'Europe will cease to exist' didn't come true (a good woo-master would somehow link this prediction to Brexit ). It's perhaps not surprising since she also predicted that 2010 would be when World War 3 started and that it would end in 2014, and that Muslims would wage war against Europe in 2013. So all in all, I don't think I'm being unfair when I say that we shouldn't take Baba Vanga very seriously. 


So what does Baba Vanga have to do with TEFL? The figure of 80% accuracy (60-70% here) in her predictions was reported by a group of scientists who worked for the Bulgarian institute of Suggestology and Parapsychology. The head of the institute was Georgi Lozanov who was the inventor of the TEFL method known as Suggestopedia

Lozanov said of Baba Vanga (source) "The stories about Vanga Dimitrova are not fantasies...She is extraordinarily talented....Vanga does read the future for those who go to her personally...she has psychic capabilities..." (p. 275). Lozanov also reveals that he has psychic power and was able to 'block' Vanga to some extent. (p. 276)

The genetic fallacy means we shouldn't write off an idea, just because of where it came from. However, in Lozanov's case I think we have been a bit too generous. The same research group which produced the amazing results on the effectiveness of suggestopedia also took a psychic seriously and produced 'scientific research' showing how effective a psychic she was. It isn't therefore that Lozanov had some whacky ideas but his research was solid. We have evidence that his research was extremely unreliable. 

SEAL - a lozanov inspired org.
All of this information was available in the 70's and yet Suggestopedia was generally treated fairly credulously. It receives serious coverage in works by Krashen, Larsen-Freeman, and many, many othersIn Tomlinson's 'Materials development in language teaching' a whole chapter is devoted to writing and grammar presentation in 'the Lozanov method.' The author Hansen, tells us that these days (1998) it's easier to understand here Lozanov was coming from since "quantum science has become more familiar" meaning we can perceive in "multidimensional" ways. Even today you can find published papers (here, here and here for instance) examining the effectiveness of the method and even the ELTJ recently had an article citing Lozanov

Baba Vanga died in 1996 but almost every year an article appears talking about one or more of her predictions and trying to link it to some current eventLozanov died in 2012 but his influence lives on in suggestopedia courses, books and in articles. Usually defenders of Suggestopedia say we should take the 'good stuff' and leave the rest. I suppose we could do that with Vanga too. I don't believe in seeing into the future or magic powers but suggestopedia does seems to have something of a charmed life and I don't predict that changing any time soon. 



*difficult to find reputable sources for these claims. Webpages tend to vanish when things don't come true

Sunday, 9 September 2018

When critical thinking is not critical thinking

Science and social justice
The strange case of Lindsay Shepherd and Laurier University hit the news in 2017. During one class in order to illustrate how gender pronouns have caused controversyShepherd, a 23 year old teaching assistant, showed a clip of Canadian Psychology professor Jordan Peterson. The clip was of a TV show in which he discussed his opposition to legally enforced gender pronoun use. 

After the class, a student (allegedly) complained about the video and the university launched an enquiry. Shepherd was asked to attend a meeting and was castigated by her employer for showing the video. The conversation, which Shepherd recorded, included this exchange: 

Rambukkana: So bringing something like that up in class, not critically, and I understand that you're trying to-
Shepherd: It was critical. I introduced it critically.
Rambukkana: Howso?
Shepherd: Like I said, it was in the spirit of debate.
Rabukkana: Okay, "In the spirit of debate" is slightly different than "This is a problematic idea that maybe we want to unpack"
Shepherd: But that's taking sides.

This conversation shows two competing version of the term 'critical' crashing into each other in real time. So how do these two version of 'critical' differ?

The 'critical' schools 
From the 1960's there was a flourishing of academic subjects using the term 'critical' in the title. These include but are not limited to such things as:
These subjects often seem to be concerned with similar things. For example, Critical Discourse Analysis focuses on:
the role of discourse in the (re)production and challenge of dominance. Dominance is defined here as the exercise of social power by elites, institutions or groups, that results in social inequality, including political, cultural, class, ethnic, racial and gender inequality.
Critical pedagogy is defined as
an approach to language teaching and learning which, according to Kincheloe (2005), is concerned with transforming relations of power which are oppressive and which lead to the oppression of people. It tries to humanize and empower learners...The major goal of CP, as Vandrick (1994) claims, is to emancipate and educate all people regardless of their gender, class, race, etc
Critical EAP similarly seeks to take account of factors previously ignored in EAP, like "gender, class, race and power relations..." (Benesch) The key themes, then of 'critical' fields are 1) power and oppression, 2) 'social justice' and 3) the notion of using academia to transform society. This is quite different from the usual sense of 'critical' in phrases like 'critical thinking'. Burbules and Berk suggest that the traditional sense of being critical:
...basically means to be more discerning in recognizing faulty arguments, hasty generalizations, assertions lacking evidence, truth claims based on unreliable authority, ambiguous or obscure concepts, and so forth.
I recently had a couple of papers published. One was titled 'a critical look at NLP in ELT' and the other 'A critical examination of perceptual learning styles in ELT'. Both of these papers use 'critical' in the sense of something akin to scientific skepticism. Questioning the veracity of claims, asking for evidence to support arguments and evaluating claims. I would guess this is what most people understand 'critical' to mean. 


The other 'critical' thinking 

In contrast, the 'critical' in Critical Pedagogy means something akin to 'Marxist'. Proponents can be a bit coy about this, but Scholem (in Hammersley) notes that after the Nazi takeover of Germany, Marxists of the Frankfurt school fled to the US, a country not particularly welcoming to Marxism. There they adopted the term 'critical' to describe the kind of research they were interested in. Freire's critical pedagogy is an example of this:
Freire’s philosophy was continuous with what has been euphemistically termed “western” Marxism, which embraces the quest for a sufficient theory of subjectivity identified in the post-war periods with the Critical Theory of the Frankfurt school, psychoanalysis, and phenomenology.” (Aronowitz)
Freire was a Marxist with a fondness for approvingly quoting Mao Tse Dong*. The Marxist roots are important to note because they represent the underpinnings or tenants of 'critical' subjects and include such things as: 
Both types of 'critical' would describe what they are doing as 'critical thinking' but this seems to be, in the critical theory sense a case of humpty-dumptying (after the character's insistence that 'When I use a word it means just what I choose it to mean'). Freire's definition of critical thinking, namely "thinking which discerns an indivisible solidarity between the world and the people and admits of no dichotomy between them" (92) is not one most people would recognise as 'critical thinking'. 

It's worth noting too, that those who advocate for critical approaches don't necessarily see a difference between the two forms of critical thinking. One is merely the logical conclusion of the other. If your analysis identifies a problem in the world, naturally you would work to fix it. That is to say, "The philosophers have only interpreted the world, in various ways. The point, however, is to change it"(Marx). 


What's the difference?

So how would a 'critical' article differ from a traditionally critical one? Recently a useful example popped up in my twitter feed. It's a critical look at the book 'Visible Learning' called 'Seven reasons to question the hegemony of Visible Learning'.  Those not aware of critical approaches might take this to be an examination of Hattie's arguments and the evidence supporting them, but the authors are very clear that that is not the case:
Critique of this program [...] has tended to centre on the mechanisms of meta-analysis. We consider what Visible Learning puts to work in relation to cultural politics and find it closely aligned with agendas of neoliberalism, sexism and ableism...
That is, they are not going to criticise Hattie for factual errors but rather for having the wrong ideology. The journal in which it is published, 'discourse studies in the cultural politics of education' may just sound like any other journal name but if we examine its scope we note that it:
adopts a broadly critical orientation, but is not tied to any particular ideological, disciplinary or methodological position. It encourages interdisciplinary approaches to the analysis of educational theory, policy and practice
Recent articles include 'Key elements in the naturalisation of neoliberal discourse in higher education in Chile' and 'Christmas in U.S. K-12 schools: categorizing and explaining teacher awareness of Christo-normativity'. 


There are a many critiques of Hattie's work, based on issues with his statistical analysis (see for example hereherehere and here) but as this is a critical paper, the focus is elsewhere. The authors are concerned that "colonising metaphors" like Visible Learning, with its focus on the "dominance of the visual" and "seductive neoliberal style" are in danger of becoming tools of "fascistic education". Visible learning is "sexist and masculinist" in it's pornographic "preoccupation with the visual" that forces a "feminized profession" (teaching) to submit to a "heteronormative, sexist and ableist" vision of education which revolves around “ejaculatory outcomes”. 

Being critical of 'critical' 

Science, when it works, is a self-correcting system (see for instance the recent replication crisis and trial registration). So we can ask, 'is this the right way to go about thinking about a problem?', 'is this the right problem to be thinking about?' 'Are these criticisms valid?', 'how can we tell?', and so on.  The critical academic subjects generally do not:  
it is characteristic of CDA, and of much 'critical' work in the social sciences, that its philosophical foundations are simply taken for granted, as if they were unproblematic. This reflects the fact that, in many ways, the term 'critical' has become little more than a rallying cry demanding that researchers consider 'whose side they are on'.”(1997:244)
The ideas central to the critical subjects cannot be challenged. We cannot, for example, ask if Freire is right that people are not currently 'fully human' and that praxis and inquiry would make them 'fully human'. Nor can we ask if it's useful to divide the world into oppressors and oppressed. In short, critical subjects are not, themselves, subject to criticism. 

When we do approach them critically we notice problems. For instance, the seemingly simplistic division of people into either oppressor or oppressed class. It's never exactly clear how a person finds themselves in one of these groups. 
Freire deals only in vague generalities. Oppression is never clearly defined. Freire concentrates on the oppression of the poor and fails to deal realistically with oppression as it is found at all levels of society. It is a mistake to see only the poor as oppressed and all others as oppressors. (Elias 1976)
Among Radical Feminists a woman would be a member of the oppressed class 'woman' and a victim of the 'patriarchy' system. However, the same woman, if she is white would, in critical race studies be a privileged member of the oppressor class in the system of 'white supremacy'.


If we start from the position that women are part of an oppressed class, then our research will tend to look for examples that support that narrative whereas a fact based approach may tend to throw up problematic data. For instance, a recent trend on twitter was for female PhD holders to affix 'dr' in front of their names. This was in response to a viral tweet from 'Sci Curious' about how male colleagues were far less likely than female colleagues to correctly address a female colleague. When the researcher actually checked her emails she found no difference. 

There is also an unfortunate tendency to characterise opponents as fascist or at least unwitting agents of fascism. For instance, in the meeting with Lindsay Shepherd, Professor Rambukkana (who's written on topics like 'From #RaceFail to #Ferguson: Digital Intimacies, Racism and the Politics of Hashtag Publics.' and 'Taking the Leather out of Leathersex: BDSM Identity and the Implications of an Internet-Mediated Sadomasochistic Public Sphere.') thought showing a clip of Peterson's was comparable with showing a clip of Hitler (a position for which he later apologised). Tying opponents' opinions to unsavoury movements like fascism can in some cases, be a substitute for refutation.  

Widdowson, responding to a critical paper, characterises such approaches as having an 'epistemological intolerance' noting that:
There is here a sort of fundamentalism: a zealous adherence to a way of conceiving of the world based on an unthinking trust in the wisdom of the pronouncements of some guru, sage, or prophet, whether this be Karl Marx or Thomas Aquinas or Ron Hubbard.
Finally it's not at all clear that critical approaches actually deliver on the promise of empowerment and liberation. One reporter noted that "for years I have been searching for an instance in which peasants have broken out of their oppression, but have found none. When I asked Freire he admitted that neither has he."

The spread of a critical approach

slides from RadicalKent EAP conference
Over the last couple of years I've noticed that this critical approach seems to be gaining more popularity in ELT and applied linguistics circles. Perhaps this is just a frequency illusion or perhaps these approaches are really starting to resonate with people due to the particular political situation we find ourselves in. 

Recently, The University of Kent hosted a 'RadicalEAP' event, which included talks on subjects such as 'Learning and teaching for the post-capitalist economy', 'How can I increase my impact as a teacher upon WP and BME students?' and 'Critical Race Theory (CRT): A framework for liberating, learning, teaching, assessment and the curriculum in higher education (HE)’'. 



'White knowledge' 
similarly, the AAAL conference this year seemed to have quite a 'critical' focus. For instance, echoing the 'OscarsSoWhite' trend of 2015 the hashtag AAALsowhite was promoted by Ryuko Kubota who spoke against 'white Eurocentric knowledge' and criticised the conference for not having more PoC speakers. Another speaker dealt with the question of whether or not applied linguistics is a 'tool of white supremacy'. 

not the same

Adopting a critical perspective can mean viewing the world through a restrictive lens. Teaching English becomes enforcing 'linguistic imperialism', which in turn is pushing Western values on oppressed people and is thus a tool of white supremacy (even when 'the oppressed' don't necessarily agree). 

There is also a real danger that as critical approaches becomes influential, research which discovers uncomfortable truths will be censored or suppressed. There is evidence that this is already happening (see here and here). Alice Dredger's book Galileo's Middle Finger documents a number of cases of this kind. She argues that Good research has "to put the search for truth first and the quest for social justice second”. 

I think it's possible to worry that women or PoC often suffer discrimination without believing that there is a systematic 'neoliberal' conspiracy at work to keep them under the boot. It's also possible to want to improve the world without assigning yourself either oppressor or oppressed status. As Widdowson puts it"you do not have to be a critical linguist to have a social conscience". 



*It has been pointed out to me that the wording of this is not quite accurate. Freire does seem to talk approvingly of Mao's China up to 1985 and never walks those comments back, but he doesn't actually quote Mao in the main body of Ped of Opp.