Wednesday, 26 July 2017

New Oriental debate

If you attended IATEFL or watched it online, you might remember a debate between Jim Scrivener and Jocelyn Wang of 'New Oriental' (NO) school in ChinaA recent edition of ELGazette features an interview with  Wang. According to the article:
British teacher-training guru Jim Scrivener was roundly defeated in his defence of Communicative Language Teaching by [Wang] who argued passionately and in perfect English for the benefits of traditional Chinese teaching methods
I'm not sure how the writer was able to ascertain who 'won' the debate, but to me, Wang's argument, that CLT was in principle a great idea, just not suitable for China, put me in mind of one of my childhood heroes, Jackie Chan. In 2009, Chan, who hails from Hong Kong, wondered aloud about his country's democratic future:
I'm not sure if it's good to have freedom or not...I'm gradually beginning to feel that we Chinese need to be controlled. If we're not being controlled, we'll just do what we want." 
In this post, I want to examine some of the arguments Wang made in favour of 'traditional Chinese teaching methods'. 

Traditional teaching methods with Chinese characteristics

As the trad/prog debate rages online and particularly on twitter, some have pointed to China's impressive ranking on the PISA tests and suggested that we could learn a thing or two about teaching from listening to Chinese educators who stick to very traditional teaching methods. In simplistic terms the approaches differ in the following ways:

This idea reached its culmination in a BBC 'experiment' (reality TV show) in which Chinese teachers were brought to England to teach British kids. (You can see an example of a rather bizarre grammar lesson here). 

The debate between Scrivener and Wang was the latest salvo in the ongoing prog/trad war. It wasn't billed as such, because progressive teaching has largely been victorious in the UK ELT world and what would be labelled 'progressive' is considered, by many teachers, as just 'teaching'. 

During the debate, and in her article, Wang makes several specific claims that push against the progressive ethos of UK ELT such as:
  • CLT doesn't really work in the Chinese context.
  • The communicative approach doesn't help students memorise language.
  • It is perfectly OK for teachers to speak entirely in Chinese in the lesson
  • Students do not need to speak in the lesson (i.e to practice the target language) 
All of these claims are specific and research to either back up or contradict Wang's claims could've been presented. For instance, if we believe that input is the only thing necessary for acquisition, it might be perfectly defensible to have a class in which students say nothing. In fact, CLT's application in Chinese classrooms has been examined by a number  authors (see for instance Yu 2001Liao 2004Hu, 2005) yet none of this research is referred to, instead Wang chose to argue that Chinese learners learn best when taught using a 'Chinese approach'. 

A Chinese-centric approach 

The New York Times reported that the Japanese "once tried to ban foreign-made skis because they were deemed unsuitable for Japan's ''unique'' snow". Anyone who has lived in Japan will be familiar with this kind of argument. Japanese Stomachs are unsuited to American beef (sorry we can't import any!) and so on. As stomachs digest, so brains learn. And as Long notes “the architecture of human brains varies very little among adults or among children”(2011:375). Yet when Scrivener points out that the kind of teaching promoted by Wang was "contrary to all contemporary theory, about how people learn languages", Wang shot back with "are [the studies] based on Chinese learners?" Perhaps, like left brain, or right brained learners, there are also Chinese brained learners?

As Wang is Chinese it seems difficult to argue with her 'insider knowledge', of what Chinese students needs. When progressive education tell us that that 'everyone learns in different ways', then it makes sense that Chinese students may learn in a 'Chinese way'. So we nod along as we're told "when Chinese learners learn anything, they value quantity" and "the whole idea of practice sits awkwardly with our view of learning". 

However, there are a few problems with what she presents as the Chinese approach to learning languages. Firstly, What she describes as 'Chinese learning' is the same 'transmission approach' of teaching which was common in many countries at one point and is still common in many classrooms. Secondly, is it unfair to point out how conveniently the homogeneous 'Chinese learner' she describes, desires the kind of teaching New Oriental offers?

A final criticism is that NO's methods bear no resemblance to those of another famous Chinese educator, Yang Li, creator of Crazy English. For while NO boasts "73 schools, 803 learning centres and 20,400 teachers in 61 cities across the country", Crazy English has over 20 million students. Unlike NO, Crazy English promotes massive amounts of student oral practice and somehow still manages to draw in huge numbers of students. Crazy English teachers conduct mass rallies with lots of chanting in English which seems odd as we are to believe "the whole idea of practice sits awkwardly with our view of learning". 

Ancient Traditional Chinese wisdom! 

Another line of defence employed by Wang was cultural and historical. She defended silent language classes by referring to an old Chinese proverb:

sān sī ér hòu xíng
Three think, then act. 

Which she translated as 'think 3 times before you speak' and made the point that China had a 5000 year old history and that the teaching style is Confucian in origin.  

This would be a bit like saying that the silent way is a good method because in English we say 'Silence is golden'. Actually, that would be a better proverb since the Chinese phrase she quoted would be better rendered as 'look before you leap' which is really unrelated to speaking in a language class. This is basically ideology disguised as best practice. Chinese people are different, the culture is different. Our ways are better because they're older (argument from antiquity), they come from Confucius (argument from authority). I have discussed the problem with arguments from authority here and it should be obvious but something being old is no guarantee it's any good. The same arguments are routinely used to defend questionable practices like traditional Chinese medicine and acupuncture.  

This post isn't a defence of CLT or an attack on traditional teaching. Wang is an eloquent speaker and it's good to hear a voice from one of the most populace countries in which English is taught. I think she and other Chinese teachers can give us an interesting insight into the Chinese context, but China and the Chinese are not monolithic and teaching practices shouldn't be defended with long dead philosophers or ancient wisdom. 

Saturday, 10 June 2017

Review of ELT podcasts part 3

When I started reviewing ELT podcasts there were hardly any. Now we find ourselves drowning in them! At present I count more than 10 ELT specific podcast. However, over half seem to have fallen to the wayside. Elliott's very good 'lives of teachers' podcast has very sporadic output these days. As does 'Masters of TESOL' which started strong and has since faded. The only three podcast that I have reviewed still regularly producing output are TEFLology, the TEFL show and the TEFL commute. Clearly the secret is having TEFL in your name somewhere. 

In my last review I had a wish list asking for, among other things, 'a podcast with a female host' and what do you know, three come along at once. 

1. One stop English podcast 

This podcast has only just started and is 8 episodes in but has made quite a nice start. Already the show has featured a debunking of learning styles, as well as featuring my former presentation partner Nicola Prentis, in the same episode. They have a 'guest teacher' slot, which is a nice idea and have so far featured, among others, the wonderful Natalia Guerreiro (who I cannot convince to write a guest blog post). In only 8 episodes they have had as guests, Hugh Dellar and Andrew Wakley, Adrian Underhill, Silvana Richardson and Scott Thornbury. This is quite a solid podcast, not too heavy and even including some practical teaching advice. It's a pleasant addition to the pod-o-sphere and it will be interesting to see how it develops. 

Tea with BVP has everything I have ever asked for in a podcast. It has a well established academic (Bill Van Patten) talking about language teaching research. It has veryhigh production value. It also has a NNS female host. The main host is also a bilingual Spanish speaker so we get insight into MFL. It also provides a fascinating window into the American ELT scene (lost since the minimal pair podcast disappeared). With all this going for it, why don't I love Tea with BVP more? I puzzled over this issue and it seems to me there are a few things which stop me enjoying this show more. 

Firstly, it's not a podcast. Sure, it is released in podcast form but it is recorded as a radio show and a radio show it is. There are phone ins, there are awkward pauses when no one phones in, there are some quite 'chatty' sections and so on. Secondly, it's very strongly wedded to a certain ideological position. I've listened to the whole 4(?) series and haven't yet been able to work out what this position is. It seems to be something along the lines of 'Krashen and Chomsky are right about everything' (I jest, but only a bit). 

One of the frustrating things about the show for me is that ideas about teaching are presented as settled science. That is, that doing X or Y is the only way students will acquire language and that language is acquired through method Z. There is nothing wrong with having a position and arguing from that position per se, I just wonder if say Long, or Ellis, would agree with BVP's take on language teaching. As a teacher with scant knowledge of the research discussed it's hard to know what to think. 

The certainty with which certain views were espoused looked a little less convincing when, in a recent episode BVP gave some credence to the idea of learning styles. In the following episode he responded to listener who had written in to challenge him on this (not me, I promise) and his response was a little disappointing. Rather than say 'yes, I got it wrong, learning styles aren't real.' he stated that individual differences don't matter much in learning languages. 

Early on I wrote to the show and asked them if they would detail alternate views to the one espoused. I was hoping to find out what their position would be defined as and what other researchers think. The show is usually very good at responding to people's questions on twitter and the like. They thanked me for my email but unfortunately this hasn't happened yet.

Thirdly, related to the last point, they favour a teaching methodology called TPRS which I had never heard of. I kept thinking it was a mutant variant of TPR, but no, it's something completely different. There are also frequent references to ACTFL which again, I had never heard of. But, it is interesting to learn that despite doing essentially the same job as these people, we seem to inhabit complete different worlds. TEA with BVP is a high quality podcast, but, for a British ELT teacher not familiar with the world of ACTFL, it can be a frustrating listen a times. 

This is a new and quite interesting little podcast. What I particularly like about it is that it seems to be set in China. The TEFL scene can be dominated by Spain/UK based teachers and so it's quite interesting to get a podcast from somewhere else. The hosts are a Ross Thorburn, a British guy and  Tracy Yu, a Chinese woman

There are about 24 episodes now and it's been around for less than a year, so the output is pretty high. The episodes are also really short at around 15 minutes each time. They generally tackle very general interest, practical issues like, monitoring, autonomy and materials. The format is usually the hosts (and perhaps a guest) reflecting on these topics. In that sense it's similar to other TEFL podcasts, but the Chinese perspective is interesting. 

So that's it! If you hear about a TEFL podcast (oh gawd, not another one!) please let me know. 

Other reviews of podcasts 

part 1

Part 2


Thursday, 4 May 2017

Student Feedback dos and don'ts

Which of the following do you think has the biggest impact on 'student evaluation of teaching' (SET) feedback?
  1. How hard the course is 
  2. The grade the student gets 
  3. The teacher’s gender 
  4. The teacher’s personality 
  5. How ‘hot’ the teacher is 


There is a ton of research into SETs starting over 80 years ago (Clayson 2009) and including, as of 1990 over 2000 articles (Feldman in Felder 1992). The literature includes several meta-analyses and even one meta-analysis of meta-analyses (Wright & Jenkins-Guarnieri 2012). In short, it is a well-researched field. Since university professors' careers can depend on these evaluations, perhaps this isn't surprising. Despite the large body of research (or perhaps because of it?) the science is not settled (Spooren et al 2013). 

There are however some general observations which can be made, reasonably confidently, about the effect of certain variables on SETs. So according to the literature* which factors have the biggest effect on student feedback? What follows is my hand list of dos and don'ts to improve your student feedback.

Do be likeable!

One of the variables which correlates highly with positive student feedback is personality and there is a substantial relationship between a teacher’s personality and the feedback they will be given (Feldman 1986Cardy and Dobbins 1986, Williams and Ceci 1997). Foote et al suggest that “[instructors] who score highly on evaluations may do so not because they teach well, but simply because they get along well with students” (2003:17). One researcher writes that personality is such a strong predictor of SET results that "the SET instrument could be replaced with a personality inventory with little loss of predictive validity” (Clayson online). 

There also seems to be something of a Halo effect at work with SETs. Basically, one positive attribute (Good looks) may cause people believe other positive things about a person (they are trustworthy, for instance). This is the reason handsome criminals get shorter prison sentences for the same crime than less attractive ones. This means that student opinions of personality might colour other variables and subsequently ‘likeable’ teachers may be judged positively in areas unrelated to ‘likeability’, such as teaching ability or professionalism. 

Does attraction affect scores?
This is problematic because it means the feedback you get will be tainted by the students general opinion of you. The picture on the left shows some feedback I recently received. Clearly the student had a high opinion of my teaching. Ho-hum.

The last column asks how useful the virtual self-access centre (VASC) was, the student has written 'very useful'. Now, being the teacher of the course, I can say with some confidence that I said not a word about the VSAC nor did any part of the course use the VSAC. Studies seem to corroborate this phenomenon showing that students are more than happy, to report false information to either reward of punish teachers (Clayson & Haley 2011).It should be noted that the Halo effect also works in reverse, so whatever happens, don't be disliked! 

Do be hot! 

Company promotes bribery
There is evidence that teachers who are perceived to be physically attractive tend to score more highly than their plainer colleagues. Riniolo et al (2006) found a 0.8 advantage on a 5 point scale for ‘hot’ teachers. After analysing the website, where teachers can be given a ‘hot’ rating, Felton et al (2004) found that ‘sexy’ teachers generally rated more highly than ‘non-sexy’ teachers. The authors note:

If these findings reflect the thinking of American college students when they complete in-class student opinion surveys, then universities need to rethink the validity of student opinion surveys as a measure of teaching effectiveness (91).

Do be expressive!

Despite various methodological flaws, the landmark ‘Dr. Fox’ studies (Naftulin et al. 1973), created interest in the question of the validity of SETs and what exactly it is that students are assessing when they complete feedback. In this study (see the actual study in the video below) an actor lectured a group of medical students with a largely meaningless talk that he had learnt the previous day. The student were told the speaker, Myron Fox was an expert in 'game theory'. 

The actor’s expressiveness and charm was seemingly enough for him to receive positive feedback from three separate audiences. Later researchers showed that even the meaningless talk was unnecessary. Ambady & Rosenthal's (1993) “thin slice” study asked students to evaluate teachers based on a silent 15 second clip of them teaching. The authors found a remarkable similarity between the term-end evaluations and those made after watching the short clips. 15 silent seconds was enough time to give an 'accurate' evaluation of the teacher. 

Do be a man!

Russell is annoying, his class is boring 
Researchers tend to agree that gender plays a minor role in overall evaluation. That is, one gender is not consistently rated lower than the other. In fact, “when significant differences were found, they generally favoured the female teacher” (Feldman in Pounder 2007). So what does 'be a man' mean? Well, despite this seeming equality, different genders may be rated on the basis of stereotyped views of gender (Laube et al 2007). For example, the most highly scoring men were described as ‘funny’ whereas the lowest scoring men were ‘boring’ in contrast the highest scoring women were ‘caring’ whereas the lowest scoring were either 'too smart' or 'not smart enough' or were simply a ‘bitch’ (Sprague & Massoni 2005).

There is also the question of whether a male teacher has to work as hard to get a top SET score as a female teacher. Women may suffer from the ‘Ginger Rogers effect’. That is "Ginger Rogers, one-half of the famous dance-team of 1930s movies, had to do everything Fred Astaire did, only she had to do it backwards and in high heels" (Sprague & Massoni 2005:791).  

Do grade generously!

There is a reasonably strong correlation between the grade, expected or real, and the type of feedback a teacher gets. This correlation can be summarised thus, “to put it succinctly, university teachers can buy ratings with grades” (Hocutt in Pounder 2007:185). 

The highest rated prof on
Clayson (online) notes that in his research 50% of students asked, admitted purposefully either lowering or inflating feedback grades as retribution or reward, and adds that whether or not grades actually affect scores is perhaps less important than whether faculty believe this to be the case as the belief is potentially enough to alter the way grades are given. Pounder backs this up noting “many university teachers believe that lenient grading produces higher SET scores and they tend to act on this belief” (Pounder 2007:185). However, It should be noted though that this is something of a controversial area with a large number of studies finding no relation between SET score and grades. (see Aleamoni 1999)

And if this isn't enough...

Here are a few more killer tips taken from the literature (Pounder 2007)

  • bribe students with food 
  • let students leave early 
  • praise the class on its ability before doing SETs 
  • do the SETs when the weak students are absent 
  • do a ‘fun activity’ before the SETs 
  • stay in the room 
  • teach small classes 

Not convinced yet? 

Here's a satisfied customer's testimony. From a remarkable paper published under the pen name name "A Great Teacher". This teacher, faced with the prospect of losing his job over poor SETs decided to throw out his morals and aim for good ratings. He stopped being such a 'tough' teacher and 'sucked up' to the students instead, making the course easy and trying to build rapport with his students:
What were the results of my experiment? The consequences for learning were not good. Students did less well than expected even on deliberately easy quizzes. Their final exam papers proved to be among the worst I had seen in years. Most students displayed only a superficial knowledge of the material. It was clear that some had concluded that with a kinder, gentler me, one didn’t need to work as hard. Although the pedagogical consequences were poor, the results for me were great! My [SET] scores went through the roof (2010:495-6)
And so, armed with this information, you too can become an well-loved teacher. Alternatively, you can treat student feedback with the caution it probably deserves.   

* Seldin (2010) suggests “one can find empirical support for any common allegation pertaining to student ratings” (in Hughes and Pate 2013:50). It's also worth noting that all of this research was carried out (like much research) on American University students. THere has been very little research on carried on in this are on FL students.

Sunday, 23 April 2017

Red flags: Assessing sources

This was originally posted on 'The Scarlet Onion' webpage but it has since ceased to exist :( 

I once had an online 'discussion' with a chap claiming that the twin towers were, in fact, brought down by the US government and that 9.11 was all an inside job. He sent me a link to, in his words, a ‘peer reviewed academic journal article’ to back this theory up. 

The link led to the ‘Journal of 9.11 studies. One of the editors was Kevin Ryan who coincidentally was also the article’s author. Editors do occasionally write articles for their own journals but still...I was dubious. So I googled Kevin Ryan and found that he’d written a book on why 9.11 was actually 'an inside job' and was fired from his previous university job his views. The article was about Nano-thermites (which he said the government used to blow up the twin towers), a type of explosive, and yet Ryan was working on water testing. Things didn’t add up. 

O Red Flags  
Imagine going to a restaurant and seeing no customers, the paint peeling and a smoke coming from the kitchen. None of this necessarily means the food there is bad, -hey it might be great, but these are all the kind of things which Dorothy Bishop refers to as ‘red flags’. These red flags can acts as a kind of early warning system. 

This brings me a conversation about the relative pros and cons of learning styles/MI and the Montessori Method I had on twitter earlier in the year. I was sent a link to a series of educational articles by a teacher who thought they supported her case. When I read them i noticed a number of red flags. I thought it would be useful to blog about these and hopefully this will be useful for other teachers assessing the quality of sources. So here are a few things to look out for.

O Mode of publication
Is the article in a book or on a website or in a journal? 

This matters because anyone can say anything in a book or on a website. Books often seem impressive as if somehow putting something in a book makes it more weighty and serious. Usually, academic journals, which have been through peer-review, are more likely to have credible information than books. Websites are almost always a no-no as there is usually zero quality control. Disclaimer (of course it always depends on exactly what you are looking for and what kind of website/book it is).

What about the papers that I was directed to look at? They seem to be on a website, which is an initial red flag. However, when we get to the articles in the download section they actually seem to be from real journals, namely, the MASAUM Journal of Reviews and Surveys (volume 1), MASAUM Journal of Open Problems in Science and Engineering (volume 1) and the International Journal of Engineering Research and Applications (volume 2); so far so good. But O (first red flag) why publish education articles in journals of engineering? 

O Dubious journals  
So these articles are in journals but what kind of journals?

Ideally the journals would be Peer-reviewed, and well established. The first two of these journals fail the ‘well-established’ test as they are all only seemingly on their second or first volume and are not available anywhere on the web. The third one actually does exist and looks (roughly) like a journal should. The site looks a bit cheap and unprofessional (Gmail address for submissions etc) but it exists, which is a start.

O Pay for play
A little bit of digging around the FAQ section of the third journal and we find this:

Q: How much do I have to pay for publication fee
A: $150

Paying for publication isn’t a great sign. As it’s an open-access journal (anyone can view the articles for free) this doesn’t necessarily mean it’s a bad thing, as long as the quality control isn’t affected. If we go to the ‘stats’ page though, we can see that they accept roughly 40% of all submissions. This is quite high considering journals like ‘Applied Linguistics’ accept only about 10% of submissions. It’s also notable that they record stats monthly and from those we can see they accept about 250 papers each month which means each volume has well over 300 articles. For context that’s about 7 years’ worth of ELTJ articles in one volume. 

Peer-review is sometimes mistakenly thought of as the ‘end’ of the process. But actually all peer review means is that a couple of other people in the profession have read it and think it’s good enough to be published. This doesn’t mean it’s perfect, or that no one can question it, just that it has reached a certain level of acceptability. After peer-review, the academic community at large get their teeth into and then we often see criticisms, repeat studies and sometimes retractions. The International Journal of Engineering Research and Applications is actually peer-reviewed so that seems pretty reassuring. Except, considering they boast that peer-review only takes 4-6 days (not the couple of months journals usually take) and because they read about 800 papers a month and only have about 27 reviewers, they are getting through those papers at an astonishing rate! You have to worry about quality control. 

O The author 
If you’re at all in doubt you can always check out the author’s credentials. According to his website Dr. Qais Faryadi (or ‘Dr. Prince’ as his website calls him) has a PhD in computer science and a master’s in Sharia Law as well as being an expert in curriculum design, criminal law, software engineering and Islam. He lectures on all these subjects and has even published a number of books (or rather ebooks) on such diverse topics as teaching, Islam and “Magnesium The Health Restorer: The Missing Link To Recovery”. 

O The nose-test
When you actually start reading the paper, does it sound plausible or like an academic paper should? 

Dr. Prince’s starts his learning styles paper with the statement that:

This evaluation examines teaching and learning from the lenses of mind blowing scholars such as David Kolb, (1984), Honey, (1982), Dick and Carey model (1990), Anthony, Sudbury Model, VAK Model and Madeline. (Faryadi, 2012:222)

Language like this should set alarm bells off instantly. This paper isn’t going to be an unbiased review of learning styles, not when the writer believes their creators to be ‘mind blowing’. Not only this, the article is littered with errors. In this small section alone we have such oddities as:

  •        ‘Honey’ should be ‘Honey and Mumford’ (as in the reference section),
  •        He starts by talking about ‘scholars’ but then switches to ‘models’
  •        Anthony? Who he?
  •        Madeline? Who she?

Wouldn’t peer-review usually sort errors like these out? Then again, with reviewers getting through 50 or so papers a month in 4-6 days each, it’s perhaps not surprising that the quality suffers.

The article itself is a relatively unremarkable laundry list of things the author believes ‘good teachers’ should do. There is no attempt to critically engage with the various learning styles models presented or to talk about why and how they differ from each other.

So in conclusion we have sloppy education articles written by an expert in computer science and law who claims expertise in a number of other fields. They were published in science and engineering journals that either no longer exist or have very little reputation. The one journal that does exist, charges for publication and has numerous credibility issues.

The food may be great here, it really might…but I’m going to look elsewhere.