I really suspect this is comparatively meaningless

11 Feb

Andrew Leigh and Chris Ryan are certainly busy econometrists. (Unfortunately at the moment the web sites at ANU of both are not accessible.) Their thing in recent years has been the decline in the quality of those seeking to become teachers. For example:

…we were able to track entry scores at one of Australia’s most distinguished universities, the University of Sydney. In 1977, the cut-off for entry into a bachelor of education (365 out of 500) was nearly as high as law (390), and well above our own discipline of economics (284). But in 2005 the cut-off for entry into a bachelor of education (86.4) was below economics (91.1) and substantially below law (99.6).

The drop in Australian teacher quality is consistent with the findings of US researchers Sean Corcoran, William Evans and Robert Schwab, who estimate that the typical new female teacher in the US was at the 65th percentile in the early 1970s but at the 46th percentile in 2000.

Should we worry if the literacy and numeracy of new teachers has fallen? As the footy aphorism goes, a good player does not always make a good coach. Yet all else being equal, evidence from overseas studies suggests that children learn more when their teachers are more academically talented.

As well as charting the decline, our research also attempts to understand its causes. One factor that seems to have changed substantially during this period is average teacher pay. Compared with non-teachers with a degree, average teacher pay fell by more than 10 per cent during the period 1983 to 2003.

Another driver is pay dispersion in alternative occupations.

In the ’80s and ’90s, non-teacher earnings at the top of the distribution rose faster than earnings at the middle and bottom of the distribution. For someone with the potential to earn a wage at the 90th percentile of the distribution, teaching looked much less attractive in the 2000s than it did in the ’80s.

We believe the fall in average teacher pay and the rise in pay differentials in non-teaching occupations have contributed to the decline in the academic aptitude of new teachers during the past two decades. While our research does not look at the issue, it is also possible that non-salary aspects of teaching may have worsened during this period.

Last, it should be noted that our study focuses on the reasons for the decline in the academic aptitude of new teachers in Australia during the past quarter-century. Reversing these factors is not the only way of raising teacher aptitude. While boosting average teacher pay may be one way of encouraging more able people to enter teaching, it is also possible that increasing the returns to aptitude (or even actual classroom performance) may be a more cost-effective way of raising the quality of the teaching profession.

— originally published in The Australian.

Andrew Leigh casts his net very widely indeed, I see: Does the Lunar Cycle Affect Birth and Deaths? They also published some Julie Bishop friendly research on teacher pay: a critique is in PDF John Graham on Andrew Leigh.

In today’s Sydney Morning Herald is a story called (almost) Literacy and Numeracy have not declined since the 1960s, but that isn’t exciting, so maybe they have…

THE literacy and numeracy performance of Australian students has not improved since the 1960s, suggesting the increase in money invested on education has been misdirected, a study has found.

Research by the economists Andrew Leigh and Chris Ryan, from the Australian National University, said government investment in reducing class sizes, while allowing teacher salaries to decline, relative to other professions, may explain why standards had fallen.

“Over the past three to four decades, neither literacy nor numeracy have improved, and may have declined slightly,” Dr Ryan said. “In numeracy, the typical young teenage student in 2003 was approximately a quarter of a grade level behind his or her counterpart in 1964. Over this time, per-child spending has increased substantially. Yet this additional expenditure does not seem to have succeeded in raising literacy or numeracy.”

The research compared the numeracy and literacy test results of 13- to 14-year-olds. It found a statistically significant drop in numeracy levels between 1964 to 2003 and in numeracy and literacy between 1975 and 1998.

Data was taken from two national sets of tests. Numeracy results were compared from 1964 to 2003 and literacy scores from 1975 to 1998.

The researchers said there was a 10 per cent increase in school expenditure from 1975 to 1998 and a 258 per cent increase between 1964 and 2003.

Productivity was measured in terms of literacy and numeracy points per dollar spent on education. The results suggested a drop in productivity over the past three to four decades.

The researchers did not regard the results as being inconsistent with Australia’s strong performance when compared with other countries on international tests, as “the previous research has suggested that test scores in other OECD countries may also have flatlined over recent decades”, Dr Leigh said. “We cannot rule out the possibility that Australian students today are doing better on outcomes that were not measured in the 1960s, such as verbal communication or social skills. But it is possible the additional education spending over the past few decades was misdirected. Decisions to reduce class sizes while allowing teacher salaries to decline relative to other professions may not have been in the best interests of students.”

There are so many perhapses in that I am really not sure what it means. What I do know is the data comparisons they were able to make look very dodgy to me. A more valid measure of literacy, it seems to me, if very time consuming, would be to study representative samples of actual examination scripts over a period of time. I am told there are archives. The Poet, I believe, once did such a study back in the 1970s on spelling, and found that the same spelling errors occurred over years and years back through the archives. It would not be impossible to devise a set of linguistic rules for examining the corpus of exam scripts; such things have been done for other reasons. Then we might have harder data than that provided by comparing standardised tests — different tests with different standards — from one era to another. The difference would be that we would be applying consistent linguistic analysis to the scripts, whether they were written on the 1940s or in 2008. Could be interesting.

Then we would have to explain what we found. That would have to take many factors into account.

You see, another thing that strikes me about that report in the Herald is that the researchers seem bent on slotting their findings, such as they are, into their thesis about teacher quality and class sizes, and seem to be reductive (to judge of course from the report) in their analysis of possible factors behind any changes, such as variation in school population due to migration, longer retention, and so on.

All comparatively meaningless.

Or is it…? The Herald report may have been based on a superficial reading of the report and skewed by the conventions of the bad education news trope in journalism. ===> UPDATES

1. Andrew Leigh’s career is interesting. I notice, among other things, that he co-authored Imagining Australia: Ideas for Our Future in 2004.

2. Now the ANU site is working again, I have been able to access the actual paper. It is more promising than the report I was going on, but I still have doubts about the measures of literacy used in the study. Similar doubts may apply to the numeracy data, but there I do have to plead ignorance. Judge for yourself: School Productivity (PDF) by Andrew Leigh and Chris Ryan.


Clearly Thomas and The Rabbit are exceptional when it comes to those figures about the declining standing of new teachers; The Rabbit came to teaching via a flirtation with Law. Similar picture, I would judge, with Mikey — not the Law aspect, but the high academic level. I wonder how many of the newly minted Bachelor of Teaching (or whatever) candidates would compare with those who used to go to Teachers Colleges, not universities, in the old days and do two-year or three-year primary, Art, PE, or Industrial Arts training? Many very effective teachers came by that route.


Could it be that the more university-trained teachers we’ve had in the system, the worse have the student outcomes been? 😉

Site Meter


Posted by on February 11, 2008 in Australia, current affairs, education


3 responses to “I really suspect this is comparatively meaningless

  1. Jim Belshaw

    February 11, 2008 at 12:56 pm

    All this caught my eye as well, Neil. There are all sorts of different issues. I will try to do a disentangling post tonight, if I can, to give you something to bounce off.

  2. Jim Belshaw

    February 11, 2008 at 5:41 pm

    Like you, Neil, I finally got through and downloaded the paper. The results are interesting, but cast your mind back.

    The first year of the HSC was 1967. So in 1964 13 year olds would be in the new system. Do you remember what the rules were re quantity of maths, English?

  3. ninglun

    February 11, 2008 at 7:46 pm

    I taught that first HSC, but the issue here are the 13 year olds, only a minority of whom went past Year 10 or even Year 9 back then. If memory serves me correctly the standard load in, say, Year 9 would have been 6, maybe 7, 40-minute periods per week each for English and Maths. Both were core, not elective. That hasn’t really changed at that level since.

    What has changed is the number continuing past Year 10. English is still compulsory right through; Maths, while taken by most, is not compulsory after Year 10.

    I don’t imagine The Rabbit would want to teach Year 7 classes with 40-45 kids in them, as I did when I started, or with over 50, as was the case when I was in Year 7 at SBHS. I look in those rooms now and wonder where they put us!

%d bloggers like this: