For Teachers: An Important Paper on Rewards and Motivation

If you ask a teacher about the virtues of giving students rewards for behavior, effort, or accomplishment, there is a very good chance that he will tell you about the dangers of “extrinsic motivators”. Specifically, he me may tell you that giving students rewards for doing something will undermine their “intrinsic” motivation to continue doing that thing in the future, once the reward is no longer offered.

This is an element of folk psychology among educators, but it’s not entirely without justification. Any good (especially progressive) school of education will show its teachers-in-training any number of studies that demonstrate just that danger of rewards. Certainly, my credentialing program did.

As many of my classmates were quick to point out, however, many of those studies seem to have limitations that call their external validity into question. The studies tend to look only at certain kinds of rewards, for example, and tend to involve incentivizing tasks that students are already motivated to perform. Real-world classrooms, we noted, have the potential to include a wide variety of rewards for students and often involve tasks that students do not find very interesting, at least initially.

Still, I had never really managed to find an analysis that called the conventional wisdom about rewards and intrinsic motivation into question. So I was very glad when Harry Webb pointed to just such a paper.

It’s called “Pervasive negative effects of rewards on intrinsic motivation: the myth continues“. In it, the authors meta-analyze 145 studies and summarize their findings this way:

Our results suggest that in general, rewards are not harmful to motivation to perform a task.

There is, however, quite a bit of interesting nuance, as the effects of rewards vary by context and type of reward. So, for instance, it matters a great deal whether the task being performed is already interesting. In fact, if a task is of little interest to begin with rewards seem to have the potential to actually increase intrinsic motivation:

When the tasks used in the studies are of low initial interest, rewards increase free-choice intrinsic motivation and leave task interest unaffected. This finding indicates that rewards can be used to enhance time and performance on tasks that initially hold little enjoyment…Our results suggest that reward procedures are one way to cultivate interest in an activity. In education, a major goal is to instill motivation and enjoyment of academic activities. Many academic activities are not of high initial interest to students. An implication of our finding is that rewards can be used to increase performance on low-interest academic activities.

And even when it comes to tasks that are already very interesting for people, the effect of reward depends on the type of reward and how it is administered. Verbal rewards – like praise – can still increase motivation, and so can some tangible rewards, provided that they are used strategically.

Conveniently, the authors include a chart that summarizes their findings fairly clearly:

click to enlarge

click to enlarge

Of course, as you can see from all of the minus (-) signs in that chart – signifying statistically significant (though often small) negative effects on motivation – teachers should probably still avoid tangible rewards for behaviors students are already motivated to exhibit or tasks students are already motivated to perform.

It is also very probable, however, that many teachers are currently under-utilizing rewards – tangible or otherwise – as tools for encouraging unmotivated students. And some non-negligible number of teachers are likely avoiding these rewards in part because the relevant research literature has been misrepresented to them.

Posted in Teaching & Learning | Tagged , | Leave a comment

For Reformers: An Important Paper on Worker Compensation and Incentives

7214510228_fa7685186d_nI’ve written before that education reformers often have an unfortunate lack of perspective about the way the world works outside of education. This means that reformers often unjustifiably assume – implicitly or explicitly – that their proposed changes would make education more like other sectors. This assumption, in turn, makes reformers’ proposals seem more intuitive and leads reformers to underestimate their potential disadvantages.

As a result, if I had to pick one study that I think all would-be education reformers should read, it would be a paper that I once found via Bryan Caplan. It’s an old paper – from 1988 – and it’s not even about education. Rather, it’s an examination of why most companies don’t use the sorts of compensation and incentive schemes that a simplistic understanding of economics might imply they do or should. Here’s the abstract:

A thorough understanding of internal incentive structures is critical to developing a viable theory of the firm, since these incentives determine to a large extent how individuals inside an organization behave. Many common features of organizational incentive systems are not easily explained by traditional economic theory-including egalitarian pay systems in which compensation is largely independent of performance, the overwhelming use of promotion-based incentive systems, the absence of up-front fees for jobs and effective bonding contracts, and the general reluctance of employers to fire, penalize, or give poor performance evaluations to employees. Typical explanations for these practices offered by behaviorists and practitioners are distinctly uneconomic-focusing on notions such as fairness, equity, morale, trust, social responsibility, and culture. The challenge to economists is to provide viable economic explanations for these practices or to integrate these alternative notions into the traditional economic model.

The authors then proceed to examine all of the ways and reasons that most businesses tend not to implement the sorts of incentive and compensation schemes that reformers typically recommend for schools and teachers.

Are you shocked that teachers are not paid differentially based on their performance? Perhaps you shouldn’t be:

Evidence from research on compensation plans indicates that explicit financial rewards in the form of transitory performance-based bonuses seldom account for an important part of a worker’s compensation.. Medoff and Abraham …examine the pay of managerial and professional employees in two large manufacturing firms and find little differences in earnings resulting from superior performance…

Lawler cites six separate studies of the relationship between pay and performance, and finds that “their evidence indicates that pay is not very closely related to performance in many organizations that claim to have merit increase salary systems. . . . It is particularly surprising that pay does not seem to be related to performance at the managerial level.”

Are you surprised at how few teachers are rated “unsatisfactory” by their supervisors? Again, perhaps you shouldn’t be:

[S]upervisors tend to assign uniform performance ratings and tend not to assign poor performance ratings. Only .2 percent of the 4,788 employees in Company A received the lowest rating; 94.5 percent were rated “Good or “Outstanding”. None of the 2,841 Company B employees received an “Unacceptable” or “Minimum Acceptable” rating, and only 1.2 percent received a rating of “Satisfactory”; 95 percent of the Company B employees are rated “Good” or “Superior”.

Does it seem odd to you that teachers are granted tenure? Perhaps you are beginning to sense a pattern:

Tenure systems appear to prevail in situations where human capital, creativity and an unstructured environment are particularly important in the production process, and where long lags between actions and the observation of outcomes make performance measurement and evaluation difficult…

Tenure, partnership, and up-or-out systems tend to be associated with relatively small organizations with few hierarchical levels.

Over and over again, this paper demonstrates that many of the existing teacher incentive and compensation systems that reformers often object to are in fact not that unusual in other fields. Since reformers seem not to be aware of this – and at times actively assume otherwise – they frequently fail to consider the reasons such systems are popular in other fields.

Fortunately, this paper explores many of those reasons, which goes a long way toward making it essential reading for anybody interested in education policy.

Of course, none of this is to say that any particular aspect of today’s education system couldn’t be usefully improved. Understanding how the world can be improved, however, requires first understanding why it is the way it is.

Posted in Education Reform, Teacher Compensation | Tagged , , | 2 Responses

The Limits of Teacher Autonomy

2789011953_eb1bdc27e9_nThis post from Matt Bruenig has just enough educational implications for me to indulge some of my philosophical tendencies. Discussing Justin Green’s complaint about anti-discrimination laws infringing on personal freedom, Matt writes:

Now ask yourself this question: can people in the U.S. refuse to engage in private commerce with anyone for any reason?

The answer is clearly yes. If you do not want to engage in commerce with, say, a black person, you are not forced to. Nobody requires you to operate a hotel, a restaurant, or any other business. If you don’t want to serve a black person at your restaurant, you can refuse to do so by not opening or operating a restaurant. There is no legal penalty for that whatsoever.

We know of course what Green means. He doesn’t mean that people should be able to refuse to engage in private commerce with anyone for any reason (something they already can do). He means that they should have the affirmative ability to engage in private commerce without following the rules we establish for such engagements, in this case non-discrimination rules.

Right. You sometimes see similar arguments from teachers about their own autonomy. Consider this tweet about the Common Core standards:

I don’t actually know how to make sense of the idea that teachers shouldn’t be required to teach standards. What, exactly, are we hiring teachers for if not to teach specific content?

To wax metaphysical, though, note that the Common Core standards – like all previous standards – are in fact completely voluntary for all teachers. After all, any teacher who would prefer not to teach the CCSS is entirely free not to accept a job that requires them to teach the CCSS.

When teachers insist that you should “just let us teach” or complain that we “shouldn’t be forced to do X”, what they sometimes mean is that we should grant them the affirmative right to get hired for a job with a different description. That’s a very different, substantially less-plausible meaning.

Of course, in some cases teacher autonomy is a good thing. Teachers are often best-positioned to make decisions about what their students need or are able to do, for example.

But these are utilitarian considerations; there is no a priori point of principle about whether teachers “should” be “free” to make decisions about what their job entails because everybody is free not to teach in the first place. It’s entirely conceivable, for instance, that our education system would be better on balance if the Common Core standards were repealed or replaced, but the standards are still voluntary for teachers to the extent that nobody is legally required to be a teacher.

This isn’t a major issue in education debates, but every once in a while teachers can’t resist the urge to adopt the appealing rhetoric of “freedom” even though “freedom” is a morally-loaded concept only tangentially related to most educational issues.

Posted in Education | Tagged | 12 Responses

Musical Interlude – Ted Leo and the Pharmacists – Where Have All The Rudeboys Gone?

He’s moved in a somewhat different direction lately, but back in the day Ted Leo was a master of hooks, melody, and falsetto.

Posted in Music | Leave a comment

Educators: Don’t Assume A Can Opener

1282274161_d342cf26f7_nThere is a famous joke about the way economists often undermine the usefulness of their conclusions by making too many simplifying assumptions. Here’s one of the older formulations:

There is a story that has been going around about a physicist, a chemist, and an economist who were stranded on a desert island with no implements and a can of food. The physicist and the chemist each devised an ingenious mechanism for getting the can open; the economist merely said, “Assume we have a can opener”!

It’s probably not fair to pick on economists in this way when then the abuse of simplifying assumptions is at least as widespread in education.

For instance, arguably the trendiest thing going in education today is ‘grit‘: “the tendency to sustain interest in and effort toward very long-term goals”.

We all agree, I suspect, that a tendency to persevere is desirable, and that we should prefer that students have more of that tendency than less of it. So it is perhaps not surprising that since the term was popularized by researcher Angela Duckworth many teachers and schools have begun reorganizing their work to better promote and instill ‘grit’ in their students.

And yet, here’s Duckworth being interviewed by Alexander Russo last month:

Can you talk about how to teach grit in the classroom?
AI don’t know that anybody’s totally figured out how to teach it: What do you do exactly, even when we do have insights from research? How do you get your teachers to speak in ways that support growth mind-set? That’s why, through a nonprofit I helped cofound called the Character Lab, we’re organizing some lectures for teachers about self-control, grit, and related topics. It’s not totally prescriptive, because the science is still developing.

Not to put too fine a point on it, the world’s leading expert on grit is saying that educators who are substantially altering their work to better teach grit are doing so without much in the way of scientific backing or guidance.

In other words, in their excitement over grit many teachers and school leaders have simply assumed – without justification – that it is a trait that can be taught and that they know how to teach it.

This is by no means a problem limited to grit. Before grit it was “21st century skills“, “social-emotional learning”, “critical thinking”, or “scientific thinking”. What unites these fads is that they all, to varying degrees, suffer from a lack of rigorous scientific evidence indicating that they can be taught at all, let alone that we have reliable ways of teaching them in schools. (“Fluid intelligence” may be next.)

Meanwhile, we have good evidence indicating that schools today are reasonably – if imperfectly – effective at teaching kids the less-glamorous knowledge and skills – e.g., in math, science, and history – that we associate with “traditional” education.

So while it’s a good idea for researchers and educators to experiment with methods for teaching other, “higher-order” or “non-cognitive” abilities, it’s also important to remember that it is probably premature to ask schools to move away from their core competencies if we can’t also give them a clear alternate path forward.

Posted in Education | 4 Responses

Musical Interlude – They Might Be Giants – Road Movie To Berlin

De La Soul may have made catalog available to download for free, but TMBG really pioneered distributing music online. This appreciation of “Birdhouse in Your Soul” reminded me that Flood, like many of their albums, has held up much better than other releases of its era.

Posted in Music | Leave a comment

Me on BAM Radio on Teacher Turnover

2341098297_ba2935975dI made a brief appearance on Larry Ferlazzo’s BAM! Radio show to talk more about an upcoming piece about why teachers leave high-poverty urban schools.

Time was short, so I thought I’d collect here, in one place, many of the interesting data points I came across while thinking about the subject.

The best starting place for questions of teacher turnover is probably this report from the National Center for Education Statistics. They surveyed teachers during the 2007-2008 and 2008-2009 school years to look at teacher attrition and mobility during that time period.  They classified teachers as “stayers” if they stayed in the same school, “movers” if they moved to a teaching position in a different school, and “leavers” if they left the profession. Here’s what they found:

Context Stayers Movers Leavers
All public school teachers 84.5% 7.6% 8.0%
Charter school teachers 76.2% 11.4% 12.5%
City schools 84.5% 8.0% 7.5%
≥75% FRPL 84.6% 10.3% 5.1%

Interestingly, neither “city” schools nor lower-income schools have noticeably higher rates of turnover than schools generally. Those classifications don’t perfectly capture what’s going on in “high-poverty urban schools” – “cities” seem to be identified mostly in terms of population and FRPL eligibility is a very imperfect proxy for poverty – but they also don’t indicate that schools in poorer or more urban areas are experiencing noticeably higher rates of turnover than other schools. (I threw the charter school numbers in there mostly out of curiosity.)

It is also worth noting, however, that (via Matt Di Carlo) a more recent data set out of Washington, DC did find noticeably higher rates of turnover in lower-income schools in that district. It’s not clear whether DC is anomalous or if the DC data are in some way better.

The NCES also surveyed “leavers” – from public or private schools this time –  who worked in different contexts about why they left the profession. Excluding vague or more personal reasons (like pregnancy), there were some differences in how often teachers in different contexts cited as very important dissatisfaction with administrators:

Context Dissatisfied with administrators Lack of support from administrators
All leavers 12.8% 12.3%
City schools 17.2% 14.9%
≥75% FRPL 18.8% 18.1%

So teachers who leave teaching after working in city schools and lower-income schools seem significantly more unhappy with their administrators. (Remember, however, that teachers in these schools are nevertheless less likely to leave the profession than teachers in other schools.)

What about teachers who stayed in teaching but moved to a different school? Looking at public and private school movers together, movers cited the following factors as very important:

Context Inconvenient Location Dissatisfied with Admin Student Discipline Lack of Influence
All movers 32.7% 17.3% 11.5% 9.8%
City schools 22.7% 19.6% 14.4% 13.1%
≥75% FRPL 23.4% 21.5% 18.4% 14.8%

So teachers moving from city schools and lower-income schools are more likely to cite dissatisfaction with administrators, behavior issues, and lack of influence, but less likely to move for reasons of geography/convenience.

That NCES report also asked leavers what aspects of their new jobs were better or worse than at their teaching jobs. The five aspects of their new jobs that former teachers (in any context) were most likely to rate as better than teaching were:

  1. Ability to balance personal life and work (56%)
  2. Autonomy (53%)
  3. Recognition from managers (50%)
  4. Salary (47%)
  5. Opportunities for advancement (47%)

This doesn’t speak to high-poverty urban school turnover in particular, but it does speak to what teachers in general might be looking for when they leave the profession.

It also occurred to me that not only should we compare turnover in high-poverty urban schools to turnover in other schools, we should compare turnover in schools to turnover in other sectors.

3716520901_211c447865_nHere I’ve had a little trouble finding exactly the numbers I want. According to the NCES, between the 2007-2008 and 2008-2009 school years 15.5% of public school teachers left their job. Here’s one source estimating total turnover in all sectors in 2008 at 18.7%.

The Bureau of Labor Statistics put the monthly “separation” rate in 2008 at about 3.5% for non-farm jobs. I think that works out to an annual turnover rate of about 42% across all non-farm sectors, but I may be misunderstanding the calculation. If we look just at the “professional and business services” – a generally well-educated sector – the monthly separation rate in 2008 was closer to 5%. Separations are in general higher in the private sector than the public sector.

All else being equal, turnover is generally a bad thing to be avoided when possible. It’s not obvious, however, that high-poverty urban schools or public schools in general have a “turnover problem” compared to other professions. (I’d like to see an annual separation rate for college educated workers to get a clearer picture.)

Posted in Education | 1 Response

Musical Interlude – Randy Newman – Louisiana 1927

There’s a new documentary out about the Great Mississippi Flood of 1927. I also just finished watching Treme and was listening to Good Old Boys the other day:

Posted in Music | Leave a comment

Teacher Autonomy by State (and Salary)

As I mentioned before, the Center for American Progress just put out a report finding that teachers today still seem to feel very autonomous even in this era of accountability and reform.

Exactly how autonomous they feel varies depending on which aspect of the job you’re asking them about. So, for example, about 58% of teachers report having a “moderate” or “great deal” of control over “selecting content, topics, and skills to be taught”, but more than 93% report that much autonomy when it comes to “evaluating and grading students”.

That makes a certain amount of sense: teachers are usually hired to teach pre-existing content standards, so it’d be a little surprising if they felt as autonomous about content as they do about grading or disciplining students (about which 88% feel autonomous).

One of the most interesting aspects of autonomy, in my view, is over “selecting teaching techniques”, since that seems to get at the core of many conceptions of teacher professionalism.

So how much control do teachers report having over how to teach? Quite a lot: 91% of teachers report having a “moderate”  or “great deal”  of control over selecting teaching techniques.

Click to enlarge

Click to enlarge

That doesn’t vary much by state. In the least autonomous state – Florida – 83% of teachers report a “moderate” amount or “great deal” of autonomy over teaching techniques.

To distinguish a little more between states, I ignored teachers reporting a “moderate” amount of control and focused only on those reporting “a great deal” of control over how they teach. Here there is some notable variation between states:

greatdealofcontroloverteachingtechniques

Click to enlarge

So while overall levels are high, there do appear to be some differences in perceived autonomy between the states. Three-quarters or more of the teachers in Hawaii, Montana, and the Dakotas feel very free to teach the way they want, while only about half of teachers in Delaware, Florida, Rhode Island, and Maryland feel that way.

What could explain these differences? One hypothesis could be that teachers are granted more autonomy in exchange for accepting relatively lower salaries (e.g., “True, we’re not paying you very much, but we won’t bug you about how you teach.”) Conversely, we might expect that teacher autonomy is associated with relatively higher salaries (e.g., “We’re paying you generously because we respect your expertise; obviously we’re going to defer to your judgment about how to teach.”)

Or, at least, these were hypotheses that came to mind for me, probably because I recently looked at teacher salaries in different states.

But are either of these hypotheses supported (or ruled out) by the data?

If we just look at average teacher salaries, states with more highly-paid teachers tend to have teachers who feel (slightly) less autonomous:

Click to enlarge

Click to enlarge

That pattern persists when we adjust teacher salaries for state-level median household income:

Click to enlarge

Click to enlarge

So there is some tentative support for the hypothesis that better-paid teachers are more likely to be expected to teach in school- or district-approved ways.

Interestingly, however, the relationship between autonomy and salary flips when we look at salaries as a fraction of median income for other 4-year college graduates:

Click to enlarge

Click to enlarge

So in states where teachers are paid more relative to household income, they feel less autonomous. In states where teachers are paid more relative to other BA-holders, however, they feel more autonomous.

It’s not at all clear what’s going on here. If you can get such different results by slicing and dicing the salary numbers in different ways, then it’s entirely possible that teacher feelings of professional autonomy are not causally related to salaries in any direct, meaningful way.

So if teacher autonomy is related to salaries…how? And if autonomy isn’t related to salary…why not?

Posted in Education Reform, Teacher Compensation, Teaching & Learning | Tagged | Leave a comment

Musical Interlude – Pixies – I Can’t Forget

Yesterday I tried to convince Kevin Ferguson that there are, in fact, some very good tribute albums. Two of them are tributes to Leonard Cohen including I’m Your Fan, where the Pixies do the right thing for this song by substituting a dose of derangement for the original’s bit of senility.

Posted in Music | Leave a comment