The Heckman Curve is flat

It sounded good in theory. Interventions targeted at youths could very plausibly have been rather more effective than programmes targeted at older cohorts. Heckman's foundation thing made a pretty infographic about it.

But it's fundamentally an empirical question. The infographic is based rather more on intuition than on any real lit survey. 

Rea and Burton checked it out. They went into the big Washington State intervention database, sorted interventions by age, and just plotted out the cost effectiveness. There was no curve - just a noisy mess. And the regressions found nothing either. 

Here are the key tables. 

Figure 3's the kicker. Compare it to the stylized Heckman curve. Ocular least squares really are enough here. 

It also links into my other worries about the Perry preschool stuff that Heckman's been pushing and that seems really not to have fared well in broader roll-outs in Tennessee or in Quebec. 

If there is a Heckman Curve, it's hard to see it in the programme evaluation work compiled in the Washington State database. 

Pre-school investments

James Heckman publishes new papers faster than I can read papers. And so I can't pretend to be on top of all of the work he's been doing on early childhood education. But there has been a lot of it over the past several years.

I was pretty excited about the whole line of work when he started it. Investments in early childhood education looked to yield huge benefits, especially for children from disadvantaged backgrounds. NPR provides a decent summary of that work in an interview with Heckman in December:

If you got 13 percent back on your investments every year, you'd be pretty happy, right? Remember, the S&P 500, historically, has averaged about 7 percent when adjusted for inflation.

What if the investment is in children, and the return on investment not only makes economic sense but results in richer, fuller, healthier lives for the entire family?

That's the crux of a new paper out Monday, The Life-Cycle Benefits of an Influential Early Childhood Program, co-authored by Nobel laureate James Heckman, a professor of economics at the University of Chicago and the director of the Center for the Economics of Human Development.

The new paper looks again at the Abecedarian Project, and finds returns that last through to the long-term. It sounds like the kind of thing that would make sense under the New Zealand government's investment approach. But I have a couple of worries.

Every time we see great results like the ones Heckman demonstrates, they're from really small-scale, really resource intensive programmes like Abecedarian or Perry.

And the couple of studies I've seen looking at what happens with larger roll-outs don't suggest that these things scale well.

Michael Baker, Jonathan Gruber and Kevin Milligan found that a universal childcare programme in Quebec yielded a huge increase in use of childcare, but worsening of outcomes for kids. NBER provides a summary here.

And an evaluation of Tennessee's pre-K programme, rolled out to 18,000 lower income kids across the state, showed temporary gains followed by worse outcomes than those for kids who didn't participate. This one was focused on providing high quality care to a large group of deprived kids, rather than being a universal roll-out like Quebec's. The programme only included 4 year olds, and so could be critiqued as not having the very early intervention that the Heckman-style interventions recommend, but it should give pause that the effect wound up being negative.

I don't know if these things don't scale well because you quickly run out of teachers that match the quality of those involved in Abecedarian and Perry, or if larger scale programmes in general disappoint. But it does suggest we need to be careful in monitoring roll-outs of things that looked good in smaller scale trials.

And I worry too about what happens if it turns out that the thing really can only work in really small and focused interventions. You then wind up setting eligibility criteria to focus on the groups in most need, and really pissing off the people who are just outside the eligibility threshold. Like, "You mean if I'd had more CYFs notifications my kids would be eligible for this fantastic pre-school programme but because I was a better parent than my crazy neighbour she gets the programme and I don't?!" The usual answer on the left to that kind of concern is to demand that it be universal, but look again at the results in Quebec.

I don't pretend to have any good answers here, just worries. I wonder if anybody's used IDI data to see what happened after New Zealand shifted to 20-hours free daycare.

FWIW, both our kids were in pre-school from about 3 months old, when Susan returned to work. So our revealed preference was to rely on professional childcare.

Magic mobility bullets

There are (probably) no magic bullets, only trade-offs.

Landersø and Heckman look at intergenerational mobility in the US and Denmark. The former has far less income redistribution than the latter. One of the many welfare policy tradeoffs is that strongly progressive income tax regimes to fund large scale redistribution reduces the return to education because of the resulting after-tax salary compression. And so if part of your basic model of the world is that poor kids do not take up education because of family deprivation, then you have a tradeoff: the same policies that reduce that problem also reduce incentives to undertake education in the first place.

They find that there is a real causal negative link between generosity of public assistance and educational enrolment, using a policy change that affected welfare access by age, raising the minimum age of eligibility for full social insurance payments. The caveats:

The results presented here establish a negative relationship between educational enrollment and the level of public benefits, albeit with two caveats. First, it is beyond the scope of this paper to estimate the underlying behavioral parameters—we strongly encourage future research to explore this relationship further. Second, we neither have precise estimates of the potential gains from the greater equality in childhood investments and fewer pecuniary costs of education in Denmark than in the U.S., nor the disincentives for educational attainment that wage compression and public benefits constitute. Hence, we cannot determine whether the similarities in educational mobility in the two countries occur because the effects offset each other, though we find this to be a plausible explanation given the evidence at hand.

They also find that neighbourhood sorting remains rather important.

They provide an appropriate caution on noting the substantial gaps in cognitive, language and emotional skills among Danish kids from higher and lower income backgrouns:

While the Scandinavian welfare state invests heavily in children throughout childhood and redistributes income (consumption) during adulthood, it has not eradicated the strong influence of parents and early childhood conditions. As a consequence of the complementarity between skills and investments, later life universal schooling investments during childhood or adolescence may be ineffective in reducing gaps between advantaged and disadvantaged children. 

They conclude:

Despite Denmark’s far more generous welfare state, its extensive system of preschools, and its free college tuition, the family influence/child education relationship is very similar to that of the U.S. In both countries, much of the average association between parental resources and the educational attainment of children can be explained by factors set in place by age 15, including child skills. However, distributions of cognitive test scores of disadvantaged Danish children are much better than those of their counterparts in the U.S.

The failure to promote greater educational mobility in spite of providing generous social services is most likely rooted in the welfare state. Our findings point to wage compression and the higher levels of welfare benefits as being counterproductive in providing incentives to pursue education. The low returns to education observed in Denmark, in particular at the lower levels of education, help explain the disconnect between the egalitarian childhood policies in Denmark and the roughly equal levels of educational mobility in Denmark and the U.S. The sorting of families into neighborhoods and schools by levels of parental advantage is likely another contributing factor. While the Danish welfare state may mitigate some childhood inequalities, substantial skill gaps still remain. 

Perhaps better outcomes at the bottom in Denmark are due to the country's more extensive redistribution, or perhaps it's due to the advantage of having a country of Danes. But the resulting wage compression causes other problems.

Update: Jim Rose points me to this write-up on the study at The Atlantic.

Plain Packaging and Evidence

Otago's Public Health department and British American Tobacco seem to be talking past each other.

A few days ago, the usual bunch at Otago put out a press release about some surveys they conducted about tobacco labelling. The article is here; I've copied their main results table below.

This is the sum total of the paper, really. They ran the survey, these are the results. Again, I remain pretty amazed at how easy it is to get published in some outlets. Nothing on demographic covariates; nothing on education/smoking interactions on views... hard to see a there there.

What does the survey tell us?

1. Both smokers and non-smokers think there should be warning labels on tobacco products, though non-smokers are more likely to agree;
2. Non-smokers think attractive packaging encourages smoking initiation; smokers disagree. 
• Crampton's editorialising: Presumably smokers are more likely to know what got them started. 
3. Smokers do not think that tobacco packaging encourages smokers to try new brands; non-smokers disagree.
• Crampton's editorialising: Branded packs should encourage smokers to stay with their chosen brand - they're an effort to build customer loyalty in what's otherwise pretty easily a homogeneous commodity product.
4. Non-smokers are more likely to agree that plain packaging discourages smoking initiation; smokers just as strongly disagree.
• Crampton's editorialising: Again, why do we think that smokers would be wrong on this one? They remember what got them started on smoking.
5. Non-smokers lean slightly towards agreeing that plain packaging would encourage smokers to quit; smokers disagree pretty strongly.
6. Non-smokers think plain-packaging is fair; smokers are split.

The second survey, according to the press release, tells us that there are identities around brands. That isn't surprising; it's the point of branding.

Nothing in either survey really tells us what's likely to happen if plain packaging is imposed. Here, I put a lot more weight on basic theory than on surveys. And theory tells us that once you pull brands out, cigarettes become more homogeneous. Smokers then become more cost sensitive across brands and are more likely to shift downwards towards cheaper brands. When that happens, smoking rates among smokers may increase; Clarke, linked above, doesn't think the price shifts will be substantial enough to have large price effects - he reckons it can be offset by tax increases in any case. But neither he nor I really know the scale of likely effects. And Janet Hoek doesn't know either. 

The Otago Daily Times reports on BAT and Hoek's dispute. Recall that Hoek's evidence above is entirely survey stuff.

A University of Otago researcher has hit back at comments by a tobacco industry spokesman who said plain packaging would not reduce tobacco consumption.

On Wednesday British American Tobacco (BAT) general manager Steve Rush responded to the findings of Otago University's Aspire2025 research group, which found that two-thirds of respondents supported plain packaging. Mr Rush said the research did not prove plain packaging would reduce tobacco consumption.

"The research is attitudinal and is based on opinions. It's about how people think they might behave in the future. It's not about how people actually behave," he said.

Tobacco packaging did not influence why people started smoking or quitted, he said.

Prof Janet Hoek, from university's marketing department, hit back yesterday, saying Mr Rush's remarks about her team's research were "illogical, unsupported" and ignored a "well-established evidence base".

"He has trotted out the same tired arguments in a failed attempt to disguise the fact that there is very strong public support for plain packaging, and very little sympathy for the arguments his company is investing hundreds of thousands of dollars trying to promote.

She says people like plain packaging. He says there's no evidence that it'll work. She says he's ignoring the evidence that people like plain packaging. What evidence there is in the Hoek paper suggests, at least to me, that the cohort of people who are smokers don't think that plain packaging will do much to reduce uptake or to encourage quitting; I don't put much confidence in survey measures of this sort, but if I had to bet, I'd bet on the estimates from the folks who are smokers rather than the ones from the folks who aren't. And the rest of the international literature on which Hoek would draw suffers from the problem that, because nobody's ever tried plain packaging, we can't really tell what would happen.

I worry that most of the support for plain packaging comes less from potential effects on smoking rates but rather from this kind of sentiment:

@ericcrampton who cares if it works or not - any handicap put on tobacco companies is good in my books @metiria @dpfdpf @danylmc @keith_ng
— Stuart Maxwell (@stuartm) October 11, 2012

It's stuff like this that makes Baby Pareto cry.

Nobody really knows what the effects of plain packaging will be. If we and the Australians cared about finding out whether plain packaging had any effect, we'd be setting up a randomised control trial across the two countries such that some areas would get plain packages and others wouldn't. Instead, both countries seem to be bundling plain packaging with a host of other anti-smoking measures that will make it pretty hard to sort out later on whether any change in smoking rates was due to plain packaging, changes in tax rates, time trends, or something else.

I'm less than convinced by the argument that plain packaging encourages smuggling and black market cigarettes: you can always add anti-counterfeiting tax-stamp holograms to the packs to make it obvious which ones are legit.* But I can see a mechanism where black market cigarettes are encouraged: breaking brand loyalty and making smokers re-evaluate their consumption choice may lead some smokers to choose black market products as there will be less product differentiation. I have a hard time seeing other mechanisms driving it - it has to come from smokers being less reluctant to switch to black market products. If we start seeing divergences between excise counts on tobacco sales and self-reports of smoking behaviours in the households surveys, we just won't be able to tell whether increased use of illicit tobacco is due to plain packaging or the big set of price hikes that will hit the legal market.

Harry Clarke, who supports plain packaging, suggests that one of the main potential benefits of its adoption in Oz and NZ is figuring out whether the policy does anything:

This conclusion does not mean PP legislation will have strong effects in reducing demands. This depends on the dissuasive impact of PP on consumption. But it is very unlikely that such policies would be counterproductive in the sense of increasing demands. A weak argument endorsing Australia’s adoption of PP policies is that they will have some positive effect in reducing demand without causing any harmful effects. An important gain is determining whether such policies will be very effective or not. Such innovative policies provide a “global public good”. There is often a Prisoners’ Dilemma case against delivering such public goods but, because the policies are likely to be non-disadvantageous, there are no extra costs if Australia is the first country to try out such policies. Indeed Australia has advantages over other developed countries in performing such an experiment. It is remote and physically isolated compared with European, developed Asian or even North American nations. This reduces problems of illegal cigarettes and private importing. Possibly only New Zealand is better as a laboratory. Altruistic provision of global information on the effectiveness of PP should provide benefits in reducing smoking but, at worst, will cost Australia nothing and boost its image as a good global citizen.

But it sure will be hard to run that policy evaluation. We'll likely be able to say "the bundle of plain packaging and substantial excise hikes and other stuff led to X", but disentangling effects would require that there's reasonable ex ante price variation across Australian states so we'd have different proportional changes in total prices with the excise increases.

I'd also disagree pretty strongly with the "will cost Australia nothing" conclusion. Even if you just hate tobacco companies and count their utility negatively, every policy of this sort that's adopted increases the chances that it'll be adopted for other products. The UK Health Select Committee just a couple of months ago was talking about considering plain packaging for alcohol, while promising that they'll not consider it for food. I'm still nervous for Count Chocula. The policy buys some fights on trade policy that could well be pointless. And, it's another step down the road that says that even if we provide all the relevant information to consumers, consumers are incompetent to make their own choices and public health experts should get to choose for them. I don't like that road; every step on it is costly if you like consumer sovereignty.

* I generally concur [that's too strong as I've not gone through it in enough depth. Let's say "generally sympathetic to"] with the points Philip Morris makes in its submission. But the points on black market and plain packaging really seem to be an argument in favour of putting anti-counterfeiting measures on unbranded products. They really bring out the heavyweights on it: James Heckman is their consultant; he cites his Nobel on causal inference among his qualifications to be an expert. His report on youth smoking, the appendix to the submission, deserves its own post sometime soon.