Saturday, 28 February 2015

Meta-IRB

Here's a fun one for those of you still based at a university.

All of you put together a Human Ethics Review proposal for a field experiment on Human Ethics Review proposals.

Here is the proposal within my proposal.
Each of you would propose putting together a panel of researchers at different universities. You would propose that each of your panel members - from diverse fields, seniority levels, ethnicities and such - would submit a proposal to his or her ethics review board or Institutional Review Board for approval, and each of the panellists would track the time it took to get the proposal approved, which legitimate ethical issues were flagged, which red herring issues also held things up, and how long and onerous the whole ordeal was.

Still in your proposal, you would then propose gathering the data from your panellists and drawing some conclusions about what sorts of schools have better or worse processes. Specific hypotheses to be tested would be whether universities with medical schools were worse than others because medical ethicists would be on the panel, and whether universities with faculty-based rather than centralised IRBs would have better approval processes.

You would note that members of your panels could ask their University's HR advisers to get data on the people who are on the IRBs - race, gender, ethnicity, area of study, rank, age, experience, time on panel, number of children, marital status, and sexual orientation (though not all of those would be in each place's HR database); you'd propose using these as control variables but also to test whether a panel's experience made any difference and whether having a panel member from your home Department made any difference. It would also be interesting to note whether the gender, seniority, ethnicity and home department of the submitter made any difference to the application.
End of the proposal-within-the-proposal.

Now for the fun part: each one of you reading this is a potential member of a panel for a study for which nobody has ever sought ethical approval, but which will be self-approving in a particularly distributed fashion: The IRB proposal to be tested is the one I've just outlined. Whichever of you first gets ethical approval is the lead author on the paper, is a data point, and already has the necessary ethics approval. Everybody else, successful or not, is a data point.

I expect that they might raise legitimate concerns about accessing private HR data and that you limit yourself to publicly available data, or that you survey your IRB members AFTER they issued a decision on your proposal. Those would be very legitimate things for them to point out, and they are the practices I'd want you following anyway: don't bug your HR people. They could also very legitimately point out that since you have zero reason to expect that marital status, children, or sexual orientation have any effect, you shouldn't even survey them asking for it. A good ethics review process, I'd expect, should raise both of those.

More meddlesome ones might ask whether you have appropriately considered the value of the IRB's time across the different institutions. You might then note that getting some 'best practice' guidelines out of this could save many multiples of that time for anybody who's ever sent stuff to an IRB.

But if they tell you you'd need to seek up-front approval from each of the IRBs for your study on IRBs, well, they're proposing killing the study and it would be interesting to know which universities would do that. While they’d be raising the deception of IRB members as an issue, the deception would be necessary for the study to be undertaken.

If there seem to be enough potential folks to make a go of this, I think it could be a lot of fun.

Now it could be that I've just wrecked the potential for running this particular study.

But it could also mean that IRBs that have read the post would be more reasonable in assessing your proposal to assess other IRB proposals. So either way it's good.

If you wind up submitting the proposal above, let me know that you're doing so (so I can tell you if anybody else at your school's already done it) and then let me know how it turns out. If enough people get back to me, then that's the study; first one successfully through can be lead and I'll forward the other data points on to that person.

If we don't have enough data points, well, the first one who did get approval will have to run the study as I actually outlined it above the hard way: setting up a panel of people who will formally submit an IRB proposal. I have a couple of really fun field/audit ones that would be worth doing in their own right, but I'm not going to post them here for fear of skewing things all the way down.

And, to be very clear, lead author above means "you do all the work".

Previously: The Ethics of Ethical Review Boards.

Friday, 27 February 2015

Missed opportunity

Sadly, our tech regulations do not put us in the outside of the Asylum.

I wrote in May 2013:
New Zealand keeps ranking at or near the top of the various indices of economic and social freedoms. We could do well by encouraging greater immigration of American techies fed up with that the American governmentseems to be archiving and storing just about everything for later searches. Just show them Novopay as example of how we couldn't, even if we wanted to.
Alas, we're not immune to the shenanigans going on elsewhere. Our NSA, the GCSB, is getting a legislative redraft. Thomas Beagle of TechLiberty summarisesNoRightTurn has a few additional comments. I'm not a lawyer - maybe things aren't as bad as they seem. David Farrar is considerably less concerned....When the US seems to be doing everything it can to convince its tech guys that the government really does want to be spying on everybody, and that the IRS wants to know everything you talk about at political meetings if you have small-government leanings, the last thing we need are headlines suggesting we're heading down similar paths if the legislation doesn't actually do that. And if it does, it does need changing.
At the time, a whole pile of people offered me big assurances that the TICSA legislation was far more innocuous than the press made out and that it wouldn't just work to kill innovative startups who couldn't handle the regulations and that it was all just a beat-up by people who didn't understand the regulations and hadn't had all the super-secret briefings.

The NZ Herald now tells me that we've scared off a pile of tech investment. I guess that they didn't understand the regulations or profit from the super-secret briefings either.

Charting the Minimum Wage

It's never enough, is it?

As reply to those clamouring for a higher minimum wage, after the National government just hiked it by $0.50 to $14.75, some charts from MBIE's advice on the minimum wage hike.




The minimum wage is 64.9% of the median wage and will rise to about 65-66% of the median wage depending on what comes out in the next Income Survey release. 

Keyhole solutions: alcohol edition

One of the main real external costs of alcohol use comes via alcohol-associated crimes. And so it's good to see the government attempting a more finely tuned intervention on this than using excise tax.

David Farrar points to the Drug and Alcohol Testing of Community-based Offenders and Bailees Legislation Bill coming up. In short, it allows drug and alcohol testing to be a condition of bail.

Why is this a good idea? Something similar seems to have done a lot of good in Hawaii. Here's Heritage on the Hawaii Opportunity Probation with Enforcement (HOPE) project, which targeted methamphetamine users, and here's an evaluation of South Dakota's programme that targeted alcohol use.

To be very clear: I only support this kind of thing for offenders who have committed real violent or property crimes, and not those caught simply for drug possession.

But for those offenders who go out and do bad things to people while drunk or high, well, they've probably well signalled that they should be kept away from the stuff for a while.

Please review the ethics of human ethics review

Human ethics review panels are the worst. Designed to prevent atrocities like Tuskegee, they then metastasised: any time your research could possibly involve a human subject, there are a billion forms to fill in. So it's usually easiest just to avoid doing field research.

U Queensland's Paul Frijters didn't avoid doing field research. He sent actors out onto Brisbane buses with not quite enough money on their fare cards to see whether drivers were more likely to give white passengers a pass.

Adam Creighton and Julie Hare write in The Australian on what happened next.
Following national media interest arising from a media release published by the university in March 2013, Professor Frijters was told his research was “banned”.

“The university then proceeded to pursue charges of research misconduct against me, eventually (demoting) me from professor to associate professor in March 2014 and threatening me with dismissal if I did more research like this,” he said.

The demotion was later overturned following an independent inquiry showing the university’s own processes had failed and that punishment meted out was “overly harsh and inappropriately punitive”.

Vice-chancellor Peter Hoj yesterday said the university would not comment.

A spokeswoman for Brisbane City Council, which owns the Brisbane bus company, said the research was not “authorised” by it and it was published without the council’s knowledge.
Here's the original paper.

Here's the Guardian, whose story makes it a bit clearer why human ethics review panels are the worst - simply the worst bunch of Vogons ever to be imposed on academics working in the social sciences.
The university then banned further publication or promotion of the study on suspicion that Frijters not sought the necessary approval from the university ethics committee.
The university was concerned there was no “voluntary informed consent” from the bus drivers or “gatekeeper approval from the Brisbane city council”.
You cannot test whether a bunch of Brisbane bus drivers are racist by first asking them for permission to test whether they are racists.

Here's the kind of ethicist from whom Frijters would have needed to have sought permission:
Clinical ethics expert Dr Andrew Crowden told Guardian Australia that Frijters had made a “common mistake” by underestimating the ethical risks of his own research.
But the university had “shared the mistake” when his department signed off on the study, which showed a “systemic failure” that UQ had chosen not to address despite Crowden’s recommendations in response to the case.
Nicholas Gruen is dead right:
Economist Nicholas Gruen said the case was a “terrible” example of how universities dealt with ethics considerations as “a matter of bureaucratic arse covering and the avoidance of any kind of discomfort for anyone”.
“This sorry saga illustrates the way ethics approvals [are] genuinely strangling all kinds of research initiatives,” he said.
“Essentially the entire ethics procedure is an attempt to avoid anything that might make anyone squeamish or uncomfortable. Of course good research, certainly in social sciences will often do that.”
Ethicists.

Update: Helen Andrews points to some more thorough work on IRB horrors:

4 years on

The government rightly took a lot of criticism for its initial attempts to artificially restrict downtown land supply to force a compact city form and encourage higher-valued development. The planners here exhibited basic cargo-cult thinking: because successful cities have high downtown property prices, they thought they could make Christchurch successful by forcing prices to be high. Well, that doesn’t work: high prices in successful cities reflect that people get a lot of value from being located in great downtowns, not the other way around. 
In the longer term, because so much has moved on to the suburbs and to neighbouring districts, downtown land prices will have to drop. When that happens, developers will be able to bring to market properties with rental rates that could draw in tenants – if the planners don’t mandate that everything be plated in gold. Downtown will then come back but as part of a polycentric city. 
This too should not be overly lamented. While I really love the tight downtown core in my new home, Wellington, it is risky. Faultlines can open unexpectedly and in unanticipated locations. Wellington really cannot have multiple downtown cores because of geography but Christchurch can build in resilience against future events by having lots of centres of economic activity. And, fortunately, while the economic literature points strongly to the benefits of urban agglomeration and of having lots of people in a city, it is far from clear that those benefits require having a single dense centre. 
Hit the link to read the whole thing...

This is what happens when you don't read Demsetz

Way too many policy arguments take the following form.
  1. Markets in an ideal world are efficient.
  2. Here is a potential deviation of the real world from blackboard conditions, so we're in a second-best.
  3. Policy can ameliorate outcomes when there is a market failure. So, here's what we must do.
What's missing? Any evaluation of whether the policy cure is actually an improvement on the status quo. Some policies are like using tweezers to pull out an irritating splinter - great idea. Others would have you hack off the arm to avoid the splinter. 

Harold Demsetz very nicely made this point way back in 1969. He was there critiquing Arrow on information market failures, but the lesson is more general. It isn't enough to simply point out a potential market failure. Markets fail but policies don't automatically induce nirvanas. We need comparative institutional analysis to tell us which world sucks less: the world with a market failure that isn't addressed by policy, and the world in which a real-world policy involving actual tradeoffs comes in to try to solve it.

Today's lesson in "this is what happens when you don't read Demsetz" comes from Dean Baker over at Cato Unbound.

Baker's argument:
  • In a first-best world, we would have proper congestion charging and the like;
  • We don't have proper congestion charging;
  • Taxicabs can increase congestion;
  • Uber likely increases the number of cabs;
  • Therefore we need a complicated regulatory structure for Uber imposing fees by time of day and location of service.
Baker also reckons that while Uber's drivers are contractors, not employees, and despite that those drivers like the flexibility, the drivers should be treated like employees for minimum wage and overtime purposes.

Some folks just hate the idea of voluntary transactions among consenting adults that don't route through the State somehow along the way. 

A couple things to note:
  • It's eminently unclear that Uber increases congestion. In the longer term, it will reduce the amount of street space that need be devoted to parking, freeing up more road for driving. It will also reduce peoples' need to take a car for the day because of that one trip they need to make mid-day and instead let them commute in using the bus, then take an Uber for the part when they do need a car. 
  • Using charges on Uber to solve congestion instead of broad-based congestion charging is nuts. Unless Uber is a very large proportion of cars on the road, having any effect on congestion using charges on Uber would have to involve just massive variability in ultimate Uber charges on consumers, which would deter any use of the service. I favour congestion charging, but implementing it on Uber only makes as much sense as imposing congestion charging only on blue cars. 
Update: a reader points out that Uber surge pricing is already a form of congestion charging. It's a good point.