Meta-IRB

Here's a fun one for those of you still based at a university.

All of you put together a Human Ethics Review proposal for a field experiment on Human Ethics Review proposals.

Here is the proposal within my proposal.

Each of you would propose putting together a panel of researchers at different universities. You would propose that each of your panel members - from diverse fields, seniority levels, ethnicities and such - would submit a proposal to his or her ethics review board or Institutional Review Board for approval, and each of the panellists would track the time it took to get the proposal approved, which legitimate ethical issues were flagged, which red herring issues also held things up, and how long and onerous the whole ordeal was.

Still in your proposal, you would then propose gathering the data from your panellists and drawing some conclusions about what sorts of schools have better or worse processes. Specific hypotheses to be tested would be whether universities with medical schools were worse than others because medical ethicists would be on the panel, and whether universities with faculty-based rather than centralised IRBs would have better approval processes.

You would note that members of your panels could ask their University's HR advisers to get data on the people who are on the IRBs - race, gender, ethnicity, area of study, rank, age, experience, time on panel, number of children, marital status, and sexual orientation (though not all of those would be in each place's HR database); you'd propose using these as control variables but also to test whether a panel's experience made any difference and whether having a panel member from your home Department made any difference. It would also be interesting to note whether the gender, seniority, ethnicity and home department of the submitter made any difference to the application.

End of the proposal-within-the-proposal.

Now for the fun part: each one of you reading this is a potential member of a panel for a study for which nobody has ever sought ethical approval, but which will be self-approving in a particularly distributed fashion: The IRB proposal to be tested is the one I've just outlined. Whichever of you first gets ethical approval is the lead author on the paper, is a data point, and already has the necessary ethics approval. Everybody else, successful or not, is a data point.

I expect that they might raise legitimate concerns about accessing private HR data and that you limit yourself to publicly available data, or that you survey your IRB members AFTER they issued a decision on your proposal. Those would be very legitimate things for them to point out, and they are the practices I'd want you following anyway: don't bug your HR people. They could also very legitimately point out that since you have zero reason to expect that marital status, children, or sexual orientation have any effect, you shouldn't even survey them asking for it. A good ethics review process, I'd expect, should raise both of those.

More meddlesome ones might ask whether you have appropriately considered the value of the IRB's time across the different institutions. You might then note that getting some 'best practice' guidelines out of this could save many multiples of that time for anybody who's ever sent stuff to an IRB.

But if they tell you you'd need to seek up-front approval from each of the IRBs for your study on IRBs, well, they're proposing killing the study and it would be interesting to know which universities would do that. While they’d be raising the deception of IRB members as an issue, the deception would be necessary for the study to be undertaken.

If there seem to be enough potential folks to make a go of this, I think it could be a lot of fun.

Now it could be that I've just wrecked the potential for running this particular study.

But it could also mean that IRBs that have read the post would be more reasonable in assessing your proposal to assess other IRB proposals. So either way it's good.

If you wind up submitting the proposal above, let me know that you're doing so (so I can tell you if anybody else at your school's already done it) and then let me know how it turns out. If enough people get back to me, then that's the study; first one successfully through can be lead and I'll forward the other data points on to that person.

If we don't have enough data points, well, the first one who did get approval will have to run the study as I actually outlined it above the hard way: setting up a panel of people who will formally submit an IRB proposal. I have a couple of really fun field/audit ones that would be worth doing in their own right, but I'm not going to post them here for fear of skewing things all the way down.

And, to be very clear, lead author above means "you do all the work".

Previously: The Ethics of Ethical Review Boards.