Thursday, August 24, 2017

Solutions to bias in the modern scientific process

There is wide recognition of inherent bias problems in the modern scientific process. I attended a good talk by Brian Nosek today discussing the challenges, barriers and solutions. Among the scientific process challenges are flexibility in analysis of data, selective reporting, ignoring nulls and lack of replication. Some of these problems can be traced to basic human behavior and psychology including perceived norms, motivated reasoning, minimal accountability, and people are just plain busy.

Nosek proposes that the solution to these problems is to show the work and share it early in the research cycle. Signals and incentives to make this behavior visible is necessary.

Towards that end here are a few efforts to address these problems:

Open Science Framework -- tools for sharing work across the research cycle
http://osf.io

Registered reports -- peer review after research design, before collection and analysis
http://cos.io/rr

Tracking switched outcomes in trials:
http://compare-trials.org/

AsPredicted:
https://aspredicted.org/

For computation and data science -- code sharing:
https://gist.github.com


No comments: