伯克利行为经济学和金融研究计划改进实验设计和预测的准确性-新东方前途出国

　　Were the results of that experiment really so predictable? These researchers aim to find out.

　　那次实验的结果真的那么容易预测吗这些研究人员的目的是找出答案。

　　October 29, 2019October 30, 2019 Laura Counts

　　2019年10月29日2019年10月30日劳拉计数

　　Researchers have launched a beta version a new platform to predict research study results.

　　研究人员推出了一个beta版的新平台来预测研究结果。

　　They say that hindsight is 20-20, and perhaps nowhere is that more true than in academic research.

　　他们说后见之明是20-20，也许没有什么比学术研究更真实的了。

　　“We’ve all had the experience of standing up to present a novel set of findings, often building on years of work, and having someone in the audience blurt out ‘But we knew this already!,’” says Prof. Stefano DellaVigna, a behavioral economist with joint appointments in the Department of Economics and Berkeley Haas. “But in most of these cases, someone would have said the same thing had we found the opposite result. We’re all 20-20, after the fact.”

　　“我们都有过这样的经历：常常在多年工作的基础上，站起来提出一系列新颖的发现，并让观众脱口而出‘但我们已经知道这一点了！经济学系和伯克利哈斯联合任命的行为经济学家斯特凡诺·德拉维尼亚教授说。“但在大多数情况下，如果我们发现相反的结果，有人会说同样的话。事实上，我们都20-20岁了。”

　　Prof. Stefano DellaVigna

　　Stefano DellaVigna教授

　　DellaVigna has a cure for this type of academic Monday morning quarterbacking: a prediction platform to capture the conventional wisdom before studies are run.

　　Dellavigna有一个治疗这种学术性的星期一早上的四分之一支持：一个预测平台，在研究开始前捕捉传统智慧。

　　Along with colleagues Devin Pope of the University of Chicago’s Booth School of Business and Eva Vivalt of the Research School of Economics at Australian National University, he’s launched a beta website that will allow researchers, PhD students, and even members of the general public to review proposed research projects and make predictions on the outcome.

　　与芝加哥大学布斯商学院的Devin Pope和澳大利亚国立大学经济研究院的Eva Vivalt一起，他推出了一个测试版网站，允许研究人员和博士生使用甚至公众也要对提出的研究项目进行审查，并对结果做出预测。

　　Making research more transparent

　　使研究更加透明

　　Their proposal, laid out in a new article in Science’s Policy Forum, is part of a wave of efforts to improve the rigor and credibility of social science research. These reforms were sparked by the replication crisis—the failure of reproduce the results of many published studies—and include mass efforts to replicate studies as well as platforms for pre-registering research designs and hypotheses.

　　他们的建议发表在科学政策论坛的一篇新文章中，是提高社会科学研究严谨性和可信度的努力浪潮的一部分这些改革是由复制危机引发的，复制许多已发表研究成果的失败，包括复制研究的大规模努力以及预先注册研究设计和假设的平台。

　　“We thought there was something important to be gained by having a record of what people believed before the results were known, and social scientists have never done that in a systematic way,” says DellaVigna, who co-directs the Berkeley Initiative for Behavioral Economics and Finance. “This will not only help us better identify results that are truly surprising, but will also help improve experimental design and the accuracy of forecasts.”

　　伯克利行为经济学和金融研究计划的共同负责人德拉维格纳说：“我们认为，在结果公布之前，通过记录人们的信仰，可以获得一些重要的东西，而社会科学家从来没有系统地做到这一点。”“这不仅有助于我们更好地识别出真正令人惊讶的结果，还将有助于改进实验设计和预测的准确性。”

　　Identifying truly surprising results

　　确定真正令人惊讶的结果

　　Because science builds on itself, people interpret new results based on what they already know. An advantage of the prediction platform is that it would help better identify truly surprising results, even in cases where there’s a null finding—which rarely get published because they typically aren’t seen as significant, the researchers argue.

　　因为科学是建立在自己的基础上的，所以人们根据他们已经知道的来解释新的结果研究人员认为，该预测平台的一个优势是，它将有助于更好地识别真正令人惊讶的结果，即使在很少公布无效结果的情况下，因为它们通常不被视为有意义的结果。

　　“The collection of advance forecasts of research results could combat this bias by making null results more interesting, as they may indicate a departure from accepted wisdom,” Vivalt wrote in an article on the proposal in The Conversation.

　　维瓦尔特在一篇关于这一建议的文章中写道：“收集对研究结果的预先预测，可以使无效结果更有趣，从而克服这种偏见，因为它们可能表明偏离了公认的智慧。”

　　A research prediction platform will also help gauge how accurate experts actually are in certain areas. For example, DellaVigna and Pope gathered predictions from academic experts on 18 different experiments to determine the effectiveness of “nudges” versus monetary incentives in motivating workers to do an online task. They found the experts were fairly accurate, but there was no difference between highly cited faculty and other faculty, and that PhD students did the best.

　　一个研究预测平台也将有助于衡量专家在某些领域的准确程度。例如，DellaVigna和Pope收集了学术专家对18个不同实验的预测，以确定“轻推”和金钱激励在激励员工做在线任务方面的有效性他们发现这些专家相当准确，但被高度引用的教员和其他教员没有区别，博士生做得最好。

　　Understanding where there is a general consensus can also help researchers design better research questions, to get at less-well-understood phenomena, the authors point out. Collecting a critical mass of predictions will also open up a new potential research area on whether people update their beliefs after new results are known.

　　作者指出，了解普遍共识的地方也有助于研究人员设计更好的研究问题，以获得不太容易理解的现象。收集大量的预测也将开辟一个新的潜在研究领域，研究人们是否在知道新的结果后更新他们的信仰。

　　Making a prediction on the platform would require a simple 5-to-15-minute survey, DellaVigna says. The forecasts would be distributed to the researcher after data are gathered, and the study results would be sent to the forecasters at the end of the study.

　　Dellavigna说，在平台上进行预测需要5到15分钟的简单调查。这些预测将在收集数据后分发给研究者，研究结果将在研究结束时发送给预测者。

　　Berkeley Haas Prof. Don Moore, who has been a leader in advocating for more transparent, rigorous research methods and training the next generation of researchers, says the prediction platform “could bring powerful and constructive change to the way we think about research results. One of its great strengths is that it capitalizes on the wisdom of the crowd, potentially tapping the collective knowledge of a field to help establish a scientific consensus on which new research results can build.”

　　伯克利哈斯大学（Berkeley Haas）教授唐·摩尔（Don Moore）一直倡导更透明、更严格的研究方法，并培训下一代研究人员，他说，预测平台“可以给我们思考研究结果的方式带来强有力的建设性改变。它最大的优势之一是利用了大众的智慧，潜在地利用了某一领域的集体知识，帮助建立科学共识，新的研究成果可以在此基础上建立。”