Live chat

Research paper topics, free example research papers

You are welcome to search thousands of free research papers and essays. Search for your research paper topic now!

Research paper topic: Could Gambling Save Science: Encouraging An Honest Consensus - 4785 words

NOTE: The research paper or essay you see on this page is a free essay, available to anyone. You can use any paper as a sample on how to write research papers or as a source of information. We strongly discourage you to directly copy/paste any essay and turn it in for credit. If your school uses any plagiarism detecting software, you might be caught and accused of plagiarism. If you need a custom term paper, research paper or essay, written from scratch exclusively for you, please, use our paid research papers writing service!

Could Gambling Save Science: Encouraging an Honest Consensus To appear in Social Epistemology, 1992. (version appeared: in Proc. Eighth Intl. Conf. on Risk and Gambling, London, 7/90.) C O U L D G A M B L I N G S A V E S C I E N C E? Encouraging an Honest Consensus by Robin Hanson Visiting Researcher, The Foresight Institute P.O. Box 61058, Palo Alto, CA 94306 USA 510-651-7483 The pace of scientific progress may be hindered by the tendency of our academic institutions to reward being popular, rather than being right.

A market-based alternative, where scientists can more formally "stake their reputation", is presented here. It offers clear incentives to be careful and honest while contributing to a visible, self-consistent consensus on controversial (or routine) scientific questions. In addition, it allows patrons to choose questions to be researched without choosing people or methods. The bulk of this paper is spent examining potential problems with the proposed approach. After this examination, the idea still seems plausible and worth further study. INTRODUCTION After reviewing the discrepancy between what we want from academic institutions and what we get from current institutions, a market-based alternative called "idea futures" is suggested. It is described through both a set of specific scenarios and a set of detailed procedures.

Over thirty possible problems and objections are examined in detail. Finally, a development strategy is outlined and the possible advantages are summarized. THE PROBLEM THE SCIENTIFIC REVOLUTION Four centuries ago, some Europeans complained that the existing academic institutions were biased against them. Insiders, it was said, were "inflated by letters" and shunned anyone who dared "speculate on anything out of the common way" [De]. Outsiders -- astrologers, chemists, and people like Bacon and Galileo -- argued that they and their theories should be judged by how well they agreed with observations, and not by how they agreed with the authorities of the day [Gal].

This was the age of utopias [Whi], as these rebels debated possible academic reforms and imagined whole new social institutions, for both academia in particular and society in general. Within a century or so, the intellectual descendants of these outsiders became the new insiders in a process now called the "Scientific Revolution". They introduced a new respect for observations along with new social institutions, such as the Royal Society of London, inspired by those utopian ideals. Since then science has made impressive progress. Most controversial issues of four centuries ago seem long settled by now, and continued research may well settle most of today's controversies. Academia can claim some credit for this, and academic institutions have continued to evolve in response to perceived problems, formalizing publication in journals, credit in citations, and evaluation in anonymous peer review.

PROBLEMS WITH ACADEMIA Yet little has really changed. Academia is still largely a medieval guild, with a few powerful elites, many slave-like apprentices, and members who hold a monopoly on the research patronage of princes and the teaching of their sons. Outsiders still complain about bias, saying their evidence is ignored, and many observers [Gh,Red,SmP,Syk,Tr,Tul] have noted some long-standing problems with the research component of academia. {footnote: Teaching reform is beyond the scope of this paper. I am content to observe that there are no obvious reasons why the changes I will propose should make teaching worse.} As currently practiced {footnote: Early peer reviewer consisted more of personally observing experiments and trying to reproduce analyses.} peer review is just another popularity contest, inducing familiar political games; savvy players criticize outsiders, praise insiders, follow the fashions insiders indicate, and avoid subjects between or outside the familiar subjects.

It can take surprisingly long for outright lying by insiders to be exposed [Red]. There are too few incentives to correct for cognitive [Kah] and social [My] biases, such as wishful thinking, overconfidence, anchoring [He], and preferring people with a background similar to your own. Publication quantity is often the major measure of success, encouraging redundant publication of "smallest publishable units" by many co-authors. The need to have one's research appear original gives too little incentive to see if it has already been done elsewhere, as is often the case, and neglects efforts to integrate previous research. A preoccupation with "genius" and ideological wars over "true" scientific method [Gh] needlessly detract from just trying to be useful.

Perhaps the core problem is that academics are rewarded mainly for telling a good story, rather than for being right. (By "right" I include not only being literally correct, but also being on the right track, or enabling work on the right track.) Publications, grants, and tenure are based what other insiders think today, independent of whether one's ideas and results are proved correct or valuable later. Even for researchers with a good track record, grant proposals must usually describe in some detail exactly what will be discovered and how; true exploratory work is done on the sly. This emphasis on story-telling rewards the eloquent, who know how to persuade by ignoring evidence that goes against their view, and by other standard tricks [Cia]. Admittedly, someone who has published an unusual idea that has proven right is thought of more highly, all else being equal.

But all else is usually not equal. Outsiders find it hard to get an unusual idea published, and being able to say "I told you so" is of little help to academics who have failed to gain tenure. The powerful often get credit for the successes of those under them [Re]. Only in the most experimental fields, where feedback is direct and frequent, can we expect people who are disliked -- but usually right -- to be rewarded through informal reputations. Perhaps our biggest problem is the distortion evident when a science question becomes relevant for public policy, as in the recent debates over "Star Wars" or the greenhouse effect. The popular media tend to focus on those scientists prone to hyperbole.

An honest consensus of relevant experts is often lost from public view, as advocates on each side accuse the other of bias and self-interest. Public policy can suffer dramatically as a result, a consequence that becomes more serious as the pace of technological change quickens. On the whole, current academic institutions seem less than ideal, with incentives that reward being popular, fashionable, and eloquent, instead of being right. INCENTIVES MATTER Are these complaints just sour grapes? Those who do well by an existing system tend to believe problems are minor. But even if the best ideas eventually win, we should worry if the people who advocate those ideas don't win. Good intentions and culture can only go so far in countering bad incentives; if you must publish or perish, you will do what it takes to publish (or perish). The social organization of any human effort can have a tremendous effect on its efficiency.

Consider that different past societies with different ways of organizing science have had very different rates of scientific progress; compare Europe with China over the last five centuries. Our rate of progress may be less than 2% of what it could be [Be]. Are we wasting precious resources? Imagine what would happen if we used academic peer review to decide what products to manufacture. Proposals for new products would be reviewed anonymously by powerful people who produce similar products. These reviewers would pass judgement without taking any personal risk, and those judged favorably would win regardless of how useful their product turned out to be.

I much prefer our current business system, with all of its problems, where investors must take a personal risk when they endorse a product. Institutions like the stock market are comparatively egalitarian and flexible, allowing most anyone to participate in the ongoing debate about the profit potential of any public business or the relative potential of various industries, management styles, etc. Why can't we have academic research institutions more like this? ACADEMIC REFORMS Most efforts to improve academic institutions focus on incremental reform within the existing peer review framework. Should reviewers be anonymous? Should submissions be anonymous? How many people should review each proposal? Occasionally someone proposes a more radical reform within the current framework. The surprising lack of agreement among reviewers [Cic] has lead some [Gi] to suggest we fund equally or randomly among "qualified" applicants, and let everything be published.

Conversely, the fact that a small fraction of scientists receive most citations [Co] has lead some [By] to suggest that we simple give $1M a year, no strings attached, to the top thousand scientists, chosen by an iterated popularity poll. Some have suggested universities and private labs be funded in proportion to their publication [Ro] or citation [Ts] count. And some [Tu] advocate prizes, once a central method for funding research [He]. Still others suggest scrapping the whole thing, abolishing tenure [SmP] or government funding [Fe,Wa] in favor of some existing alternative like private patrons, popular media, patents, or research tax credits. Once in a while a whole new social institution is proposed. Science courts [Kan] (also called "scientific adversary procedures") were invented to blunt hyperbole on science controversies by using court-like proceedings to encourage cross-examination and to document areas of agreement.

Hypertext publishing [Dr,Han88] imagines an electronic publishing medium where any critic could directly link a criticism to any published item, and where readers could decide what is worth reading by have software automatically combine the direct evaluations of previous readers they respect. In this paper I propose a new academic institution, tentatively called "idea futures", intended to overcome some of the limitations of existing alternatives. It is utopian in the sense of describing a coherent vision of how things might be rather different, but hopefully practical in the sense of considering what could go wrong and how to start small. WHAT WE WANT Before considering specific mechanisms, let us reflect a moment on what we want from academic incentives. We want to encourage honesty and fair play; the game should be open to anyone to prove him/herself.

Patrons who fund research, either private foundations or governments, presumably want research to be directed toward the academic subjects and questions of interest to those funders. (Patrons also include the researchers themselves, to the extent that reduced salaries are understood to be in exchange for some research autonomy.) On controversial questions, we want a clear measure of the current opinion of relevant experts, a measure which political advocates could not easily distort. And those who contribute to such a measure should have clear incentives to be careful and honest. Presumably we want as much progress as possible per effort invested, at least in situations where the following notion of "progress" makes sense. Consider a well-posed question, such as "Is the Earth basically spherical?", with a handful of possible answers (such as "No, it's flat").

Experience indicates that, with enough study and evidence, one of the answers will eventually stand out as best to most anyone who considers the question carefully. At least this seems to happen for most questions that have been traditionally labeled "scientific"; questions about the morality of abortion or the nature of God may not fare as well. Where there is such a limiting "right" answer, "progress" can mean the rate at which general scientific opinion converges to that answer. {footnote: This definition of progress is more objective than citation counts [Co], and hopefully avoids debates about whether more knowledge is good, or whether there is really an ultimate truth.} Translating these goals to an individual level, we want our institutions to reward academics for pushing scientific opinion toward the "right" answer, presumably by somehow increasing their reputation, influence, or resources. Let us imagine an academic who, after some reflection or observation, comes to a tentative conclusion which he/she would like others to consider.

If most everyone already agrees with this conclusion, even without seeing the new supporting evidence or analysis, the academic should receive little credit for just making an "obvious" claim. However, credit should be possible if the claim is surprising, i.e., if people who have not yet seen the evidence are not yet willing to agree. If, upon reviewing the evidence, most everyone now agrees with the surprising claim, then the academic should certainly receive some credit. And, in fact, peer review can handle this case. But what if there is not uniform agreement? It still seems that the academic should be rewarded, if this surprising claim is eventually born out. And others who supported this claim in the face of disagreement should also gain credit [Led], since they helped push the general opinion in the right direction. Why shouldn't savvy academics now win credit by supporting as many claims as possible, or by multiplying controversies? Clearly they should risk losing credit when they are wrong, so that credit is in some ways conserved. The ratio of possible loss to gain should depend on how unusual one's position is. Siding with the majority and being right should gain one less than siding with a minority and being right.

The total amount gained or lost should depend on how much of their reputation each academic has chosen to stake on this issue, as well as on how interesting the issue is to the ultimate research funders. In summary, part of what we want from academic incentives is a fair game for staking our reputation, so that on questions of interest to funders, we converge as fast as possible to the "right" answer. THE PROPOSAL Surprising as it may seem, such a social institution exists. It is relatively simple, cheap, decentralized, and egalitarian. It could create a consensus on disputed science questions that would be clear, expert, honest, and self-consistent across a wide range of issues.

This consensus should respond quickly to new information, and predict at least as well as any other co-existing consensus mechanism. It is well-grounded in our best theories of decision and incentives. And it is ancient. We need only revive and embellish a suggestion made back during the utopian scientific revolution. Chemical physicians, excluded by the standard physicians from teaching in the British schools, repeatedly offered challenges like the following (circa 1651): Oh ye Schooles.

.. Let us take out of the hospitals, out of the Camps, or from elsewhere, 200, or 500 poor People, that have Fevers, Pleurisies, etc. Let us divide them into halfes, let us cast lots, that one halfe of them may fall to my share, and the other to yours; .. we shall see how many Funerals both of us shall have: But let the reward of the contention or wager, be 300 Florens, deposited on both sides: Here your business is decided. [De] They proposed to bet on their medical therapies, apparently believing bets to be a useful augmentation of the existing academic incentives! Bets are a long-established and robust reputation mechanism, widely seen as a cure for excessive verbal wrangling; you "put your money where your mouth is". In science and elsewhere, phrases like "you bet" are standard ways to express confidence. Offers to make token bets are particularly compelling, and scientists of equal stature often make and publicize such bets, with recent bets on resource depletion, computer chess, black holes [Hal], solar neutrinos, nuclear weapon yields [Ev], and cold fusion [Gar,Lew,WSJ].

Nor is gambling foreign to science funding. King Charles II, founding patron of the Royal Society of London, was fond of laying wagers on the outcome of the Society's experiments [ShS]. Until 1830, public lotteries funded Colombia, Harvard, and Yale [Gei]. In 1872 Leland Stanford, founder of Stanford University, hired Eadweard Muybridge to help win his bet that a trotting horse has all four legs off the ground at some point; in the process Eadweard invented moving pictures [Jac]. Consider the example of Piers Corbyn, a London astrophysicist who has been unable to get academic meteorologists interested in his unusual theory of long-term weather cycles [NS]. Since June 1988 he has been making bets to gain publicity, betting against the bookmaker William Hill, who uses odds posted by the British Metrological Service.

And he has been winning. Over the last 26 months (4/89-5/91), Corbyn has made at least 9 bets a month (and averaged over 20 bets a month) and has won 80% of these bets, gaining an average rate of return of over 25% per bet. (Depending on what independence you assume between bets in a given month, the chance of this happening randomly is between one in 400 and one in 1050.) Yet the Service still refuses to take Piers seriously, or make even token bets against him. Which doesn't seem quite fair; hasn't Pier earned the right to be considered? William Hill has taken on the bets for the publicity, but is tired of losing, and has adjusted their odds accordingly. Why shouldn't these be the odds used for official British agricultural policy, instead of the Service's predictions? Or consider Julian Simon, a population and natural resource optimist, who found he could not compete for either popular or academic attention with best-selling doomsayers like Paul Ehrlich.

So in 1980 Simon challenged Ehrlich to bet on whether the price of five basic metals, corrected for inflation, would rise or fall over the next decade. Ehrlich accepted, and Simon won, as would most anyone who bet that way in the last two centuries. This win brought Simon publicity [Ti], but mostly in the form of high-profile editorials saying "Yeah he won this one, but I challenge him to bet on a more meaningful indicator such as .." In fact, however, not only won't Ehrlich bet again, though his predictions remain unchanged, but none of these editorial writers will actually put their money where there mouths are! And the papers that published these editorials won't publish letters from Simon accepting their challenges [Si]. Shouldn't Simon's open challenges count as much as best-sellers in setting environmental policy? - If the primary way that academics are now rewarded for being right, rather than popular, is an informal process for staking their reputation, which has various biases because of its informality, and if we want a better reputation game, why not literally make bets and formalize the process? Imagine a betting pool or market on most disputed science questions, with the going odds available to the popular media, and treated socially as the current academic consensus. Imagine that academics are expected to "put up or shut up" and accompany claims with at least token bets, and that statistics are collected on how well people do. Imagine that funding agencies subsidize pools on questions of interest to them, and that research labs pay for much of their research with winnings from previous pools. And imagine that anyone could play, either to take a stand on an important issue, or to insure against technological risk.

This would be an "idea futures" market, which I offer as an alternative to existing academic social institutions. Somewhat like a corn futures market, where one can bet on the future price of corn, here one bets on the future settlement of a present scientific controversy. This is admittedly an unusual (though not entirely original [Bru,Ho81,Ho84,Lea,So]) suggestion; but consider what might happen. SCENARIOS CONTINENTAL DRIFT In 1915 German meteorologist Alfred Wegener published his theory of continental drift, for which he had collected extensive evidence. But contemporaries considered his theory to be "impossible", and Wegener died an intellectual outcast in 1930 [Mar]. Yet in the 1960's his theory began to be taken seriously, and is now the established view.

Wegener eventually gained fame, but overall academia seems to discourage activity like his. Some of Wegener's peers, for example, probably found his thesis plausible, but decided that to say so publicly would be a poor career move. With idea futures, Wegener could have opened a market for people to bet on his theory, perhaps to be judged by some official body of geologists in a century. He could have then offered to bet a token amount at, say, 1-4 odds, in effect saying there was at least at 20% chance his claim would be vindicated. His opponents would have had to accept this estimate, and its implications about the importance of Wegener's research, or they would have to bet enough to drive the market odds down to something a little closer to "impossible".

They could not suppress Wegener merely by silence or ridicule. As Wegener increased his stake, buying more bets to move the price back up, his opponents would hopefully think just a little more carefully before betting even more to move the price back down. Others might find it in their interest to support Wegener; anyone who thought the consensus odds were wrong would expect to make money by betting, and would thereby move the consensus toward what they believe. Everyone would have a clear incentive to be careful and honest. The market would encourage more research related to continental drift, as one could make money by being the first to trade on new relevant information.

Eventually the evidence would more clearly tip in Wegener's favor, and the price of his bets would rise. Wegener, or his children, could then sell those bets and reap some rewards. While those rewards would not make up for years of neglect, at least he would get something. As the controversy became settled, and opinions converged, people would gradually sell and leave the market. Few people, if any, need be left for the final judging, which could usually be avoided (using mechanisms to be described below). COLD FUSION A more recent controversy began in March 1989, when Pons and Fleishman announced "fusion in a jar" at a dramatic press conference.

In the months that followed, media aftershocks of confirmation attempts were tracked by thousands of scientists and others, who argued with each other about the chances of cold fusion being real. Proposals to bet came up often, even in the public debates. Critics, uncomfortable with airing scientific disputes in public, complained that Pons and Fleishman broke the rules by going to the popular media instead of through normal peer review channels, unfairly gaining extra attention and funding. Supporters countered that popular media spread information quickly to other scientists; cold fusion, if right, was too important to wait for normal channels. In the journal Science, Robert Pool speculated that a market in cold fusion might have gone something like Figure 1 [Poo]. If there really had been a betting market, then there really would have been a market price that journalists like Pool could publish as news.

A table of going prices might appear on the science page in the newspaper, much like the stock page in the business section, conveying current scientific opinion better than the current "balanced" interviews with extremists on all sides. It's been suggested [Ze] that the added information in betting market prices might have helped resolve the debate more quickly. FUSION CONFIDENCE INDEX Georgia confirms Russia heat Stanford | neutrons confirms Announce fusion TexaxAM | | | in bottle confirms | | U.Wash * | Hungarian | | * tritium *** | BYU neutrons | *** * | * * | confirms | | * * * * ** | | * * * ** * ***** * | * * * * * ** ** * * * * * * * * * * Georgia * * * * * ** * reverses---* * ** ** * * * * * * * | TexaxAM * * * MIT sees hedges----* * nothing ** Mar23 Apr3 Apr10 Apr14 Apr18 Figure 1 A Hypothetical Market in Cold Fusion (Science 28Apr89) There needn't be a conflict between going through slow proper channels and getting the word out, if a fast market were a proper channel. The effect of staged media events might be reduced as it might not be news if the price didn't change; advocates would have to convince, not the average listener, but those people willing to make bets. Remaining biases, such as the overconfidence evident in figure 1, would be reduced by technical traders and other trading specialists. Cold fusion businesses would have been less risky to start. As it was, a new fusion business had to bet both that cold fusion was real, and that they were the best group to develop and market it in that case.

With idea futures they could, by both starting a business and betting against cold fusion (essentially taking out insurance), really only be betting on their ability to develop cold fusion if it were real. Insights from a great many people whose opinions on the cold fusion controversy were ignored, such as inarticulate folks without Ph.Ds, could have been integrated in a decentralized manner. Popular play would end up subsidizing professional efforts on questions of popular interest, offering more "direct democracy" in setting research priorities. NEUTRINO MASS Betting markets could also function in the absence of overt controversy, as in the following (hypothetical) story. Once upon a time the Great Science Foundation decided it would be a "good thing" to know the mass of the electron neutrino. Instead of trying to figure out who would be a good person to work on this, or what a good research strategy would be, they decided simply to subsidize betting markets on the neutrino mass. They spent millions.

Soon the market odds were about 5% that the mass was above 0.1eV, and Gung Ho Labs became intrigued by the profits to be made. They estimated that for about $300K spent on two researchers over 3 years, they could make a high confidence measurement of whether the mass was above 0.1eV. So they went ahead with the project, and later got their result, which they kept very secret. While the market now estimated the chance of a mass over 0.1eV at 4%, their experiment said the chance was at most 0.1%. So they quietly bought bets against a high mass, moving the price down to 2.5% in the process.

They then revealed their results to the world, and tried their best to convince people that their experiment was solid. After a few months they mostly succeeded, and when the price had dropped to 0.7% they began to sell the bets they had made. They made $500K off of the information they had created, which more than covered their expenses to get that information. If Gung Ho Labs had failed to convince the world of their results, they would have faced the difficult choice of quitting at a loss, or holding out for the long-term. A careful internal review would probably be conducted before making such a decision. Internally, Gung Ho would be free to use whatever organizational structures it found effective; even peer review, tenure, and fixed salaries.

The two researchers need not risk their life savings to be paid for their efforts. But the discipline of the external market should keep these internal institutions from degenerating into mere popularity contests. KILLER PEANUT BUTTER Once upon another time, Munchem Biolabs found compelling evidence that peanut butter was more deadly than most pesticides, a conclusion that Lunch Industries Exclusive (LIE) wanted desperately to suppress. LIE's usual procedure was to fund a bunch of competing studies to come to opposite conclusions, which usually kept the waters muddy enough that legislators and customers would ignore it all. But this time they had to deal with an idea futures market on the question, and the public was beginning to take the odds in such markets seriously. Munchem had moved the market odds of deadly peanut butter up rather high.

LIE now had two choices; either they could use overwhelming cash to move the odds back down, or use competing studies, advertising, etc. to persuade others to bet on their side. If they bet alone, they would know they were throwing their money away with no obvious limit on future spending. Not only might Munchem find allies, but LIE employees who knew they were bluffing might be tempted to pick up a little free money with some anonymous bets. If word of Lunch's bluff got out, as insider information often does, investors would flock in and wipe out the effect of LIE's bets.

If LIE tried to throw away other people's money through a persuasion campaign, they would face a market dominated, as most liquid markets are, by battle-hardened speculators. These investors, not easily persuaded by clever jingles, would quickly hook up with research insiders, who generally know which labs tend to find whatever results their customers want. So in the end, Lunch Industries accepted the market odds, and began research on non-toxic peanut butter. PROCEDURES Rather than just present an abstract utopian vision of market-based academic incentives, this paper aims to consider in some detail what problems might arise and possible approaches for dealing with them. The following is a core set of procedures tentatively selected to deal best with known problems, a core that will be expanded upon later in this paper.

No doubt, experience with real idea futures markets will show many of these suggestions to have been naive. I offer them primarily to make plausible the idea that betting markets could be applied to a much wider range of scientific questions than is presently considered feasible. (This ...

Related: consensus, encouraging, gambling, honest, peanut butter

Research paper topics, free term papers, essays, sample research papers on Could Gambling Save Science: Encouraging An Honest Consensus