World Association of Young Scientists

After having skimmed the text for keywords several times before, I actually read the paper yesterday. Here are my notes (mostly quotes, some slightly reordered):
"Thus the present system of control through peer review has yet to be tested and, as such, is not itself the result of good science."
:multiple references to Braben, D. W. (2004). Pioneering Research: A Risk Worth Taking. New York: John Wiley and Sons.

:"when a grant application is not funded, the expertise that drove it, the questions it addresses, and the possible breakthroughs it offers are often lost."

:"Peer review discourages innovation, for a simple reason. While constructive criticism is necessary at some point, it is not appropriate when a scientist is embarking on something entirely new. In this circumstance, a scientist’s own confidence in success is or should be low if they are honest with themselves. If they can’t convince themselves of the certainty of early success, why should they be expected to convince their peers? The history of science is replete with example after example of major breakthroughs occurring in the face of the harshest peer reviews, see Pasteur’s fermentation theory (Gordon, 1999), continental drift/plate tectonics (Hughes, 1994), bacterial cause of ulcers (Gozlan, 2005), etc. These are undoubtedly the tip of an iceberg of other nascent breakthroughs that simply do not see the light of day. The ability of a scientist to prevail despite the resistance of peers is not a measure of the worth of the ideas, just of tenacity and alternative opportunities. Lacking these, ideas die, and society loses."

:multiple ways of fraud

:"The kind of science done under grant competitions is short term. As in politics, the horizon for funding is at most 5 years, whereas major unsolved scientific problems usually require decades or even a lifetime of commitment. The latter is recognized in the institution of academic tenure but no money backs it up, except for salary (at least in Canada)."

:"We thus now have a hidden cost, namely, the delay between the inception of an idea and finding the cash to test it (if ever)."

:"Universities tend to value their scientists in proportion to the amount of money they bring in. [..] This undermines those scientists who are working at the frontiers of their fields, which are generally lonely unfunded territories."

:"Even well-funded scientists can fear doing something new, ... The cure is often touted to be interdisciplinary work, but without funded interdisciplinarians who can cross many boundaries, real communication and innovation between scientists in narrow disciplines is impeded."

:"the variance in reviewing decisions is so high that ranking of proposals is highly random and arbitrary. This matters because the sharp cutoff system for who gets funded arbitrarily causes drastic consequences to careers."

:"The peer review system in science may have been responsible, by stifling research creativity, for the decline of global per capita
growth per annum from 2.8% during the “Golden Age” (1951 to 1974) to half of that subsequently (Braben, 2004)."

:"In the international competition for novel products and markets, and for environmentally sustainable development, for which science is the driving engine, the country that does not examine peer review critically may be putting itself at a severe disadvantage."

:"grant agencies are much fewer in number than the more than 100,000 scientific journals"

:"An important distinction between journals and grants is that journal peer review at most delays the reporting of innovation while grant peer review suppresses innovation itself."

:Two major models for alternatives - sliding scale funding and baseline funding
"Baseline funding, in which each researcher gets the same amount per year (Gordon, 1993; Poulin and Gordon, 2001). This protects each scientist from peer review. It is open to the objection that some scientists will do nothing useful with the money, and so it needs some modest checks and balances, such as requiring a public accounting of what research was accomplished. It also underfunds expensive research."

:"Grant agencies produce lists of “excellent scientific journals,” which are the journals that their excellent peer review panel members publish in. The historical fact that much seminal work had to be published first in less popular journals questions this line of thought. Much of what is deemed excellent is merely popular or the latest fad, while many deep unsolved problems go unfunded."

:"There is nothing deliberate in all of this; no conspiracy theory applies. The crescendo of excellence verbiage is a result of a positive feedback system, like grade inflation in college courses. No individuals are responsible. By the same token, no individual within the system can stop the process. However, the Federal Government could."

:"Mathematics, like science and medicine, is a multifaceted, almost fractal web of ideas and relationships, with people following
its many paths into the unknown. A one-dimensional ranking of these scholars, with sharp and severe funding and opportunity cutoffs, leaves many of these universes unexplored."

:"In a sense, there is now but one journal, the scientific literature, recorded and accessible via the World Wide Web."

:"“The challenge here-–long recognized by practicing scientists and performance measurement experts alike-–is that few direct and quantifiable measures of the larger and longer-term results of research are available” (Mitchell, 2006)."

:"in one scenario we recommended:
1. 40% of an agency’s budget be distributed evenly as baseline
funding;
2. 50% be available for regular peer reviewed competition, for
projects requiring more funds than baseline;
3. 10% be available for industry collaborations (Poulin and
Gordon, 2001).
"

:"The “mixed portfolio” approach to overall funding counters the implication that we are proposing that all grants should be small.
We are arguing that the distribution should peak at many small grants and fewer large ones, so that all eligible scientists have a
chance of seeing their ideas, if they pan out, through the embryonic stages. Equal baseline grants to all eligible applicants, plus competition for a second pot of money, accomplishes this distribution."

:"Without peer review of baseline research funding, we would not have the current scrambling of NSERC to rearrange its committees
to keep up with emergent fields that current committees suppress (Sedra, 2008)."

:"We suggest that the best policy is to support all of the eligible faculty members, who ask for funds, with baseline grants. All of
these people have passed hurdles of hiring, promotion, and tenure hearings. We claim that there are no effective criteria that will distinguish who will produce important and lasting research."

:"“Outstanding ideas do not grow on trees, and at any one time, very few scientists would expect to be working on one. That would be true most of the time even for the best scientists at the most prestigious institutes anywhere. Nevertheless, any competent and determined researcher might expect to have an inspirational idea at least once during the course of a lifetime” (Braben, 2004).
That is the negative view. The positive view is that every scientist, given freedom from peer review at the conception stages of his or
her ideas, may make a lasting contribution to humanity. If they are trained and employed to do so, why create impediments? We just waste our investment and trust in them. Furthermore, we have no tools to predict which of them will actually produce the breakthroughs. The mixed portfolio approach permits us to take advantage of the breakthroughs when they occur."

:"Controls on appropriateness of expenditures would be achieved through standard university accounting procedures and a return to the one page summary of research proposed, with another one page summary of results, together with links to data, analysis, patents, and publications for those who are interested in the details. These could be publicly available and required on a Web site, as a condition of continued funding"

:"Either review ideas, as with the present system, or accept that talented, well-trained, and vetted researchers are best able to decide their own research priorities, at least at the idea/discovery stage."

:"In no other profession than science is one given a salary and space and asked to beg outsiders (who are not customers) for
the means to do one’s work. This anomaly makes being a scientist unattractive to young people."

:"Grant Systems for Developing Countries
As developing countries make their way from poverty to higher levels of prosperity, they find themselves wanting to partake in the world culture of science (Goonatilake, 1984; Varmus, 2008). At some point, initially for addressing practical, local, urgent problems, they begin funding scientific research within their borders. They look to the granting systems adopted by the countries leading in scientific research, and start to emulate them. Yet those systems are trapped in their own dysfunctional behaviors, and perhaps had best not be copied. We would suggest, in fact, that developing countries could leapfrog ahead by adopting systems from the start that encourage innovation."

:"The central problem is for scientists to let go control of other scientists, or for this control to be wrested from them by the Federal Government. Scientists have to allow their peers to do their own thing and give up the control and judgment of peer review of
grants. It is counterproductive and does not accomplish the aims for which it is designed."

:"Granting agencies are superfluous middlemen for ordinary grants such as NSERC’s Discovery Grants."

:"In summary, we recommend that fair minded and practical government, business, and science agents work together to conduct a ‘natural experiment’ to test our hypothesis: innovation will be increased by the elimination of peer review at the idea/discovery stage. Here the NSERC Discovery Grants program could be the test case, with the past experience with this program used as the control case (Yin, 1989). There should be little objection on scientific or business grounds, since costs are comparable and the risks are no greater and most likely less. We anticipate that the elimination of the shadow of peer review at idea formulation stages, together with the advantages of fiscal carryover rights and freedom to combine funds with colleagues, would permit Canadian scientists to create and take advantage of many new opportunities and greatly increase their innovation."

:"An important example of the failure to recognize real excellence is the discovery of a chemical reaction that creates spatial
and temporal patterns (Belousov, 1958), whose publication was almost completely suppressed by peer review (Winfree, 1984). The theory of these reaction-diffusion chemical systems was developed by another science outlier (Turing, 1952; Hodges, 1983; Leavitt, 2006), and both theory and experiment have since generated a vast literature."
:Winfree, A. T. (1984). The prehistory of the Belousov-Zhabotinsky oscillator. J. Chem. Ed., 61: 661–663.

To continue the discussion and to keep up with the incoming literature on this topic, I have set up this group at friendfeed (latest posts also displayed below).