More dialogue on strategic funding of Open Science
The following are the slightly redacted notes taken during a phone conversation this morning between Janet Haven and me on ways in which the Open Society Institute's Information Initiative could support Open Science.
JH (per email):
We'd like to ask you to think about two to three emerging opportunities for--or threats to--open society institutions and values that you are aware of which are not receiving sufficient attention and where a funder like OSI could usefully intervene. We encourage you to suggest issues that are still very much on the horizon; there need not be an obvious solution to the points you raise.
DM (in blog post mentioned above):
- support open collaborative environments for research funding, research, and knowledge structuring (see post and discussion at http://ff.im/gpry3 )
- support science prizes/ competitions for research done in the open (see http://ff.im/gpry3 ), or specific scientists/ labs working in the open (possibly part-time on "open", part-time on "science")
- promote diversification of the measures used to assess the impact of a researcher - http://ff.im/ghGML and http://ff.im/gvfKg
- support a test of the efficiency of non-public peer review - http://ff.im/gvfKg and
On the phone:
--what are the barriers to this, culturally and institutionally?
DM: "PIs are wise to propose things they have already done but not disclosed" http://ff.im/gpry3
Why is there a relatively small group of scientists advocating for these types of changes if they are beneficial to all?
While working through these notes, I came across this quote from Machiavelli, 1469: "It must be remembered that there is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage, than the creation of a new system. For the initiator has the enmity of all who would profit by the preservation of the old institutions and merely lukewarm defenders in those who would gain by the new ones" (seen at http://knol.google.com/k/knol-challenges-the-medical-journals-stronghold... )
--most people afraid of losing their ideas and putting them on the internet, albeit these concerns are not well-founded (short discussion at http://blog.openwetware.org/scienceintheopen/2008/01/27/sharing-is-carin... , longer ones at http://blog.openwetware.org/scienceintheopen/2007/11/14/getting-scooped/ and http://onebiglab.blogspot.com/2008/01/is-danger-of-being-scooped-field.h... ).
--traditions: nobody has done it so far, no funding agency is supporting it, some support concept but not research or enforced -- in UK major research councils suggest data should be made public, nobody checks though
--publishing in the sense of doing science in the open is different from the way it's used now (to mean scientific paper) -- open science is published before it is (formally) published, and for many, only the latter counts
--no real tools where can do all things online -- best set of tools is a combination (see slideshare on blog post), no single set of tools (OpenWetWare simply does not cover the whole chain yet) -- but individual services can disappear (etherpad, friendfeed have recently been bought)
--Friendfeed (in contrast to Twitter is archived, threaded, can import feeds from anywhere; http://ways.org/en/node/12335 )
-how does the tenure process fit into the puzzle?
Not well yet, mainly because the major metrics used in assessments for tenure do not include ones that would reflect open activities. http://ff.im/ghGML
I understand that assessing impact is part of tenure process; is there another agenda to change tenure process altogether, reform the structure of the university?
--open science critical of the whole structure of science, including tenure process, hurdles and barriers that impede science as a whole and impede you as an individual
--what are the incentives that would cause individuals to come over to this cause?
DM: Hopefully (still not really tested yet), open science approaches are more efficient (less resource use, faster results - consider what the Polymath project delivered in six weeks and how long this would have taken by means of normal grant schemes) and also more personally rewarding because everything you do is visible and invites interactions with other people who share an interest in the subject or methodology of your work. So you have multiple interactions with people relevant to your work (or to the way you work) and who you would rarely meet otherwise - specialists and lay people alike, or even librarians.
--are there vocal opponents to open science?
--not really in terms of the whole research cycle, but for specific aspects thereof they do exist: the individual scientific publishers, for instance are organized b/c open movement is strong enough to change something
--other aspects no visible opposition other than the above-mentioned traditions
--open data issues become more and more pressing, what is best way to do it legally, impose restictions for re-use or not (best license: cc 0/public domain), problem with interoperability; generally the legal tools are not there or unknown, scientists are unsure about whether they are legally allowed to publish their work openly -- see also http://pantonprinciples.org/ and http://ff.im/fak7J
-- fear of being scooped by someone else (as addressed in another question above)
--public peer review movement (of papers) may provide additional momentum and serve as a model also for grant proposal peer review and opening up other steps in the cycle
what are their arguments?
--on Open Access: mainly straw men arguments to defend current position
who is it bad for? what are the risks? are there situations when scientific data should be kept secret, or research processes should be secret? how does state vs privately funded research fit into this?
DM: Open Science, in the current predominantly non-open system may first and foremostly be bad for those doing their research in the open because they give something to the scientific community without being sure of the returns. In the little practice that exists so far, though, the community has generally given positive feedback along with negative one, and there is no need to switch from "completely closed" to "completely open" all at once. So even Open Scientists sometimes submit their research to non-OA journals (but they always practice green OA), and very often to journals whose peer review does not take place in public (e.g. because journals with public peer review are very rare still). Data concerning personal information of research subjects or patients should not be made public, and there is always discussion about whether or not (or to what extent) research on things like bioterrorism should be published. Most of the Open Science movement concerns publicly funded research, though, for instance, even some companies outside the IT sector (which has learned to live fine with open Source) have started to go public with some of their patents: http://greenxchange.force.com/ .
--what role do you see individuals scientists playing vs institutions (science commons, for instance)?
--Open Scientists need role models (similar to gender, diversity), individuals play this role, and they act as beta testers of the new system, and talk/write about it
(this is an open access paper on research that DM does with outlook section on open data)
--Institutions: They could free individual scientists or groups from the publish or perish game, but that is very rarely done. Also problem that most the people working in most funding agencies do not actually do science (any more) and judge the suitability of the prgrams they set up on the basis of their interactions with scientists or their memories from the time when they were still doing science. But the times have changed tremendously in the last couple of years, so that many of these past assumptions or experiences are not really applicable any more, and they do not even notice.
--two conferences: Science Online (2 per year)
-relevant sessions at related conferences (e.g. WikiSym; Wikimania as well as ESOF). Next up: http://eurodoc2010.doktorat.at/category/science-20/
-I don't fly to conferences
Which institutions could be considered partners of Open Science?
--PLoS, Frontiersin.org on OA publishing, repositories on other aspects of OA
--ScienceCommons, Open Knowledge Foundation and institutions like the Oxford Internet Institute on legel framework for and societal implications of Open Science,
--FundScience.org, Innocentive, Hypios, NineSigma as first prototypes of (semi-)public research funding environments
-- Citizendium (see charter, first wiki writing up a charter)
http://en.citizendium.org/wiki/CZ:Charter_drafting as first (or actually second, Wikipedia being the first one) prototype of open collaborative knowledge structuring environment; others include Scholarpedia and Encyclopedia of Earth
--OpenWetWare as a prototype of an open research environment
--the Open Source communities
--to some extent the library communities, but interactions between them and scientists are rather rare (they do take place on online platforms like Friendfeed, though)
--a good place to watch out for new developments and players is
--does the open science community frame any of this discussion as being about inclusion of scientists working in developing world contexts? in general, how do the ideas behind open science interact with ideas about expanding the field of scientists (to other professionals cut off from publications, to "citizen" or "crowdsourced" science?)
--in general, above is not a focus for open science, but it is for WAYS, one of the reasons WAYS was established -- problems with stable internet access from those countries, etc, so http://ways.org/ is providing infrastructure that people there cannot easily use, and multiple similar networks compete for attention of scientists in the developed world
--science policy in those countries is very often concentrated around copying policy elements from scientifically "successful" countries (as measured in terms of number of and citations to publications, for instance), but rethinking the science process on the basis of its long-term purpose and the currently available technologies or experiences, developing countries could actually "leapfrog ahead by adopting from the start science grant systems that encourage innovation." ( http://ways.org/en/node/3496 )
Is there some interaction between Open Science and Citizen science?
DM:--citizen science is one way of engaging public, giving back from what science receives, friendfeed group dedicated to citizen science: http://friendfeed.com/citizen-science
in some fields large overlap (astronomy, ornithology, palaeontology - example at http://opendino.wordpress.com/2010/02/23/the-analyses-ahead/ )
JH:see: http://www.eifl.net/ (electronic Information for LIbraries)
DM:--I know them from some of their projects in West Africa, where WAYS has quite a few members
--who (or are there) funders with a major interest in this area?
DM: On science in developing countries, there are of course http://twas.ictp.it/ and http://www.icsu.org/index.php and http://www.bibalex.org/bva2010/home/home.aspx but all of them focus - like any other funder - on projects and individuals, not on the structural foundations of science that are affected by open movements. All of these collaborate with WAYS.
--what is an "Open Science Lab" exactly? (I assume one that incorporates all the elements above, but is there something more "official" to the name?)
Nothing "official" there yet, just the visible strive for doing research in the open is usually enough to wear the label at the moment, since even the most open people/groups do some aspects of their research (particularly grant submission/ manuscript submission) through non-open channels, simply because open ones are not yet there. But things are moving quickly on these fronts, so I wouldn't be surprised to see something similar to the Panton Principles (on Open Data) being declared on Open Science as a whole in about five years from now.
--what are the best points of pressure in this field? is it to support experiments/demonstrations, policy work (on legal structures),
DM: It is first and foremostly to do good science, and to do it in a way that others (e.g. those who are usually seen as your competitors) take notice and start to think about their approach to science from a new perspective. Many of the other aspects have to be tackled in parallel - providing infrastructure (always hard to get funded with a long-term perspective) for doing science in the open, providing the legal framework and means to facilitate the transition such that neither the science nor the scientists suffer too much. And yes, scientists have to speak up on matters of politics in their area of expertise, but even this is facilitated in principle by doing science in the open, as this demonstrates to the public that science is a process, in the result of which assumptions are placed under scrutiny on a regular basis, which may seem as an overturn from the outside. However, this process of opening science up has to be accompanied with more outreach on these procedural aspects, so as to avoid the negative hype created by things like ClimateGate.
--There is also a need for funding for open collaborative environments (or to advocate for others to fund), e.g. science prizes related to political targets like the Millennium Development Goals, but make condition to follow open science principles -- if there is not a funding opportunity, people won't even think about it
--Diversification of measures can be promoted, although perhaps OSI can't do much more about it
--Efficiency of peer review has never been tested in detail, that's something that would be worth looking at -- treated as holy dogma, but no foundation for this
-- My expectation: It would probably be more effective if it were to happen in the open, community is likely to find weak and strong points than two or three reviewers under time pressure.
--opening up peer review is a priority (it comes in handy that the growing number of journals expericne a shortage of qualified reviewers, partly because people are not willing to work hard when these efforts are not seen by their funders and promotion committees). As mentinoed above, setting up a prize for someone who comes up with an efficiency test would be a good start.
--funding hybrid positions part active science part science policy (participation in work on open data, etc that isn't directly related to research, infrastructure for open science, etc)
Example for such a funder that concentrates on infrastructure:
http://www.stfc.ac.uk/ (scientific home to Cameron Neylon, one of the most prominent advocates of Open Science).
--There is an upcoming blog post on opening up science funding, following up on the "fantasy science funding" theme that ran through several blogs (including this one).
OSI Information Initiative's delicious stream:
I am pretty confident that this is not the last interaction we are to have with the Open Society Institute on matters of Open Science, and of course, other funders and interested parties are always invited to comment, as open scientists do.