Evaluating the quality of scientific information, or its relevance for a specific purpose, is hardly a simple endeavour. With the transition from traditional paper-based means of communication to a wide variety of communication channels, particularly online, this problem is even growing in complexity.
Fortunately, scientific information is not the only type of information on the internet, and quality assessment — to various degrees and by diverse populations — is common in many branches of society. So it may be worthwhile to consider whether some of the systems in use for other kinds of online content may be transferable to adaptable for scientific purposes.
Consider the example of movies: If you're a scientist, it may have been a while since you last went to the movies, but your memory might still hold some references to this. Anyway, a rating scheme for movies is available via OpenCritics, a non-profit startup dedicated to facilitating and openly archiving online ratings (using a Creative Commons license, as given below), and let's see how well this goes with the Science Commons video (by Jesse Dylan)
And while we are at it, what about other kinds of content? Well, you might give it a try with this presentation by Björn Brembs which addresses important shortcomings of the current scientific publication system:
The rating procedure is the same, though some of the options will probably have to be finetuned, and one could envision several such widgets being associated with one item but targeted at different evaluation criteria:
For feedback on the usefulness of such an approach for scientific purposes (also at article level, as currently being implemented at PLoS ONE, Mendeley and discussed also elsewhere), please leave a comment here or directly at OpenCritics. Thank you!
Licensing information: Creative Commons video: CC-BY-NC-SA, Ratings: CC-BY-ND, Text & Slideshow: CC-BY.
Found at http://openmed.nic.in/2789/ :
Open access to knowledge and information: scholarly literature and digital library initiatives – the South Asian scenario
Das, Anup Kumar (2008) Open access to knowledge and information: scholarly literature and digital library initiatives – the South Asian scenario. UNESCO, New Delhi, India. ISBN 9788189218218
Full text available as:
PDF - Requires Adobe Acrobat Reader or other PDF viewer.
The South Asia sub-region is now in the forefront of the Open Access movement within developing countries in the world, with India being the most prominent partner in terms of its successful Open Access and Digital Library initiatives. Institutional and policy frameworks in India also facilitate innovative solutions for increasing international visibility and accessibility of scholarly literature and documentary heritage in this country. This publication has its genesis in the recommendations and proceedings of UNESCO-supported international conferences and workshops including the 4th International Conference of Asian Digital Libraries (ICADL2001, Bangalore); the International Conferences on Digital Libraries (ICDL2004 & ICDL2006, New Delhi); and the International Workshop on Greenstone Digital Library Software (2006, Kozhikode), where many information professionals of this sub-region demonstrated their Digital Library and Open Access initiatives. This book describes successful digital library and open access initiatives in the South Asia sub-region that are available in the forms of open courseware, open access journals, metadata harvesting services, national-level open access repositories and institutional repositories. This book may be considered an authoritative Source-book on Open Access development in this sub-region.
Would [insert journal name] publish exactly the same study if all coauthors were [insert country name] scientists?Fri, 07/03/2008 - 2:11pm | by daniel
This question came up in a letter to Science, published in today's issue (subscription required):
Take a look at this journal - it is dedicated to the scientific study of what is today's most urgent topic (sustainability), approaches it from a cross-disciplinary perspective and in a way that stimulates collaboration between scientific generations, and is most appropriately (and ambitiously) named Consilience:
http://consiliencejournal.readux.org/ (full name: Consilience: The Journal of Sustainable Development).
As any ordinary journal, Consilience will accept research papers, but like Eduzendium, it will also accept manuscripts written for coursework.
Finally, it is entirely Open Access.
An online journal entirely run by Young Scientists for Young Scientists has already published three issues, with more to come. For more information, or if you want to join in, take a look at
Nature has learned, a group of big scientific publishers has hired the pit bull to take on the free-information movement, which campaigns for scientific results to be made freely available. Some traditional journals, which depend on subscription charges, say that open-access journals and public databases of scientific papers such as the National Institutes of Health's (NIH's) PubMed Central, threaten their livelihoods.
While the debate between the merits and pitfalls of Open Access and Peer Review rages on, those with the greatest economic stake in its outcome are pulling no punches in trying to ensure their survival.
What the big journals fail to mention is that open access is not meant as a subsitute for peer review, but as a companion to it. Open Access - essentially free online access to scholarly papers - is meant to complement peer review. In fact, the most succesful open access journals still use some form or another of peer review in their process. Indeed, current methods of achieving open access still use one form or another of peer review.
And what of peer review itself? While conceptually it is supposed to serve the needs of the scientific community (and therefore, by extension, the whole of humanity), there's evidence to prove that it has been responsible for more than its share of problems as well:
It has been suggested that peer review is an inherently
conservative process, that encourages the emergence of
self-serving cliques of reviewers, who are more likely to
review each othersâ€™ grant proposals and publications
favourably than those submitted by researchers from
outside the group. This could have a number of
consequences. For instance, it may:
- discourage researchers from moving into new fields in
which they have no track record;
- make it difficult for junior researchers to obtain grants
or publish their research;
- present difficulties for multidisciplinary work, since
peer review committees that do not contain individuals
qualified to judge all aspects of a proposal may be less
likely to approve the funding;
- result in the funding/publication of â€˜safeâ€™ research that
fits neatly into the conventional wisdom and work
against innovative, â€˜riskyâ€™ or unconventional ideas.
Peer review can be relatively slow and inefficient both for
funding and publication. Reasons for this may include:
- failure of referees to keep to deadlines -reviewers are
commonly given 3-4 weeks to complete and submit
reviews, but typically only 50% keep to this deadline;
- inconsistency between referees often means that more
must be sought, thus slowing the process;
- recruiting and retaining referees is increasingly difficult
(acceptance rates are typically as low as 50%);
- the lengthy time taken for editors and funding bodies
to reach a decision regarding the fate of an application
(sometimes up to six months).
The real issue is - as always - a threat to the profit margins of established corporations who are unable or unwilling to adapt to the evolving scientific ecosystem. Even the AAP (Association of American Publishers) themselves are very blunt about it:
"We're like any firm under siege," says Barbara Meredith, a vice-president at the organization. "It's common to hire a PR firm when you're under siege." She says the AAP needs to counter messages from groups such as the Public Library of Science (PLoS), an open-access publisher and prominent advocate of free access to information
It bears repeating that this isn't a debate at all between Open Access and Peer Review. "Peer Review", by nature, is an essential part of scholarly editing and screening and will likely be present in one form or another for years to come. The debate is whether it will continue to be monopolized, commercialized and hoarded for personal and monetary gain by certain publishers, instead of serving to promote the quality and dissemination of scientific articles as it should.
Open Access is the natural evolution of scientific information availability on the world-wide web, and provides access to quality scientific publications to a much, much wider audience. Combined with a structurally sound, reformed peer review process, it can form the basis for a viable, self-sustaining, highly mobile, agile scientific publishing and dissemination platform - Peer-Reviewed Open Access (PROA). As young scientists, you are called on to do the Lion's share of the work to ensure the development of the protocols, standards and resources around this platform... but you are also those who stand to gain the most from it.
Concordantly, by engaging "pitbull" PR firms to spread FUD (Fear, Uncertainty, Doubt) about Open Access, corporate scientific publishing firms are basically the moral equivalent of a child kicking at the waves as the ocean comes to bring down his sandcastle.