LINFO

Peer Review Definition



Peer review is the evaluation of creative work or performance by other people in the same field in order to maintain or enhance the quality of the work or performance in that field1.

It is based on the concept that a larger and more diverse group of people will usually find more weaknesses and errors in a work or performance and will be able to make a more impartial evaluation of it than will just the person or group responsible for creating the work or performance.

Peer review utilizes the independence, and in some cases the anonymity, of the reviewers in order to discourage cronyism (i.e., favoritism shown to relatives and friends) and obtain an unbiased evaluation. Typically, the reviewers are not selected from among the close colleagues, relatives or friends of the creator or performer of the work, and potential reviewers are required to disclose of any conflicts of interest.

Peer review helps maintain and enhance quality both directly by detecting weaknesses and errors in specific works and performance and indirectly by providing a basis for making decisions about rewards and punishment that can provide a powerful incentive to achieve excellence. These rewards and punishments are related to prestige, publication, research grants, employment, compensation, promotion, tenure and disciplinary action.

Peer review is used extensively in a variety of professional fields, including academic and scientific research, medicine, law, accounting and computer software development. Even trial by jury is a form of peer review. Peer review is legislatively mandated in some situations, particularly in law and medicine. In others it is required by tradition and/or by administrative rules, such as in academia. In some fields, such as software development, it occurs naturally without any formal structure or requirements.

In the case of peer reviewed journals, which are usually academic and scientific periodicals, peer review generally refers to the evaluation of articles prior to publication. But in a broader sense, it could also refer to articles following publication, as such articles often continue to be studied and debated for a longer period and by a much wider audience.

Despite its advantages, there have also been some criticisms of peer review. One is that it can be slow, particularly in the case of academic journals, for which many months or even a year or longer are sometimes required for submitted articles to be reviewed and published.

Some critics believe that peer review has a built-in bias against highly original works and results because reviewers (as do people in general) tend to be more tolerant of works and results that are consistent with their own views and more critical of those that contradict them. It should be kept in mind that history is replete with examples of innovations that were originally ridiculed by their peers because they contradicted the common wisdom of the day2. The bias by academics against highly innovative work may be in part a result of the fact that they have vested interests in maintaining the status quo after having spent many years or decades supporting it.

Moreover, it has been suggested that peer review is not always good at detecting fraud, particularly in the case of articles submitted to scientific journals. One reason for this is that the reviewers often do not have immediate or full access to the data on which the articles are based (except perhaps in fields such as mathematics where it is easy to provide the data and attempt to replicate the results). However, longer term peer review (i.e., after the articles have been published) has proven to be much better at detecting fraud.

The Internet is beginning to have a major effect on peer review. One way has been to increase the speed and lower the cost of the communications involved in reviewing works, such as articles prior to publication. In addition, the movement of publications from hard copy format to online format, which is still in its early stages, will further increase the speed of publishing, reduce the cost of publishing, and make the publications much more widely available, thereby facilitating the post-publication review of articles by a larger number and greater variety of people.

In addition, because it allows anyone to publish and to do so at virtually no cost, the Internet makes it much easier to give swift and widespread exposure to highly innovative and controversial works that might have a difficult time being getting into mainstream, peer reviewed hard copy publications. However, there are so many articles and other items of obviously dubious quality on the Internet that it can be difficult to find those that are truly promising.

The Internet has been having particularly profound effects on peer review with regard to computer software. Most significantly, it has made possible the development of free software (also known as open source software) as a viable alternative to proprietary software (i.e., commercial software). Proprietary software is usually given relatively little independent peer review because its source code is a tightly guarded secret and is generally reviewed only by programmers, or a subset thereof, within the company developing it. Without source code, which is the form that software is originally written in by humans using a programming language, it is much more difficult for programmers to detect errors, security vulnerabilities and other weaknesses in software.

Free software, in contrast, tends to be subject to review by a much larger and more diverse group of programmers both because its source code is freely available on the Internet and because of the substantial number of people who are passionate about studying and improving it3.


________
1 The word peer is often defined as a person of equal standing. However, in the context of peer review it is generally used in a broader sense to refer to people in the same profession who are of the same or higher ranking.

2Many of the individuals responsible for the world's greatest discoveries and inventions were originally mocked and ridiculed by their peers. Among the more famous examples are Darwin's discovery of evolution in the nineteenth century, the discovery of continental drift (also referred to as plate tectonics) by Alfred Wegener and others in the early 20th century, and the Wright brothers' first heavier-than-air flights at Kitty Hawk in 1903.

3These motivations have been the subject of a great deal of study. For a brief summary, see Incentives to Develop Free Software, The Linux Information Project, July 2005.






Created December 26, 2005.
Copyright © 2005 The Linux Information Project. All Rights Reserved.