Delphi method
From Wikipedia, the free encyclopedia
The Delphi method has traditionally been a technique aimed at building an agreement, or consensus about an opinion or view, without necessarily having people meet face to face, such as through surveys, questionnaires, emails etc. This technique, if used effectively, can be highly efficient and generate new knowledge.
To build consensus, the Delphi method often uses the Hegelian dialectic process of thesis (establishing an opinion or view), antithesis (conflicting opinion or view) and finally synthesis (a new agreement or consensus), with synthesis becoming the new thesis. All participants in the process shall then either change their views to align with the new thesis, or support the new thesis, to establish a new common view. The goal is a continual evolution towards 'oneness of mind', or consensus on the opinion or view.
Contents |
History of the Delphi method
The Delphi method was developed at the Rand Corporation at the beginning of the cold war to forecast the impact of technology on warfare.
In 1944, General Arnold ordered the creation of the report for the U.S. Air Force on the future technological capabilities that might be used by the military. Two years later, Douglas Aircraft company started Project RAND to study "the broad subject of inter-continental warfare other than surface".
Different approaches were tried, but the shortcomings of traditional forecasting methods, such as theoretical approach, quantitative models or trend extrapolation in areas where precise scientific laws have not been established yet, quickly became apparent. To combat these shortcomings, the Delphi method was developed in RAND Corporation during the 1950-1960s (1959) by Olaf Helmer, [http://www.rand.org/pubs/research_memoranda/RM5888/ Norman Dalkey, and Nicholas Rescher.
The Delphi method was used by Rand Experts when they were asked to give their opinion on the probability, frequency and intensity of possible enemy attacks. Other experts could anonymously give feedback. This process was repeated several times till a consensus emerged.
Role of the facilitator
The person co-ordinating the Delphi method can be known as a facilitator, and facilitates the responses of their panel of experts, who are selected for a reason, usually that they hold knowledge on an opinion or view. The facilitator sends out questionnaires, surveys etc. and if the panel of experts accept, they follow instructions and present their views. Responses are collected and analysed, then common and conflicting viewpoints are identified. If consensus is not reached, the process continues through thesis and antithesis, to gradually work towards synthesis, and building consensus.
The Delphi method and forecasting
The Delphi method is a systematic interactive forecasting method based on independent inputs of selected experts.
The name "Delphi" derives from the Oracle of Delphi. The authors of the method were not happy with this name, because it implies "something oracular, something smacking a little of the occult", whereas in reality precisely the opposite is involved. The Delphi method recognizes the value of expert opinion, experience and intuition and allows using the limited information available in these forms, when full scientific knowledge is lacking.
Delphi method uses a panel of carefully selected experts who answer a series of questionnaires. Questions are usually formulated as hypotheses, and experts state the time when they think these hypotheses will be fulfilled. Each round of questioning is followed with the feedback on the preceding round of replies, usually presented anonymously. Thus the experts are encouraged to revise their earlier answers in light of the replies of other members of the group. It is believed that during this process the range of the answers will decrease and the group will converge towards the "correct" answer. After several rounds the process is complete and the median scores determine the final answers. From that the road-map or timetables of future developments can be derived.
The following key characteristics of the Delphi method help the participants to focus on the issues at hand and separate Delphi from other methodologies:
- Structuring of information flow
- Regular feedback
- Anonymity of the participants
Structuring of information flow
The initial contributions from the experts are collected in the form of answers to questionnaires and their comments to these answers. The panel director controls the interactions among the participants by processing the information and filtering out irrelevant content. This avoids the negative effects of face-to-face panel discussions and solves the usual problems of group dynamics.
Regular feedback
Participants comment on their own forecasts, the responses of others and on the progress of the panel as a whole. At any moment they can revise their earlier statements. While in regular group meetings participants tend to stick to previously stated opinions and often conform too much to group leader, the Delphi method prevents it.
Anonymity of the participants
Usually all participants maintain anonymity. Their identity is not revealed even after the completion of the final report. This stops them from dominating others in the process using their authority or personality, frees them to some extent from their personal biases, minimizes the "bandwagon effect" or "halo effect", allows them to freely express their opinions, encourages open critique and admitting errors by revising earlier judgments.
History
First applications of the Delphi method were in the field of science and technology forecasting. The objective of the method was to combine expert opinions on likelihood and expected development time of the particular technology in a single indicator. One of the first such reports, prepared in 1964 by Gordon and Helmer, assessed the direction of long-term trends in science and technology development, covering such topics as scientific breakthroughs, population control, automation, space progress, war prevention and weapon systems.
Later the Delphi method was applied in other areas, especially those related to public policy issues, such as economic trends, health and education. It was also applied successfully and with high accuracy in business forecasting. For example in one case reported by Basu and Schroeder (1977) Delphi method predicted the sales of a new product during the first two years with accuracy of 3–4% compared with actual sales. Quantitative methods produced errors of 10–15% and traditional unstructured forecast methods of about 20%.
Acceptance
Overall the track record of the Delphi method is mixed. There have been many cases when the method produced poor results. Still, some authors attribute this to poor application of the method and not to the weaknesses of the method itself. It must also be realised that in areas such as science and technology forecasting the degree of uncertainty is so great that exact and always correct predictions are impossible, so a high degree of error is to be expected.
Another particular weakness of the Delphi method is that future developments are not always predicted correctly by iterative consensus of experts, but instead by unconventional thinking of amateur outsiders.
One of the initial problems of the method was its inability to make complex forecasts with multiple factors. Potential future outcomes were usually considered as if they had no effect on each other. Later on, several extensions to the Delphi method were developed to address this problem, such as cross impact analysis, that takes into consideration the possibility that the occurrence of one event may change probabilities of other events covered in the survey. Still the Delphi method can be used most successfully in forecasting single scalar indicators.
Despite these shortcomings, today the Delphi method is a widely accepted forecasting tool and has been used successfully for thousands of studies in areas varying from technology forecasting to drug abuse.
Delphi applications not aiming at consensus
Traditionally the Delphi method has aimed at a consensus of the most probable future by iteration. The Policy Delphi launched by Murray Turoff instead is a decision support method aiming at structuring and discussing the diverse views of the preferred future. The Argument Delphi developed by Osmo Kuusi focuses on ongoing discussion and finding relevant arguments rather than focusing on the output. The Disaggregative Policy Delphi developed by Petri Tapio uses cluster analysis as a systematic tool to construct various scenarios of the future in the latest Delphi round. The respondent's view on the probable and the preferable future are dealt with as separate cases.
Directing the flow to a predetermined goal
The Delphi Technique can also be abused to give an appearance of community input when in reality the facilitator is directing the flow to a predetermined goal.
Education Reform
Critics of Education Reform recognize that for many issues, such as instituting a new testing system, curriculum standards, or standards-based mathematics teaching methods, typically a blue-ribbon panel is sent to study a new system. But rather than being asked whether or not it is a good idea to replace multiple choice tests with performance based tests and holistic scoring, or whether it is a good idea to adopt math textbooks which replace long division with cutting, pasting and singing, they are asked how they will implement the new system. Similarly, when public input is asked, they are given a presentation, and then broken down into facilitated groups who discuss how to implement the new math goals, rather than to debate whether the new standards are worth adopting in the first place. Since the change is often supported by an expensive and extensive public relations campaign, very few informed citizens are present to oppose the proposed changes.
See also
- John Brunner's description of a Delphi pool in his novel The Shockwave Rider
- DARPA's ill-fated Policy Analysis Market
External links
- The Delphi Method: Techniques and Applications, edited by Harold A. Linstone and Murray Turoff — a comprehensive book on Delphi method (free download, 11Mb PDF, 618 pages)
