Kirkpatrick's Four Levels of Training Evaluation

The topic of training evaluation seems particularly relevant to those who would be educators, whether or not they would work in a corporate or an academic environment. I thought that those of you who have not been exposed to Kirkpatrick's taxonomy for training evaluation might find it interesting and potentially useful in considering the nature of learning and some of the goals of education.

Level 1 = Reactions
- A Level 1 evaluation merely guages a learner's reaction to a program.
- The focus at this level is on questions like, "Did you like it?", "Was it relevant?", "Would you recommend it to your colleague?"
- This is the least expensive, the most common, and also considered the weakest form of training program evaluation.
- According to Kirkpatrick, this is the minimally requisite level of training program evaluation.
- This level of evaluation has consequences for Level 2. Although a positive reaction does not guarantee learning, a negative reaction can significantly inhibit learning. If learners do not like the training, they will not spend time with it, will not advocate it, and may spread bad new about it.
- Issues such as equipment problems, and at a superficial level, instructor effectiveness (e.g, "charm" and aestheticism) can be captured in a Level 1 evaluation.
- If you are a course developer, gathering information like this can be essential in determining whether or not the course was appreciated (and thus, deemed desirable and valuable) by your external (or internal) client.
- Tools: "Happy sheets" or "smile sheets." Questionnaires, interviews, observations and focus groups are used less frequently, but are still quite viable.

Level 2 = Learning
- Evaluation at this level measures whether learning took place, i.e., change in K/S/A's (knowledge, skills, attitudes).
- Most Level 2 evaluations are administered at the end of a course and so have the limitation of not being able to assess retention over time.
- Thus, information about mastery of the content is also limited in a typical level 2 assessment.
- Tools: Pre- and post-tests. Empirical evidence of bona fide learning (vs. mere accessing of prior knowledge) requires both a pre-test and a post-test.

Level 3 = Transfer/Performance
- Level 3 evaluation assesses change in behavior as a result of training. For example, after taking a face-to-face leadership class, did people modify their leadership strategies in accord with the demands of the situation?
- For obvious reasons, level 3 evaluations are more difficult to perform and for some content areas, more difficult to reliably assess (e.g., ethics). Evidence of behavioral or attitudinal change may only be perceived by the learner over time.
- Tools: Observation and interview before and after, over time are required to assess change, relevance of change, and sustainability of change.

Level 4 = Business or Environmental Results

- Did the training program make a measurable difference in the organization's productivity or have an environmental impact? This level of evaluation measures the impact of a training program on business results, e.g., production, quality, sales, accident frequency.

- This is the most expensive and the least likely performed of Kirkpatrick's four levels of evaluation, but it can also be the most persuasive means of arguing the benefits of training to a corporate organization focused on profits.

- Aside from the need to access confidential financial information, due to the difficulty in linking business results directly to training, it is particularly difficult to interpret results in financial terms.

Level 5 (Not Kirkpatrick's, but mentioned by some) = ROI
- Sometimes classified as a type of Kirkpatrick Level 4 evaluation
- Level 5 evaluation assesses the impact of the training on the organization's bottom line--ROI (Return on Investment).
- There are so many factors that are required to accurately assess ROI of training that there are also a variety of methods with variable reliability.
- Tools: Various, e.g., ROI equations

Note: Kirkpatrick also classifies competence into four levels.


References

Kirkpatrick, D. L. (1996). Evaluating training programs:The four levels. San Francisco: Berrett-Koehler. Amazon.com

Kirkpatrick's Four Levels of Evaluation
San Diego State University College of Education
http://coe.sdsu.edu/eet/Articles/k4levels/index.htm

The New Kirkpatrick Four Level Evaluation Certification Program
Corporate University Enterprise, Inc.
http://www.cuenterprise.com/cue_evaluation.php

Symbolic frame.....Debi and Bala

http://garnet.acns.fsu.edu/~bg03/Class_assignment%5b1%5d.mht

April 5 Change Management Group Assignment

This is where each of the four groups should post its response to the class activity assigned at the end of the Change Management class on April 5.

Scenario: You are one of four new faculty members that have just been hired one week ago by a College of Information (COI). Yesterday, the university Provost directed the Dean of the COI to revamp and significantly expand the school’s extant distance learning program to include all of the face-to-face courses now offered in the new undergraduate program. All fourteen faculty (10 continuing faculty and 4 new faculty) will participate in the redesign and implementation. What is more, all fourteen faculty members will participate in teaching the 5-credit hour distance Information Science course required of all new distance IS students. Per the instructions of the Provost, this revised instructional program must be implemented within 30 days, at the beginning of the new semester.

The Provost has become enamored with full-featured Web conferencing systems (Webcons) as a means for alleviating the distancing effect inherent in distance education and as a strategy for positioning the new distance program as one of the most modern. She is committed to technology that will enable remote guest lectures, open question-and-answer sessions, small-group work, and collaborative writing--all of which can be conducted live, with participants able to see and hear each other. The planned full-featured Webcon will include live audio and video. You happen to know that Webcons now on the market are able to smoothly integrate live video and audio to create a higher-order virtual learning environment that approximates its face-to-face counterpart, in addition to permitting synchronous and asynchronous communication and having many of the functions of course management systems.

Typical barriers to widespread Webcon implementation include funding and human resource issues.

Though you are a new Ph.D., your interview, your background in multimedia and knowledge of Internet technologies, in addition to your considerable experience in project management and teaching at-a-distance, both as a T.A. and as an adjunct instructor, make you the most qualified faculty member (new or old) to successfully lead and expedite this project. Rather than starting your new faculty job a few months from now, as you had negotiated, the Dean has asked you to relocate immediately in order to take charge.

Assignment (30 minutes):
Using the frame that your group was assigned, develop a management plan. In doing so, answer the following questions.

1) What factors would you consider?

2) Specifically list as many barriers to this change as you can.

3) Identify relevant strategies that you would use to combat/accommodate each barrier and effectively manage these changes and this project?

Reference

Foreman, J., and Jenkins, R. (2005, April/May). Full–featured Web conferencing systems. Innovate: Journal of Online Education, 1(4).
http://www.innovateonline.info/index.php?view=article&id=45

Change Management Preclass Assignment

Please post your preclass assignment here.

Provide a brief paragraph describing an authentic example of a problem dealing with managing change at the personal or organizatoinal level, especially in your work or academic career. For instance, you might provide a description of a problem you have had in a meeting dealing with change. Ideally, the example you provide will be one in which you have been personally involved, but it can also be a story that you have heard about and know well. Obviously, you will not reveal the names of individuals or institutions. We will discuss the examples in class on April 5.

Thank you in advance for your help in preparing for our class.

Alison

Change Management April 5 Class

The following review of the supplemental (and N.B., optional) reading, Change: Principles of Problem Formation and Problem Resolution, represents content that I will not be able to adequately cover in class on April 5. I realize that few will be able to review this in its entirety prior to class. However, you may want to consider it more carefully after the class.

Change: Principles of Problem Formation and Problem Resolution
Paul Watzlawick, John Weakland, & Rober Fisch. (1974).

I. Change’s Subject Matter and Authors’ Purpose

A. Subject

• Change Management
• Persistence and Change in Human Affairs

B. Purpose/Goals

• Provide a theoretical framework for Brief Therapy (a type of psychotherapy).
• Through analysis, others will be able to replicate treatment success.

Chapters in Change
1. The Theoretical Perspective
2. The Practical Perspective
3. “More of the Same” or, When the Solution Becomes the Problem
4. The Terrible Simplifications
5. The Utopia Syndrome
6. Paradoxes
7. Second-Order Change
8. The Gentle Art of Reframing
9. The Practice of Change
10. Exemplifications


II. Theoretical Framework

Watzlawick frames his conception of change on three theories:

A. Group Theory (Évariste Galois, 1811-1832)
B. Theory of Logical Types (Alfred North Whitehead, 1861-1947 & Bertrand Russel, 1872-1970)
C. Systems Theory (Ludwig von Bertalanffy, 1901-1972)

I will review group theory and the theory of logical types, but forgo a discussion of systems theory.

Through the authors’ observations of individuals, families, and wider social systems enmeshed in a problem in a persistent and repetitive way, despite desire and effort to alter the situation, two questions typically arose: “How does this undesirable situation persist?” and What is required to change it?” (p. 2). The authors use two abstract and general theories from the field of mathematical logic to structure their conclusions. Group theory is used to provide a framework for patterns of change.

A. Group Theory (Évariste Galois, 1811 - 1832)

• Basic postulates are concerned with the relationships between elements and wholes.
• Provides a general description of things that do not change.

The authors use the Theory of Groups to frame thinking “about the peculiar interdependence between persistence and change which we can observe in many practical instances where plus ça change, plus c’est la même chose” (p. 6).

The components of the totality are called “members” (or, "elements) and the totality is called the “group.”

Four main principles of Group Theory (Properties of Groups)

A group* G is a set with a binary operation G x G --> which assigns to every ordered pair of elements x, y of G a unique third element of G (usually called the product of x and y) denoted by xy such that the following four properties are satisfied:

“Every psychological extreme secretly contains its own opposite or stands in some sort of intimate and essential relation to it…There is no hallowed custom that cannot on occasion turn into its opposite and the more extreme a position is, the more we may expect an enantiodromia, a conversion of something into its opposite” (Jung, 1952, p. 375, cited by Watzlawick, p. 20).

1. Closure law: A group is the set of all elements that are all alike in one common characteristic. The elements of the set (or, members of the group) can be numbers, objects, concepts, events, etc., as long as they have a common denominator and as long as the outcome of any combination of two or members is itself a member of the group.

This group property allows for myriads of changes within the group, but makes it impossible for any member or combination of members to place itself outside of the group.

*Dolciani, Beman & Wooton (1970) provide the following definition of a group.

"A 'group' in the mathematical sense is simply a set of elements, together with an operation defined for them, which has the properties of closure, associativity, existence of an identify element, and existence of an inverse or reciprocal for every element" (p. 461).

According to Encyclopedia Brittanica Online a group is a "set that has a multiplication that is associative [a(bc)] = (ab)c for any a, b, c] and that has an identity element and inverses for ell elements of the set."

2. Associative law: No matter what order elements of a group are added to each other, the result is always the same. E.g., 4 + 2 = 2 + 4

The members of a group can be combined in varying sequences, yet the outcome of the operation remains the same. “…there is changeability in process, but invariance in outcome” (p. 5). In the realm of human interaction, “Behavior B” applied to “Behavior A” = “Behavior A” applied to “Behavior B” (i.e., aob=boa)

The members of a group are conceived of as unchanging in their individual properties; what may undergo considerable change is their sequence, ther relations to each other, and so on.

Application of the Associative law to human behavior:

Two people (e.g., two spouses) may, for one reason or another, maintain a certain emotional distance between each other. “In this system, it does not matter if either tries to establish more contact, for every advance by one partner is predictably and observably followed by a withdrawal of the other, so that the overall patter is at all time preserved” (p. 16).

The drinker who responds to his spouse’s criticism for drinking by drinking more, whose wife then criticizes more, and so on.

Both examples result in the system’s homeostasis. The causality of the sequence is circular, rather than linear; whether a given action is the cause or effect is not determinable. Other examples include armaments races

3. Identity element: A group always includes an identity member. The addition of the identity member to any member in the group results in the element remaining unchanged. In groups whose rule of combination is additive, the identity member is zero (e.g., 5 + 0 = 5). “In groups whose combination rule is multiplication, the identity member is one, since any entity multiplied by one remains itself. If the totality of all sounds were a group, its identity member would be silence; while the identity member of the group of all changes of positions…would be immobility” (p. 5). Watzlawick uses the identity member as a special case of group invariance. Specifically, a member of a group may act without making a difference.

The identify element, “0,” represents zero first-order change when combined with any other member.

4. Inverses: In a system satisfying the group concept, any element in a group has a reciprocal element. E.g., The reciprocal of - 5 is 5. In the equation 5 + (-5) = 0, the addition of the reciprocal produces a marked change, but the result is, itself, a member of the group (of positive and negative integers, including zero).

(For a more mathematical [and precise] explanation about the properties of groups in Group Theory, reference Wolfram Research's "Group" page and the "Abstract Group Concept Page.")

Our concepts of things are bounded and reified by the notion of their opposites, which serve as a type of non-example, e.g., light and dark, good and evil, past and future. (For more on this topic, see Freud’s essay “The Antithetical Sense of Primal Words.”)

Watzlawick, et al., relates a change intended to radically break with the past. During the early stage of the Chinese Revolution, the Red Guards oversaw the destruction of all public signs (of streets, shops, buildings, etc.) which contained any reference to the reactionary, ‘bourgeois’ past, and their replacement by revolutionary names. Paradoxically, “in the wider context of Chinese culture, this break is fully in keeping with that basic rule which Confucius called the ‘rectification of names’ and which is based on the belief that from the [‘correct’] name, the [‘correct’] reality should follow. In effect, therefore, the renaming imposed by the Red Guards was of the first-order change type; it not only left an age-old feature of Chinese culture intact, but actually re-emphasized it. Thus there was no second-order change involved, a fact that the Red Guards probably would have had difficulty appreciating." (Watzlawick, et al., p. 19)

Watzlawick points out that though things may seem to be “’as different as night and day,’ and that the change is extreme and ultimate, paradoxically, nothing may have changed at all…One of the most common fallacies about change is the conclusion that if something is bad, its opposite must of necessity be good” (p. 19).

Heraclitus called the interdependence of opposites “enantiodromia.” Jung adopted the idea and saw it as a fundamental psychic mechanism:

Watzlawick identifies several examples of enantiodromic [footnote 1] patterns in history, one of which is the romantic idealization of women by the troubadours in the eleventh to thirteenth centuries and its religious counterpart in the cult of the Virgin Mary from the eleventh century onward was paralleled by the witch hunts.

“What Group Theory cannot give us is a model for those types of change which transcend a given system or frame of reference. It is at this point that we have to turn to the Theory of Logical Types” (Watzlawick, et al., p. 6).


B. Theory of Logical Types* (Alfred North Whitehead, 1872-1970 & Bertrand Russel, 1872-1970)

• Provides a model for those types of change which transcend a given system or frame of reference
• Similar to group theory in that it talks about groups of things
• Hierarchies of logical levels

As in Group Theory, the components of the totality are called “members” however the totality is called “class,” rather than “group.”

*Note: Watzlawick refers alternatively to the "theory of logical types" and the "theory of logical levels." These variable denominations signify the same theory.

Principles of the Theory of Logical Types

1. There is a collection of things which are united by a specific characteristic common to all of the things in the collection.

2. Components of the totality are members.

3. The totality, itself, is called class rather than group.

Fundamental axiom of Group Theory: “Whatever involves all of the collection must not be one of the collection” (Whitehead & Russel, Principia Mathematica, p. 101, cited by Watzlawick, et al., p. 6) Namely, a class (consisting of specific members) cannot be a member of itself. For example, the human race consists of all the human beings but the human race is not a human being. A "logical typing miss" here will produce, for instance, confusion between the map and the territory or make a schizophrenic eat the menu instead of the food which is described on it.

Watzlawick utilizes Gregory Bateson’s (1979) insight to describe the phenomena of change in terms of hierarchies of logical levels:

1) y = static placement
2) change in place = y’ = motion/velocity
3) change in motion/velocity = y” = change of change (or, metachange) of position = acceleration/deceleration.
4) change in acceleration/deceleration = y”’ = change of change of change (or, metametachange) of position = jerk

This example illustrates the notion that change involves the next higher level: to proceed for instance, from position to motion, a step out of the theoretical framework of position is necessary. Within that framework, the concept of motion cannot be generated…and any attempt at ignoring this basic axiom of the Theory of Logical Types leads to paradoxical confusion” (p. 7). Watzlawick identifies an analogous situation in dealing with language.

“Myriads of things can be expressed in a language, except statements referring to that language itself. If we want to talk about a language, as linguists and semanticists have to, we need a metalanguage which, in turn, requires a metametalanguage for the expression of its own structure. As early as 1893, the German mathematician Frege pointed to the need to for differentiationg clearly ‘between the cases in which I am speaking about the sign itself and those in which I am speaking about its meaning. However pedantic this may seem, I nevertheless hold it to be necessary. It is remarkable how an inexact manner of speech or of writing…can eventually confuse thought, once this awareness [of its inexactitude] has vanished’ (Frege, 1893, p. 4, cited by Watzlawick, p. 8).

Watzlawick, et al., draws two important conclusions from the postulates of the Theory of Logical Types:

1) Logical levels must be kept strictly apart to prevent paradox and confusion; and
2) Going from one level to the next higher (i.e., from member to class) entails a shift, a jump, a discontinuity or transformation—in a word, a change—of the greatest theoretical and practical importance, for it provides a way out of the system.

[The Theory of Logical Levels is evident in Edwin Abbott’s (1838-1926) satirical work Flatland. Abbott illustrates the difficulty that “A Square,” who lives in two dimensions, experiences in imagining a three-dimensional reality.]

Watzlawick reminds us of the famous liar paradox of Epimenides:

"Epimenides was a Cretan who said, 'Cretans always lie.'" I have presented this paradox in the form of a quotation with a quotation, and this is precisely how the paradox is generated. The larger quotation becomes a classifier for the smaller, until the smaller quotation takes over and reclassifies the larger, to create contradiction." (Bateson, 1979, p. 125)

From the point of view of Russell and Whitehead's theory of logical types, the paradox is a result of the fact that a class cannot be its own member.

Group Theory provides a framework for thinking about the kind of change that can occur within a system that itself stays invariant; the Theory of Logical Types is not concerned with what goes on inside class, i.e., between member and class and the peculiar metamorphosis which is in the nature of shifts from one logical level to the next higher.

Watzlawick distinguishes two different types of change: one that occurs within a given system, which itself remains unchanged, and a second type of change whose occurrence changes the system, itself. He uses dreaming to exemplify the distinction. First-order change happens when a person having a nightmare does a variety of things in the context of the dream, e.g., running, screaming, hiding, fighting—none of which will terminate the dream. The only way out of the dream is to wake up. Waking is not a part of the dream, but a change to a different state, and as such, constitutes a second-order change. According to Watzlawick, second-order change is “change of change;” it is always in the form of a discontinuity or a logical jump, so may appear as illogical and paradoxical.

First-order change

• Elements of a system are rearranged, but the rules by which they inter-relate remain unchanged.
• No systemic change

First-order change = homeostasis = different behaviors out of a finite repertory of possible behaviors are combined into different sequence, but leading to “identical” outcomes. [I think Watzlawick meant “equivalent” outcomes.]

Second-order change

• Changes in the body of rules governing their structure or internal order
• System changes qualitatively and in a discontinuous manner

Second-order change involves stepping out of an old framework into a new one. While first-order change always appears to be based on common sense...second-order change usually appears weird, unexpected, and uncommonsensical. The use of second-order change techniques lifts the situation out of the paradox-engendering trap created by the self-reflexiveness of the attempted solution and places it in a different frame.

"But, as mentioned already, it is only from inside the box, in the first-order change perspective, that the solution appears as a surprising flash of enlightenment beyond our control. In the second-order change perspective it is a simple change from one set of premises to another of the same logical type" (p. 26).

The nine-dot problem is referred to throughout the book as an example of a problem for which second-order change perspective is effective. [I will demonstrate the nine-dot problem in class.]

Watzlawick, et al., promote the idea of second-order change through paradox. This strategy is remarkably similar to training by Zen masters. The typical Zen koan is designed to force the mind out of the trap of assertion and denial into that quantum jump to the next higher logical level called satori. The same principle is the basis of Hegelian dialectics, which emphasizes the process that leads from an oscillation between the dichotomies of thesis and antithesis to a transcendent synthesis. According to Wittgenstein, the way out of the fly bottle is through the least obvious opening.

"What is your aim in philosophy?--to show the fly the way out of the fly-bottle." --Wittgenstein

The essence of second-order change is illustrated in Chaucer's tale of the wife of Bath, in which a young knight is caught in the dilemma of choosing between undesirable alternatives until he chooses to reject choice itself. The knight finds his way out of the fly bottle by deciding not to continue to choose "one alternative (i.e., one member of the class of alternatives) as the lesser evil, he has to choose and thereby deals with the class (all alternatives) and not just one member" (p. 91)


III. Chapter 8: The Gentle Art of Reframing (pp. 92-109)

The authors provide two delightful examples of reframing. One example is that of Tom Sawyer marketing whitewashing. The second involves Flemish women in the film “Carnival in Flanders” effectively redefining themselves as helpless women in need of protection by the invading Spaniards, appealing to the proverbial gallantry of the Spaniards.

Reframing is defined as a “means to change the conceptual and/or emotional setting or viewpoint in relation to which a situation is experienced and to place it in another frame which fits the 'facts' of the same concrete situation equally well or even better, and thereby changes its entire meaning. The mechanism involved here is not immediately obvious, especially if we bear in mind that there is change while the situation itself may remain quite unchanged and, indeed, even unchangeable. What turns out to be changed as a result of reframing the meaning attributed to the situation, and therefore its consequences.” (p. 95)

Reframing is an effective tool of change primarily because once we perceive the alternative class membership(s), we cannot so easily go back into the trap and the anguish of the former “reality” (p. 99).

Note that this view of “reality” is contrary to the notion of an objective reality. “Real is only what a sufficiently large number of people have agreed to call real—except that this fact is usually forgotten; the agree-upon definition is reified…and is usually experienced as that objective reality, which apparently only a madman can fail to see” (p. 97).


IV. Chapter 9: The Practice of Change (pp. 110-115)

The authors’ of Change briefly stated, four-step formula for problem resolution is as follows.

1) Generate a clear definition of the problem in concrete terms.
2) Investigate the solutions attempted so far.
3) Generate a clear definition of the concrete change to be achieved.
4) Formulate and implement a plan to produce this change.

The authors explain the role that paradox plays in problem resolution.

"All human problems contain an element of inescapability, otherwise they would not be problems. This is especially so in the case of those problems that are usually called symptoms. To refer to the insomniac once more: it will be remembered that by trying to force himself to sleep, his placing himself in a "Be spontaneous!" paradox, and we suggested that his symptom is therefore best approached in an equally paradoxical way, namely by forcing himself to stay awake. Symptom prescription--or, in the wider, non-clinical sense, second-order change through paradox--is undoubtedly the most powerful and most elegant form of problem resolution known to us" (p. 114).

Footnote 1: enantiodromia: The process by which something becomes its opposite, and the subsequent interaction of the two: applied esp. to the adoption by an individual or by a community, etc., of a set of beliefs, etc., opposite to those held at an earlier stage. Hence "enantiodromiacal," "enantiodromic" adjs., resulting from enantiodromia. (from the OED Online) As in, "George W. Bush is in enantiodromic danger of becoming the oppressive wielder of weapons of mass destruction against whom he is currently inveighing."

References

Bateson, G. (1979). Mind and nature: A necessary unity. London: New York: Ballantine Books.

Dolciani, M. P., Berman, S. L., Wooton, W. (1970, Rev. ed). Book two modern algebra and trigonometry: Structure and method. New York: Houghton Mifflin.

Frege, G. (1893). Grundgesetze der Arithmetik, begriffs-shriftlich abgeleitet [Basic Law of Arithmetic], Vol. 1, p. 4. Jena: Verlag Hermann Pohle.

Freud, S. (1957). The antithetical sense of primal words. Trans. Joan Riviere. Collected Papers, Vol. 4, pp 184-191. London: Hogarth Press.

Jung, C. G. (1952). Symbols of transformation. New York: Bolingen Foundation.

Watzlawick, Paul, Weakland, John, & Fisch, Robert. (1974). Change: Principles of problem formation and problem resolution. New York: W. W. Norton & Co.

What is Science?

Daring as it is to investigate the unknown, even more so it is to question the known. –Kaspar.

Exploring what we think we know can prove fruitful in reinforcing what we thought we knew and in dispelling discontinuities and errors. So, I believe this exercise has been in my case, especially with regard to the issue of Popper’s criteria of falsification. But, that is skipping a bit ahead. In compliance with Carroll, I will “Begin at the beginning,” but not necessarily “go on till [I] come to the end; then stop.” This topic is worthy of a report, if not a dissertation.

Perhaps because the question of “what is science” is a philosophical issue, complex, or, I think more likely, so much taken for granted that there are graduate research texts that do not even index the word (Creswell, 2002, Patton, 2001). Ironically, my copy of the Dictionary of Science (Mandel, 1969) offers no definition of "science."

A useful place to begin is with the history of the word “science.” According to the Oxford English Dictionary (O.E.D.) the etymology of “science” is as follows.

[a. F. science = Pr. sciensa, Sp. ciencia, Pg. sciencia, It. scienza, ad. L. scientia knowledge, f. scient-em, pr. pple. of scre to know.]

The selected O.E.D. definitions below map the evolution of the term “science” and distinguish between general and restricted uses.

“…2. a. Knowledge acquired by study; acquaintance with or mastery of any department of learning. Also pl. (a person's) various kinds of knowledge.

b. Trained skill. Now esp. (somewhat jocularly) with reference to pugilism (cf. 3c); also to horsemanship and other bodily exercises.

3. a. A particular branch of knowledge or study; a recognized department of learning.
In the Middle Ages, ‘the seven (liberal) sciences’ was often used synonymously with ‘the seven liberal arts’, for the group of studies comprised by the Trivium (Grammar, Logic, Rhetoric) and the Quadrivium (Arithmetic, Music, Geometry, Astronomy).

b. Contradistinguished from art: see ART n. 8. The distinction as commonly apprehended is that a science is concerned with theoretic truth, and an art with methods for effecting certain results. Sometimes, however, the term science is extended to denote a department of practical work which depends on the knowledge and conscious application of principles; an art, on the other hand, being understood to require merely knowledge of traditional rules and skill acquired by habit.

4. a. In a more restricted sense: A branch of study which is concerned either with a connected body of demonstrated truths or with observed facts systematically classified and more or less colligated by being brought under general laws, and which includes trustworthy methods for the discovery of new truth within its own domain.

b. with defining word. The many conflicting systems proposed in recent times for the classification of the sciences, and the need frequently arising (apart from any formal classification) for a common designation applicable to a group of sciences that are related by similarity of subject or method, have given currency to a large number of expressions in which the word science is qualified by an adj. The application of these collocations, so far as it is not obvious, is explained under the adjs. Among the most prominent of the adjs. designating particular classes of sciences are: abstract, concrete, biological, descriptive, exact, experimental, historical, mathematical, mechanical, moral, mixed, pure, natural, physical. Also with preceding n., as life science, and combined with a prefix, as bio-, geo-, neuroscience. (See under the first element.)

5. a. The kind of knowledge or of intellectual activity of which the various ‘sciences’ are examples. In early use, with reference to sense 3: What is taught in the schools or may be learned by study. In mod. use chiefly: The sciences (in sense 4) as distinguished from other departments of learning; scientific doctrine or investigation. Often with defining adj. as in 4b. In the 17th and 18th c. the notion now usually expressed by science was commonly expressed by philosophy.

b. In modern use, often treated as synonymous with ‘Natural and Physical Science’, and thus restricted to those branches of study that relate to the phenomena of the material universe and their laws, sometimes with implied exclusion of pure mathematics. This is now the dominant sense in ordinary use.”

Encyclopedia Britannica Online notes that, “Taken broadly as the progressive improvement of the understanding of nature, the intellectual enterprise of science originally formed an integral part of philosophy, and the two areas of inquiry have never finally separated. Little more than a hundred years ago, theoretical physics—concerned with the fundamental debate about physical nature—was still described as ‘natural philosophy,’ as distinguished from the two other chief divisions of abstract discussion, viz., moral philosophy and metaphysical philosophy—the latter including ontology, the study of the deepest nature of reality or being. In fact, only during the 20th century, following the professionalization and specialization of the natural sciences, did the philosophy of science become recognized as a separate discipline.”

While the O.E.D. definitions are interesting, they still do not get to the heart of the demarcation issue--what is not science? How does one distinguish between science and non-science? And, is this distinction a matter of perspective? Would a natural scientist disagree with a social scientist about the parameters of scientific inquiry?

The answer to the latter question seems to be “yes.” Maxwell (2004) in writing about qualitative inquiry addresses this very issue:

“The reemergence of a narrowly defined ‘scientifically based research’ that marginalizes qualitative approaches represents a major threat to qualitative research. A postmodern perspective not only challenges this essentialist definition of ‘science’ but also critiques tendencies of qualitative researchers to essentialize proponents of ‘science.’ These arguments raise the issue of dialogue across the differences between researchers working in different paradigms” (Maxwell, p. 35, 2004).

Fisher (2003) critically evaluates what he claims is “the often repeated maxim that fields of study are only as scientific as they are mathematical,” an idea which would seem to dismiss any science in qualitative research. While Möller (2003), a sociologist, merely emphasizes the systematic nature of the scientific method in what seems to be a very loose definition of the concept of science: “Science points to the application of systematic methods to obtain knowledge and the knowledge obtained by those methods (Henslin, 1999)…The term scientific research implies that a disciplined method of thinking is followed about social phenomena (e.g., order, change, poverty, etc.) in order to arrive at a better understanding of the world in which live…The term scientific method refers to ‘the building of a body of scientific knowledge through observation, experimentation, generalization, and verification” (Möller, Vurm, and Petr, p. 29-30, 2003).

Still and Dryden (2004) provide welcomed insight, elucidating the conundrum. They identify the philosopher Larry Laudan (1983), who “showed that it is impossible to arrive at a definition of science which will distinguish all scientific from all pseudosicientific or non-scientific statements” (Still & Dryden, 2004, p. 265). Still and Dryden rightly note that despite Laudan’s claim, serious discussions of the problem continue in psychology, psychotherapy, and medicine, however more impassioned than the discussions in the 1980’s, which were dominated by philosophers and sociologists. They link the effort to define science as something distinct from other activities to the goal of granting epistemic authority to science. “Inclusion or exclusion can determine the future of a discipline, and even its survival” (Still & Dryden, 2003, p. 267).

Still and Dryden trace the history of the demarcation effort, and in doing so, distinguish between what they call “two overlapping traditions of big and little pseudoscience” (p. 269). “Little pseudoscience” entails claims and ideas that may have a following for a period of time, but that are seen as bizarre and easily identified as non-scientific in retrospect (e.g., flying saucers, Lysenkoism, Orgonomy, Dianetics, ESP, medical quacks). Conversely, “big pseudoscience” is associated with the philosophical issue of demarcation, and includes Psychoanalysis, Marxism, Psychometrics and Biological Psychiatry, and historically, Astrology and Phrenology. Of note, among Martin Gardner's (1952) list of characteristics of pseudo-scientists is their isolation. "Most pseudo-scientists have a number of characteristics in common...; they work in almost total isolation from their colleagues" (Gardner, 1952, pp. 8 and 10).

Still and Dryden attribute the common use of the word “science” and an “implicit, [folk] belief in some form of unity” (p. 270) to the Unity of Science Movement that was associated with the Vienna Circle during the 1920’s and 1930’s.

“The word ‘pseudoscience’ emerged from the Vienna Circle and logical positivism associated with it. In English the prefix ‘pseudo’ was used to demarcate genuinely empirical statements from others. A pseudo-question was one to which there can be no clear answer, like ‘What is the meaning of life?’ (Passmore, 1966, p. 373). The intention behind the prefix was not always pejorative. Wittgenstein (1922, p. 169) referred to the propositions of mathematics as ‘pseudo-propositions’ without wishing to diminish them…This technical use of ‘pseudo’ was the background to the use of ‘pseudoscience’ in work by…Karl Popper..[to which has been] linked his criterion of falsifiability…” (Still & Dryden, 2003, p. 270-271).

Imre Lakatos (1970), who proposed a theory of scientific research programs, and like Popper, dismissed Marxism and Freudism as pseudosciences since, according to Lakatos, they do not predict new facts or lead to the discovery of new facts, and so are not progressive. Lakatos also questioned the benefit of statistics in the social sciences: he "wonders whether the function of statistical techniques in the social sciences is not primarily to provide a machinery for producing phoney collaboration and thereby a semblance of 'scientific progress' where, in fact, there is nothing but an increase in pseudo-intellectual garbage" (Lakatos, 1970, p. 176, cited by Still & Dryden, 2004, p. 274). Lakatos went as far as recommending legislation to control such "intellectual pollution" (Lakatos, 1970, p. 176).

The physict and philosoper Mario Blunge (1967) characterized pseudoscience as aiming to influence people rather than discovering reality and as lacking a self-correction mechanism. He described the doctrines of pseudoscience as being deficient in grounding in science and untested by experiment proper (Still & Dryden, 2004, p. 274). Still and Dryden note that these criteria are "general enough to include both psychoanalysis and dowsing" (ibid, p. 274).

References

Fisher, W. P., Jr. (2003, December 1). Mathematics, measurement, metaphor and metaphysics I. Implications for Method in Postmodern Science. Theory and Psychology, 13(6), 753-790. Abstract accessed online March 19, 2005 at http://tap.sagepub.com/cgi/content/abstract/13/6/753

Mandel, S. (1969). Dictionary of science. New York: Dell.

Maxwell, J. A. (2004, February). Scientism, postmodernism, and dialogue across differences. Qualitative Inquiry, 10(1), 35-41.

Oxford English Dictionary Online. Accessed March 17, 2005.

Patton, M. Q. (2002). Qualitative research & evaluation methods, 3^rd ed. Thousand Oaks, CA: Sage.

Still, A. and Dryden, W. (2004). The social psychology of "pseudoscience": A brief history. Journal of the Theory of Social Behavior, 34(3), 266-290.

Brainstorming on Information Science education proposals

Hi Folks,
Looks like the next few months are crucial to submit grant proposals. Is anybody interested in brainstorming on simple next generation tools for information science education. I think if graduates in our school can sit and brainstorm, ideas will just start flowing in.....atleast let us give it a shot. What do you say folks?