March 2001

Psychology, Expert Testimony and the Law

by Dr. Gordon Cochrane

Lawyers and judges are regularly called on to evaluate expert scientific and medical testimony. In many cases the information — once presented — speaks for itself. The laws of physics generally inform accident reconstructions. But when it comes to the science of the mind and human behavior — psychology — the range of "expert" opinions expands, and the ability of most attorneys to really understand, much less analyze, what they hear contracts. In the following article, Vancouver, B.C., clinical psychologist and consultant Dr. Gordon Cochrane offers basic analytical tools that attorneys can use to separate valid psychological testimony from "junk science." Ed.

Most lawyers and legal decision-makers have had limited exposure to the research methods and analysis-of-variance used in psychology.¹ They also have little or no experience with the application of research outcomes to practical clinical situations. Sometimes, for example, psychology research is based on statistically sound methods, but the generalizability of the outcome data is limited, because the studies were conducted in an academic setting rather than a clinical setting. Sometimes too, the research methods of studies are sound, but key factors such as each person's unique perspectives and an individual's faith in himself to do what is required are not included in the design. For example, over the last 30 years volumes of cognitive-behavioral research have been published on coping strategies² and on treatments for obesity,³ but this research has not resulted in the development of effective treatments in either of these realms.

Purveyors of psychological pseudo-science often follow this pathway. It is true that the cognitive behavioral model is the treatment model used in most psychology research, that emotion and self-efficacy are important factors in well-being, and that these variables need to be more effectively incorporated into psychological research. It is also true that results from longitudinal studies and meta-analyses of existing studies are conducted too infrequently. "Therefore, it is true that the limitations of mainstream research make it appropriate and praiseworthy to promote pseudo-science in the guise of creative innovation." The logic is flawed, and therefore so is the conclusion.

The acknowledged limitations of mainstream research are being addressed by researchers who are exploring more effective ways to meld research design with the subjectivity of human experience. These limitations do not diminish the value of most scientific research in psychology, and they definitely do not justify the use of unproven or disproven treatments or expert testimony arising from these treatments.

But attorneys are not psychologists, and they need practical tools and guidelines for ascertaining the validity and reliability of the expert testimony that is provided by professionals in psychology.⁴ The following information and concepts offer a preliminary yet reliable framework for assessing the "expertness" of expert testimony in psychology.

An expert witness is usually asked to provide understanding of a relevant issue that is outside the ordinary knowledge of a judge or jurors, and in so doing, most experts draw upon mainstream research. Mainstream research adheres to scientific standards of research and is published in peer-reviewed journals approved by the psychology or psychiatry associations of a given country (for example, the American Psychiatric Association or the Canadian Psychology Association). Books on psychology subjects may also be reliable if original research is appropriately cited to support the theories presented in the book.

However, some authors of psychology books do not cite mainstream research to support their theories, and instead frequently cite parapsychology sources that are usually accepted without question by the unsophisticated segment of the public for whom these books are written. Unless one knows what to ask, parapsychology journals can seem as valid as mainstream journals. However, the editors of parapsychology journals rarely expect or require authors to adhere to sound research principles.

It is helpful for lawyers and judges to have a simple but reliable framework that they can call upon to assess whether expert testimony in psychology rests upon a firm foundation of science, or upon myth, bias and magical thinking.

Throughout history, many cultures in arid parts of the world have relied upon the mystical rituals of a rainmaker to bring on much-needed rain. The rainmaker was usually a charismatic and mysterious person who fostered the belief that he or she had knowledge and influence in realms unavailable to the average person. The pomp and circumstance surrounding the rainmaker and the raindance were so impressive that many important questions were never asked. When it appears that an expert witness might be a psychological rainmaker, the following questions may be helpful.

• After the  raindance, did it rain?
• Does it always, or even usually, rain after each raindance?
• Are the raindances held on all types of days or just on dark cloudy days?
• If it rained, could there be other reasons or was it because of the raindance?
• If it rained because of the raindance, is the whole dance needed or just some of it?
• If only key parts are necessary, which parts are they?
• Can anyone perform this raindance and get comparable results?
• Will the rainmaker answer questions about the raindance?
• If yes, are the answers sprinkled with specialized terminology?
• Can the raindance and its (supposed) results be explained in everyday English?

This simple example illustrates the key principles of analysis-of variance⁵ (ANOVA) and provides a framework for questioning the reliability and relevance of expert testimony. ANOVA is the primary reason for doing mainstream research. Researchers seek to determine, by comparing outcomes from the treatment in question to a control group, to other valid and established treatment groups or to a pharmaceutical product, whether there is a statistically significant difference in the treatment outcomes. The variation among the outcomes from the treatments tested indicates whether the treatment in question reliably brings about positive results (after the raindance, does it rain?). Specific aspects of a treatment model can be isolated, and their significance can also be tested in this manner.

When a particular treatment or psychological concept is presented as part of expert testimony, the expert should be asked to cite the research that supports the authenticity of this treatment or concept. Without the backing of mainstream research, the testimony can easily be challenged. If research is cited, the expert should be asked whether the journal in which it was published is an APA- or CPA-approved journal. If it isn't a sanctioned journal, the testimony can again be challenged. Of course, the research cited may be credible, but it also has to be relevant to the situation in question.

The second question (does it rain after each dance?) is about reliability. Is this treatment reliable across populations over time? One study with short-term follow-up data does not provide sufficient information to determine whether a treatment is reliable.

The research design used in mainstream research controls for extraneous variables that could influence the treatment outcome⁶ (does the raindance work on all types of days or just on cloudy days?).

By "dismantling" studies, we seek to clarify which aspects of a treatment are necessary and which are not. For example, eye movement desensitization and reprocessing (EMDR) has been a very popular treatment for trauma since 1989. However, current type-A research indicates that the patient's directed eye movement, which was originally presented as the fundamental aspect of the treatment, is not a factor in the treatment outcome.⁷ If eye movement is the distinguishing feature of EMDR and eye movement is not a statistical predictor of outcome, what is EMDR?

A psychological treatment, concept or strategy should be usable by other properly trained therapists. If it is exclusive to a particular therapist, the outcomes are a factor of the therapist rather than a factor of a treatment. Claims of unique and exclusive therapeutic success, frequently attributed to vaguely worded concepts and processes, cannot be verified by mainstream research and therefore can only be supported by unreliable anecdotal evidence.⁸

Research design and ANOVA permit researchers to make probability statements about outcomes arising from treatments or specific aspects of treatments. ANOVA does not generate proof. Rather, it generates probability statements such as: There is a 95 percent probability that this positive outcome occurred because of a specific treatment.⁹ While other influences remain possible, sound research design and ANOVA allow researchers to determine whether a treatment is valid and reliable. Without scientific research we have no reliable means to assess whether a treatment is effective, whether the outcome is influenced by factors other than those that are supposedly part of the treatment. It is true that psychotherapy is an art, but it is an art conducted within the context of science. Without the science, it is junk psychology.

The questions posed in the raindance analogy can be used by lawyers and judges to determine whether the psychological treatments or concepts that are presented in expert testimony are supported by mainstream research. The answers to these basic research questions will help the court determine whether expert testimony in psychology is acceptable and helpful in the case at hand.

Psychotherapy takes place between two or more people in an office setting with the relative security of confidentiality and mutual trust. The patient usually clarifies the initial goals of his therapy, though these goals may be refined or altered as the therapy progresses. The psychologist or psychiatrist employs a variety of research-validated tools and strategies to help the patient reach his therapy goals. The therapy process does not always require the same search for truth that the legal process demands. A therapeutic outcome can often be achieved from the perceived realities of the patient. The psychotherapist needs to be well-informed about the research pertaining to the issues at hand, but also needs to respect and respond to the unique circumstances of each patient. The need for flexibility and the creative art of therapy make it appropriate for the therapist to experiment with creative interventions within the context of researched principles.

However, when psychology concepts, theories, intervention strategies, and patient behaviors become part of evidence in law, there is less room for unverified perception and theoretical flexibility and a greater need for accuracy, research credibility, and an expanded professional responsibility.

The following psychology concepts, treatments and theories are frequently involved in the expert testimony provided by psychologists and psychiatrists, and therefore lawyers should be familiar with them.

Memories generally, and recovered memories in particular, have generated considerable controversy in court proceedings throughout the 1990s. Many people, including some ill-informed health professionals, felt that memory could be compared to a stack of chronologically arranged photographs stored in the vault of the human mind. Subsequently, memory was considered by some to be synonymous with fact. Actually, memory is more like a stack of paintings that may or may not be stored chronologically, and they are altered each time the artist retrieves them. A consensus now exists among memory researchers that memory is a dynamic medium of experience shaped by expectancies, needs and beliefs. It is interwoven with emotion and is enriched by the exquisite human capacity for creative thought.¹⁰ People tend to remember by reconstructing events according to their life experiences and their emotion-laden conscious and unconscious needs at the time of recall.¹¹

The accuracy of a memory is not determined by the amount of detail present, the intensity of emotion that accompanies it, or by interventions with hypnosis or with barbiturates such as sodium amytal.¹² In a therapy setting it is not always necessary to corroborate subjective memory, but in a legal setting, corroboration of memory is obviously essential. Mental health professionals have no special ability to tell whether a memory is factual, the product of creative imagination, socially influenced, or a complete fabrication.¹³ Corroboration is the only reliable way to confirm or disconfirm a memory.

Human beings are creative, active information-processing beings for whom reality is unique. Everyone experiences life personally, and everyone, at one time or another, especially in highly charged emotional situations, feels certain that his perception of reality is absolute reality. Once a person has formed a conclusion about a particular perceived reality, he selectively attends to the factors that seemingly confirm this reality while ignoring factors that challenge it. In an adversarial situation, differing perceived realities are the norm and corroboration of these perceived realities is essential.

There are two sources of meaning attribution that warrant the attention of lawyers and judges. The first and by far the most common source consists of most members of the general public. Everyone periodically forgets, or chooses to ignore, the principle of unique realities, and projects his version of reality onto other people, often with considerable fervor. This form of mind-reading often creates considerable unresolved misunderstanding in personal relationships, but in a legal setting, meaning attribution should not go unchallenged. Various forms of bias, assumption and a misplaced sense of certitude can cause an expert witness or a lay person to attribute meaning to and make uncorroborated allegations about the behavior, thoughts and intent of another person. Nonverbal cues, for example, are sometimes used as proof of a person's intent. Mainstream research repeatedly shows, however, that nonverbal cues play an important role in communication, but it is not possible to attribute meaning to specific nonverbal cues with reliable accuracy.¹⁴

Sometimes meaning attribution arises from a form of professional arrogance. The history of psychology and psychiatry has been greatly influenced by Sigmund Freud and his psychoanalytic model of treatment. This treatment model, and the theory from which it was developed, rests on the assumption that a highly trained analyst is able to reliably interpret the real and often unconscious meaning of a patient's thoughts, dreams and behaviors. In this model, the authoritative analyst provides meaning for the patient. It is not surprising that this model has fallen out of favor with most psychologists and psychiatrists in North America, but the Freudian influence remains. The power and the prestige that are part of this authoritative "rainmaker" model can be very appealing to some health professionals even though mainstream research fails to validate many of the theoretical constructs of this model. A small minority of health professionals can sometimes become overconfident about their ability to know the realities of another person when they conduct psychological and risk assessments.¹⁵ It is not a good idea to accept anything as fact simply because an expert says it's so. The ghost of Freud still causes many people in the general public to view psychiatrists and psychologists as people who "read" and generally analyze others. Popular television shows such as Frazier help to perpetuate this image of the expert who can accurately attribute meaning to the thoughts, actions and intent of others. Uncorroborated meaning attribution has no place in our courts.

The recovered memory and false memory controversies of the 1990s have contributed to the development of a negative perception of hypnosis among many lawyers and judges. This perception is understandable but unfortunate, since it is based on the misuse of hypnosis by a small segment of health professionals. This group consists of a number of minimally qualified practitioners and a few qualified practitioners who seem to have forgotten their training in research design and outcomes. These health professionals have displayed an astonishing lack of awareness about the nature of memory and about suggestibility. They held that hypnosis, employed by the therapist, can uncover the truth that is "recorded" in the "unconscious" mind of the patient.¹⁶,¹⁷,¹⁸ Yet they flourished until recent court challenges initiated greater caution among some of them. Their view of memory is simply not supported by mainstream literature.

One of the consequences of blatant and irresponsible misuse of hypnosis has been an over-emphasis on hypnosis as a cause of memory distortion. The research shows that memory distortion is neither unique nor specific to hypnosis.¹⁹ Memory distortion can occur during all forms of recall. Suggestions and leading questions can contribute to the distortion of a person's memory, with or without the use of hypnosis.

Hypnosis is a term whose meaning has defied continuous efforts by health professionals to reach a consensus. Even the American Psychological Association definition is less than clear. Therefore, when expert testimony is provided on the role of hypnosis in a court case, lawyers and judges should ask the expert to clarify what he or she means when using the term hypnosis. Without clarification, the door is open to confusion, meaning attribution, and judgment errors. When the expert has provided a working definition of hypnosis, lawyers are then able to ask the questions from the rainmaker analogy to determine whether the hypnosis component of the expert's testimony is valid.

Until relatively recently, many members of the general public and a surprising number of health professionals have held the view that hypnosis is a special and altered state of consciousness induced in the patient by the "hypnotist." In this state, the person "hypnotized" is given suggestions by the "hypnotist." Theoretically these suggestions somehow bypass the recipient's conscious mind and are accepted by his or her unconscious mind.

The implication of this view is that hypnosis is a form of treatment administered by a health practitioner to treat a variety of physical and psychological issues. From this perspective, the burden of effort and responsibility is removed from the patient and placed with the "hypnotherapist." When the raindance analogy is applied to this model of hypnosis, it becomes evident that mainstream research simply does not validate this concept of "suggestion-given-suggestion-received." If mainstream research supported this simplistic model, hypnosis would be hailed as the primary treatment for most psychological problems.

In an effort to minimize the misunderstandings arising from the use of one term to represent many phenomenon, the APA has defined hypnosis as "a procedure during which a health professional or researcher suggests that a client, patient, or subject experience changes in sensations, perceptions, thought or behavior. The hypnotic context is generally established by an induction procedure."²⁰ There is no claim that the suggestions given are reliably accepted and acted upon. There is no attempt to define "hypnotic context," nor does the APA comment on whether this context is a necessary part of the procedure or a predictor of therapy outcome.

The APA does go on to say, however, that "hypnosis is not a type of therapy." Instead, it is a procedure that can be used to facilitate therapy, but clinical hypnosis should be used exclusively by properly trained and credentialed health care professionals who have, in addition to their mainstream training, received training and supervision in the clinical use of hypnosis. Those providing the training and supervision should know and respect the mainstream research and present hypnosis as a valuable adjunct to psychotherapy.

It is important to question the scientific validity of hypnosis in its various forms, but there is no reason to discredit hypnosis completely because it is used irresponsibly by some health professionals. Considerable research has been published to confirm a valuable adjunctive role for hypnosis in a number of psychotherapy situations. A meta-analysis of 18 studies showed that the addition of hypnosis strategies to the mainstream cognitive-behavioral treatment programs significantly improved the outcomes for 70 percent of patients.²¹ Additional research supports the use of hypnosis as an adjunct to mainstream treatments for anxiety and pain control.²² It is also used extensively to facilitate rehearsal imagery in sports and the creative arts.

Therapies from the Fringe and Beyond

Each decade seems to generate a new batch of syndromes and fringe therapies in North America. These new cures are usually packaged in pseudo-scientific terminology and wellness metaphors such as "reprogramming the unconscious," "healing traditions," "energy therapies," "past lives therapy" and "flushing one's emotional toxins." The therapies usually promise immediate and miraculous results for even the most challenging of human situations. The syndromes consist of a wide-ranging array of symptoms that are presented as validation of an unverified trauma. A familiar example from the 1990s: "If you have some or all of the following symptoms, you were undoubtedly sexually abused." The list of symptoms offered is astonishingly wide-ranging.

As Canadian Supreme Court Justice John "Jack" Major stated recently,²³ expert testimony can be "dressed up" in such impressive scientific terms that jurors can find it to be infallible. Fortunately, lawyers and judges can use the ANOVA questions from the raindance analogy to challenge any testimony that appears to be based upon a fringe therapy or a symptom-laden syndrome.

Health professionals who continue to embrace unsubstantiated therapies and syndromes may find that instead of offering expert testimony, they are in court defending the validity of their particular form of raindance. For example, in April 2000, a 10-year-old girl suffocated to death in the midst of a "rebirthing" activity conducted by three therapists in Colorado. The three therapists have been charged with child abuse causing death.²⁴

Lawyers and judges need to know what questions to ask when they have reason to doubt the validity of an expert's testimony in psychology. The majority of health professionals who give expert testimony are clinically experienced and aware of the research in their field of expertise. They understand that their testimony can have considerable impact on the lives of the people involved in the court proceedings, and they are therefore very conscientious about the professional credibility of their testimony.

However, a few health professionals seem to have forgotten their training in research design and some seem to have little regard for efficacy-supported treatments. Therefore, lawyers and judges need a conceptual awareness of research design and ANOVA, so they can ask the basic questions that will quickly determine whether they are listening to impressive sounding pseudo-psychology or to valid efficacy-based psychology. The raindance analogy and the questions arising from it illustrate the fundamentals of psychology research and offer a simple but effective framework for lawyers and judges to call upon when questioning the validity of an expert's testimony.

NOTES

1 J. Arboleda-Florez & C. J. Deynaka, Forensic Psychiatric Evidence (Toronto: Butterworths, 1999), at 1-23.

2 M.R. Somerfield & R.R. McCrae, "Stress and Coping Research: Methodological Challenges, Theoretical Advances and Clinical Applications" (2000) 55, 6 American Psychologist, at 620-624.

3 G.J. Cochrane, "Lasting Weight Loss: Attacking the Underlying Problems" (1999) 16, 7 Canadian Journal of Diagnosis, at 76-92.

4 Arboleda-Florez & Deynaka, at 12-17.

5 G.A. Ferguson, Statistical Analysis in Psychology & Education (New York: McGraw Hill, 1986), at 223-224.

6 Ibid., at 211-215.

7 E.B. Foa, T.M. Keane & M.J. Friedman, Effective Treatments for PTSD (New York: The Guilford Press, 2000), at 143-152.

8 D.L. Chambless & S.D. Hollon, "Defining Empirically Supported Therapies" (1998) 66 Journal of Consulting and Clinical Psychology, at 7-18.

9 Ferguson, at 72-74.

10 S.J. Lynn & K.M. McConkey, Truth in Memory (New York: The Guilford Press, 1998), at ix.

11 D.L. Scacter, "The Seven Sins of Memory; Insights from Psychology and Cognitive Neuroscience" (1999) 54, 3 American Psychologist, at 182-203.

12 Lynn & McConkey, at 10-11.

13 Lynn & McConkey, at 414.

14 P. Ekman & M. O'Sullivan, "Who Can Catch a Liar?" (1991) 46, 9 American Psychologist, at 913-920.

15 Arboleda-Florenz & Deynaka, at 167-168.

16 Ibid., at 156-160.

17 D. Brown, A. Scheflin & C. Hammond, Memory, Trauma Treatment and the Law

(New York: Norton, 1998).

18 E.B. Foa, T.M. Keane & M.J. Friedman, at 262-264.

19 Ibid., at 247-250.

20 I. Kirsch, G. Montgomery & G. Sapirstein, "Hypnosis as an Adjunct to Cognitive-Behavioral Psychotherapy; a Meta-analysis" (1995) 163 Journal of Consulting and Clinical Psychology, at 214-220.

21 A.M. Nicholi, Ed., The Harvard Guide to Psychiatry (Cambridge: Harvard University Press, 1999), at 555-562.

22 Ibid., at 554-562.

23 K. Makin, "Expert Testimony Called Threat to Jury Role," The Globe and Mail, October 6, 2000.

24 K. Nicholson, "Charges Dropped Against Mother," The Denver Post, November 14, 2000.

Back to table of contents >>