Conflict Resolution|Negotiation|Teaching

Psychological Barriers to Accurate Risk Assessment

A recent article has been making the rounds of ADR professionals. The current issue of the American Psychological Association’s publication Psychology, Public Policy and Law (Vol. 16, No. 2, at 133-57) features a report of a study conducted by a group of scholars from Australia, Sweden and the United States. The group canvassed 481 American attorneys – in civil and criminal cases, both plaintiffs/prosecutors and defense – and found that lawyers are prone to overconfidence. That is, they predict outcomes of their cases that are not only erroneous, but generally too optimistic.

I’m wondering why this is news. I think that we mediators have known this all along; in fact, that’s why we’re hired.

Participants in the study each had a case expected to go to trial within 6 to 12 months. They were asked to designate an outcome that “would be a win” and then to state the probability of achieving that outcome or a better one.

After the case resolved, whether by verdict or by settlement, the participants were contacted again to determine the case outcome or resolution. Approximately 59% of the cases were settled; 31% were tried; and 10% were dismissed.

A mean of 64% and a median of 70% of the participants expressed confidence estimates exceeding 50%. Female lawyers tended to be overconfident only when their predicted success was high, while men tended to be overconfident whether their prediction was moderate or high probability. Both lawyers of less experience and of greater experience exhibited the same levels of overconfidence, and there was no correlation with respect to whether trial date was a few months away or imminent.

Yet only 50% of the civil lawyers achieved their goals, though their mean confidence estimate had been 64%.

This overconfidence persisted even when an effort was made to manipulate the subjects towards more realistic predictions. Participant attorneys were asked to generate arguments counter to their own predictions, or to give reasons for their predictions, in what the authors termed a “debiasing technique.” The participants’ overconfidence did not decrease.

The authors conclude that, irrespective of their trial experience, “[l]awyers frequently made substantial judgmental errors, showing a proclivity to overoptimism…. Lawyers choose a desirable outcome, the anchor, and thereafter make insufficient adjustments for uncertainty even when asked to generate reasons against their initial goal.” Lawyers are poorer predictors than, say, weathermen because (a) lawyers can influence the outcome of their cases and (b) lawyers have an interest in influencing the outcome of their cases. Meteorologists, on the other hand, neither can make it rain nor make money when it does, and thus are less prone to misinterpret the data in favor of their self-serving preconceptions.

I have no reason to doubt these outcomes. I just don’t think it ought to be a surprise. And whatever the authors say about weather forecasters, overconfidence is a phenomenon shared by businesspeople, fishermen, grooms, poker players and well-diggers, not just lawyers and their clients. And that’s why God made mediators.

god and creationWhen I was first trained as a mediator, by Michael Lewis and Linda Singer, I was taught that people – whether lawyers or their clients – suffer from cognitive dissonance and tend to ignore information that does not comport with their understanding of the facts. Dwight Golann and I had a hard time training Chinese judges in American-style mediation in Beijing in 2005, but Dwight had no difficulty at all in discussing cognitive dissonance, which the trainees found absorbing.

In his recent book, Mediating Legal Disputes, Dwight discusses the “endowment effect.” He reports on an experiment in which people were assigned to negotiate the sale (or purchase) of a coffee mug. Each participant was told to make a preliminary confidential estimate of the intrinsic value of the mug, and a control group of observers was asked to make the same estimate. Those assigned to sell the mug estimated its value at $7.12; those assigned to buy it estimated its value at $2.87; and the observers valued the mug at $3.12.

Negotiation literature is chock full of illustrations of psychological obstacles to realistic appraisal. A compelling example of cognitive distortion is related in Dwight’s book as follows:

Students at Harvard Law School are preparing to negotiate the settlement of a personal injury case. Before they begin, the students are asked to make a private prediction of their chances of winning based on their private instructions. What the students don’t know is that there is nothing confidential about the instructions: both sides have received exactly the same data, with different labels. Because both sides have the same information, they should come out with the same answers – but this is not what occurs.

In fact, hundreds of law and business students told to negotiate for the plaintiff assess her chances of winning as being nearly 20 percent higher than students who are assigned to the defense. The two sides’ predictions total nearly 120 percent.

Asked to estimate what damages a jury will award if the plaintiff does win, there is a similar disparity: plaintiff bargainers estimate her damages at an average of $264,000, while defense negotiators looking at the same data estimate a verdict of only $188,000.

There is no sense in bemoaning this data, or in adding it to the tools of the lawyer-bashers. We are who we are; part of who we are is that our observations mature into convictions, and we defend our convictions against perceived attack. Most students of negotiation readily concede that unfacilitated bargaining is necessarily inefficient, because neither negotiator will be completely candid with the other. A facilitator, on the other hand, can become a repository for the data that is not known by the adversary, and thus gains a perception of an economically efficient outcome that is denied the participants because of their own self-imposed constraints.

Not just lawyers make assessments based on imperfect knowledge. Not just lawyers’ assessments are hindered by cognitive obstacles. And not just lawyers benefit when their assessments, and the assumptions underlying them, are subjected to rigorous reality testing.

  1. What some people might look at as a cognitive flaw, I would say is also essential to good performance as a trial lawyer. Unless you can project a strong belief in your case yourself, you are going to have a hard time persuading a judge or jury of your position. So while you have to be aware of the flaws in your position, and the strengths of the other side’s position, and while you also want to give your client accurate information about their chances, you also almost have to be a bit over-confident in order to do an effective job as an advocate.

    So we shouldn’t criticize trial lawyers for having a mental bias that is actually necessary for them to do a good job. Rather, we have to design the system so that clients can get more objective information about their cases. Isn’t that where mediators step into the picture?

  2. Adding to Joe’s comment, I think that clients want trial attorneys to be “on their side” and being on their side means having confidence in their case. I was a trial attorney for 9 years before becoming a professional mediator. I left trial work when a colleague at the firm where I worked asked me, “whose side are you on?” when I was talking to her about the downsides of litigation for her client. I couldn’t even explain the obvious risks to another attorney without looking like a traitor.

    I am well aware of this issue when I mediate. I get to be the bad guy and tell parties that their chances are lower than they think. Their lawyer, from my personal expereince, does not have that opportunity.

Leave a Reply

Your email address will not be published.