Posts Tagged ‘eyewitness testimony’

A Prosecutor Defends Eyewitness Identification

Wednesday, November 11th, 2015

It’s fairly well-established that eyewitness identification sucks, as a rule. There have been tons of scientific studies going back decades — and more are conducted all the time — on the reliability of eyewitness testimony. The studies generally conclude that we’re really bad at noticing things, remembering them accurately, and identifying faces we aren’t already familiar with. Oh, and we really really suck at it when the face is of someone of another race. Meanwhile, the field of neuroscience is reporting breakthroughs literally every month in our understanding of how the brain creates memories, stores memories, distorts memories, processes sensory perceptions, processes faces, recognizes faces, et cetera et cetera et cetera. On top of all that, there have been studies demonstrating how traditional law enforcement methods can make all of this even worse, and what other methods would work better. And on top of all that is the incontrovertible data that eyewitness testimony has played a significant role in wrongful convictions that have been proven to be wrongful.

If you want to go read the scientific literature, please do. There’s a lot. If you want a quick-and-dirty version, I’ve been covering it in my comic for — holy hell, a year now? For those preferring a more, say, jurisprudential approach, Judge Alex Kozinski wrote an excellent summary of the situation in Criminal Law 2.0, his preface to this summer’s Georgetown Law Journal Annual Review of Criminal Procedure. Click on the link to read his views.

But in American jurisprudence, these are still minority views. The courts are slow to adopt change, and have been incredibly slow to adapt the law of eyewitness identifications to the scientific facts. It’s starting to happen. And I fully expect us to reach critical mass within a generation if not sooner. But the majority of courts aren’t there yet.

And law enforcement isn’t entirely on board, either. You’d think they’d want to be on board, though — after all, who wants to lock up an innocent person and leave the real bad guy free to do it again? But inertia, the investment of ego, and confirmation bias are much more powerful than you may suspect. Especially in law enforcement. You’ve seen the mind-numbingly bad arguments cops and prosecutors have come up with to insist someone’s guilty even after the DNA proves he isn’t — or even to fight tooth and nail against the DNA analysis that would prove it. There’s too much invested in having been right.

And there’s a lot of ego, inertia, and confirmation bias invested in the sense that eyewitness identifications — they way they’ve traditionally been done — are just fine and dandy.

Which brings us to the recent publication of “The Unreliable Case Against the Reliability of Eyewitness Identifications: A Response to Judge Alex Kozinski,” by Connecticut appellate prosecutor Laurie Feldman.

Feldman’s piece is an attempt to say Kozinski — and all the others who have problems with eyewitness identifications and testimony — have it all wrong. Things aren’t as bad as everyone says, and we shouldn’t jump to make unnecessary reforms. Don’t fix what ain’t broke.

Click on the link to read the whole thing yourself. I, for one, don’t find it terribly well reasoned, but your mileage may vary. From what I understand, her points are these:

(1) There’s no scientific proof that jurors put too much store in eyewitness testimony.

(2) There’s no good science saying eyewitnesses aren’t reliable.

(3) Scholarship here is agenda-driven, which distorts the results. Peer review only makes this worse.

(4) Just because someone was exonerated, that doesn’t mean he was innocent. So how does that mean the eyewitness who fingered him was wrong?

(5) So what if 72% of DNA-based exonerations were cases involving false I.D.? How many cases, where DNA proved guilt, involved an eyewitness’s correct I.D.? What if it’s the same number?

(6) We’ve only exonerated a teeny tiny subfraction of a fraction of people convicted on eyewitness testimony. That sounds like they’re reliable, not unreliable.

(7) Courts are allowing experts to testify about all this, and they shouldn’t. This exposes the jury to political agendas, and makes the scientists advocates instead of objective scientists. Courts are moving too fast to adopt reforms like this.

(8) It’s unwise to jump on the latest social-science fad to fix what ain’t broke. There’s not a single study saying that double-blind experiments are better than ones where the examiner knows who the suspect is. And simultaneous lineups simply get better results than sequential ones.

All I can say is:

(1) If her footnotes are to be relied upon, she clearly has a lot of reading to do. I can suggest a bibliography if she wants one. This recent National Academy of Sciences report is a good start. Heck, she’s welcome to take every damn book off my bookshelf right now because I need the space for all my resources for the comic’s upcoming forays into Constitutional Law.

(2) Ditto.

(3) Perhaps she should stop reading scholarly writings and “social science” sources, and focus on the objective scientific studies out there. I encourage her to maintain a healthy skepticism of sociology and meta-analysis and studies with small sample size or weird methods — but the good news is there’s a whole lot of good stuff out there with good clean numbers and healthy p-values.

(4) DNA exonerations pretty clearly show he ain’t the guy. The eyewitness who absolutely positively no doubt about it said that’s they guy? Wrong.

(5) Not the issue. The issue isn’t how many guilty people are properly convicted. The issue is, when an innocent person does get wrongfully convicted, what’s causing that, and how can we prevent it from happening again? When 72% of your wrongful convictions that you know to be wrongful because the DNA says so were based on a false identification? That’s a very strong correlation. Combined with the fact (sorry, it’s true) that juries do put a whole heck of a lot of weight on that false I.D., and it’s safe to say we’ve got causation as well. (Whereas 100% of wrongful convictions may have taken place in a courtroom with a judge, a very high correlation indeed, but there’s zero evidence that the presence of a judge played any effect on the jury’s verdict, so we can disregard that one.)

(6) Intellectually dishonest. You’re saying “you guys haven’t disproven all these other cases yet, so let’s presume those IDs were good.” Ignoring the fact that only a small teeny tiny microscopic percentage of cases where the defendant insists upon his innocence are ever taken up by folks like the Innocence Project. You’re also comparing the number of hits in the sample size to the overall population being studied. The fact is, an obscenely high number of hits in this statistically significant sample probably translates to an obscenely high number of hits overall.

(7) Fine. You’re not allowed to use DNA experts, either… Seriously? Draw up a timeline of courts adopting real meaningful reforms here and show me how it’s moving too fast.

(8) Apparently the latest fad = decades of research, and basic Science 101 recommendations for how to conduct an experiment that have been understood for ages. None of this is rocket science. None of this is remotely eyebrow raising to anyone with a genuine science background. If you really don’t think double-blind studies are proven to be effective, that they eliminate intentional and unintentional suggestion by the examiner, then you really really shouldn’t have skipped that day in 8th-grade Science class. As for simultaneous vs. sequential, you are ALMOST RIGHT!! Because sequential arrays DO result in fewer correct identifications, along with fewer false ones. Sequential arrays only solve the problem of relative-judgment witnesses who compare the faces in the lineup to each other. They’re best used with children, lower-IQ people, and all the people whose brains really work this way only they don’t know it and neither do you. All of whom, it so happens, are the ones making the MAJORITY of false identifications. Losing a few correct IDs (because during sequential procedures we all tend to raise our threshold for certainty, worrying that a better match might be coming up) is a small price to pay for ensuring so many innocents don’t have their lives and liberty and futures taken away by mistake. Isn’t it? And simultaneous lineups are only as effective once you’ve adopted the suggested reforms, including some you didn’t discuss.

This all sounds like another one of those prosecutors thwarting justice in the name of inertia.

My final recommendation to her would to be to allow herself to be receptive to the idea that innocent people do sometimes get convicted, that bad identifications do sometimes cause this, and that there are solutions that can prevent such injustices, can ensure that the cops don’t put the wrong guy in jail, can ensure that the real perp doesn’t remain free to do it again, don’t cost hardly anything, and are easy to institute. And if that’s the case, then why the heck are you so opposed to it? Do you want those preventable injustices to occur?

Because that’s kinda how you came off.

Inexpert Testimony

Monday, April 27th, 2015

The purpose of a trial is not to discover the truth. Sorry. Whether civil or criminal, bench or jury, the purpose of a trial is to decide on an “official version” of the facts. The purpose of the justice system is to make an enforceable, hopefully final, decision about a dispute. The system does this by applying the law to the facts, and determining what the appropriate outcome is. The system already knows what the law is, presumably. But it can’t apply that law — it can’t do anything — until it has a set of facts to work with. We’d like the official facts to be as close to the truth as possible, of course, but one way or another we need to decide what they are.

That’s what the jury is for. That’s all the jury is for, most of the time: to be the “finder of fact.” Obviously, there are competing versions of the facts to choose from, or else there wouldn’t be a trial. The jury has to decide which facts the justice system will get to use. (And as Scott Greenfield pointed out this morning, once that official version of the facts has been determined, the system is extremely loath to revisit them. These are the facts we’ll rely on for damages, for sentencing, for appeals, forever.)

The jury’s job is important. It is sacred. The idea that twelve honest members of the community can assess the evidence and figure out what was proven and what was not proven is integral to our concept of justice. And in a jury trial it’s important that only the jury gets to perform its sacred task. We don’t let anyone else decide the facts for them. That would mean replacing the jury of twelve with a jury of one. We tightly control who gets to testify, what questions they can be asked, and what they’re allowed to say. We limit the evidence only to relevant testimony, and try to exclude categories of evidence that are too unreliable to use — especially evidence that cannot be challenged.

And we certainly don’t let witnesses or lawyers vouch for the truthfulness of their testimony. “You have to believe this because, in my opinion, it’s true” is not something you can say to a juror.

Unless you’re an expert witness, that is. Then you get to not only opine on what evidence means, on facts the jury needs to decide, but also on the reliability of your opinion. Ideally, an expert is an objective witness with no stake in the proceedings, who has knowledge of a subject that is just too complex or arcane to expect of jurors. Whatever they’re testifying about, they’re needed because it’s not common knowledge. So the expert gets to summarize the arcane subject, draw factual conclusions for the jury, and also give his opinion about how reliable he is — how confident he is in his conclusions.

When he gives his expert opinion, the expert witness does the jury’s job for them. The jury needn’t assess his reliability — the judge called him an expert, and the expert himself said the basis for his opinion was reliable. The jury doesn’t need to assess his summary or his conclusions about the evidence — if they could do that, they wouldn’t have needed an expert in the first place. And besides, the expert gave them his expert opinion of how reliable that conclusion is. The expert witness can easily become a jury of one.

So we are really really careful about who we allow to testify as an expert witness, and strictly limit what they can testify about.



Of course there are exceptions. And of course they’re mainly to be found in criminal trials, where the stakes are highest, and the jury’s role is most important.

One exception you’ve probably heard about before. [Heck, I’ve probably griped about it here a couple of times, only just as I can’t be bothered to edit these posts before I post them, I can’t be arsed right now to look up whether I’ve written on it already.] This is the exception for expert police testimony. Instead of establishing valid academic and professional credentials to ensure that this expert knows what the heck he’s talking about, and instead of having him specify the resources and data on which he relies (and thus give the other side a chance to challenge the validity of those sources), we pretty much let the police witness provide his own opinion about whether he’s an expert, and then we call him an expert, and then we let him tell the jury what the evidence means.

Cop: “I was trained by other police officers about how drug deals work. I’ve participated in lots of arrests that involved drugs. In my opinion, I am super-familiar with how drug deals work.”

Judge: “Okay, jury, this guy’s an expert.”

Cop: “In my expert opinion, those apparently innocent bits of evidence really mean the defendant sold that other guy some drugs.”

Jury: “Well, that’s that. When’s lunch?”

It’s self-serving testimony by the government, deciding for the jury the ultimate issue of the case. It’s not from a disinterested witness, but usually from the same officer who made the arrest. He’s telling the jury that, in his opinion, he was right. And the judge is telling the jury he’s giving this opinion as an expert. The witness isn’t giving any details of what he’s basing this expert opinion on, and so its reliability cannot be challenged. All that’s happening is he’s getting to vouch for his own expertise, and the government is getting to vouch for the reliability of its evidence.

But this isn’t what I wanted to complain about today. At least the officer was first screened and offered as an expert. Eyewitnesses, on the other hand…


The eyewitness gets to give expert opinion testimony without even being admitted as an expert.

Eyewitness testimony is notoriously inaccurate. On average, it’s no better than a coin toss. More than 3/4 of the death-row exonerations to date have been from convictions based on eyewitness testimony. Humans just don’t see everything as accurately as we think we do, our memories are malleable as hell, and we’re really bad at pointing out the culprit in a lineup. But boy howdy are we confident! We think our memories really happened, even when they’re demonstrably false. We think we’re right. How can we not? But we’re wrong an awful lot.

Confidence has zero to do with the reliability of an eyewitness. Almost every eyewitness is confident that they’re remembering things accurately. Even when they’re wrong.

And yet, when an eyewitness says they’re confident that they’re right, it has a huge effect on juries. When a witness vouches for her own testimony, studies show that jurors tend to believe her. The effect is powerful — it is almost impossible to overstate it. In fact, even when a witness has contradicted herself on the stand, and has been shown to be clearly unreliable, if she tells the jury that she is very confident that she’s right, the jurors will say they found her credible.

Courts also put a lot of stock in an eyewitness’s opinion of her own accuracy. The more confident the witness feels, the more likely she’ll be allowed to testify. The Supreme Court itself has gone out of its way to say that eyewitness confidence is a factor that should be considered when assessing the reliability of her testimony.

In other words, if an eyewitness says “I’m sure I’m right,” the jury’s not only going to be more likely to believe her, but they’re supposed to believe it more.

How stupid is that?

This is opinion testimony. The eyewitness is giving an opinion on an issue of fact — the reliability of her testimony. This is a decision the jury needs to make. She doesn’t have an unusual background that lets her assess this any better than the ordinary juror. On the contrary, because it’s her own memory and perceptions, she’s the one person least qualified to give a reliable assessment of its accuracy. But she’s the one who gets to tell the jury how accurate she is.

Not only that, but in many states the defense is not allowed to even present their own expert testimony to rebut her opinion. No expert opinion that this eyewitness got it wrong. (Two main reasons: one, eyewitness unreliability is considered such common knowledge –ha! — that expert testimony is unnecessary; and two, general testimony about how people in general get it wrong isn’t evidence that this person got it wrong.) All that can be done is cross-examine to find inconsistencies and reasons to argue later that the witness was wrong. But that’s not very helpful, because again, when an eyewitness testifies that she has high confidence in her accuracy, those inconsistencies and other indicia of unreliability get ignored, and the jury tends to believe her regardless.

It’s not just a confrontation issue, but an evidentiary one as well. Witness confidence just isn’t a reliable indicator of witness accuracy. We’re wrong far too much of the time. In general the rules of evidence exclude categories of evidence that have a substantial risk of being unreliable. Everywhere else, we exclude evidence that cannot be challenged. If we were consistent, then, our rules of evidence would preclude witness assessments of their own accuracy rather than encouraging them.

It is the jury’s job to assess whether this witness got things right, and nobody else’s. Yet we go out of our way to take that decision away from them, and let the witness herself decide whether her testimony is correct. She’s giving an expert opinion on the reliability of her evidence. And not only is she not an expert, she is in fact the one person least qualified to assess the reliability of her memory.

But we let her say it. She gets to give her inexpert testimony, and do the jury’s job for them. And that snapping sound we hear right afterwards? That’s the jury’s brains turning off.

Correct, but Wrong: SCOTUS on Unreliable Eyewitness Identification

Thursday, January 12th, 2012

In this Information Age, it is hard to grasp sometimes that everybody does not know everything. And yet it is so. It is common knowledge, for example, that dinosaur fossils are the bones of creatures that lived scores of millions of years ago, that terrorist hijackers flew planes into the World Trade Center and the Pentagon, and that eyewitness identification testimony is statistically as reliable as a ’78 Chevy. And yet there are tons of people who sincerely believe that fossils are just a few thousand years old, that the U.S. government conducted 9-11, and that an eyewitness I.D. is the be-all-and-end-all of Truth.

Actually, it’s not fair to lump the I.D. believers in with 9-11 conspiracy theorists, Genesis literalists, truthers and the like. The others are sort of fringe-y. But if you put 12 ordinary citizens in a jury box, of good intelligence and sound common sense, and the victim points dead at the defendant and says “there is no doubt in my mind, THAT is the man who raped me…” you can almost hear all twelve minds slamming shut. They’ve heard all they need to hear. So far as they’re concerned, this case is over.

This despite the fact that study after study after study reinforces the fact that eyewitness testimony sucks.

And innocent people go to jail — or worse — because of it.

So you can imagine how keen the legal world was to get the Supreme Court’s decision in Perry v. New Hampshire, which came down yesterday. Perry, identified by an eyewitness as someone she’d seen breaking into cars, argued that Due Process required a judicial hearing on the reliability of that testimony before it could be admitted at trial.

Which was the exact wrong thing to argue.

Due Process requires that the government makes sure that it does not do things that make its identification procedures unreliable. It does not require that a judge do the jury’s job. Particularly when that job — weighing the reliability of a given bit of testimony — is incredibly fact-specific.

And especially given all the evidence of all the various factors that go into making eyewitness testimony unreliable — racial differences, time lapse, focus of attention, lighting, familiarity, stress, presence of a weapon, etc. — what judge in his right mind is going to want to be the one deciding whether this particular eyewitness’s memory is good enough?

So it’s hardly any surprise that the Supreme Court balked at Perry’s Due Process argument. By a vote of 8-1 (former prosecutor Sotomayor as the lone dissenter, none better to know the power of the EW ID) the Supremes held that, unless law enforcement is alleged to have gotten the I.D. under unnecessarily suggestive circumstances, there’s no Due Process issue and certainly no reason for a pre-trial hearing on reliability.


No, what Perry could have argued for is either (more…)