The Streetlight Effect

A campfire skit from my childhood has been on my mind recently. It's a simple one and you may have seen or read a version of it yourself:

A person is searching for something under a streetlight.
A pedestrian approaches and asks, "What are you doing?"
"Looking for my keys," the searcher replies.
"Well, where did you lose them?"
"Over there," the searcher indicates elsewhere.
"So why are you looking here?"
"Because the light is better."

A Joke and a Fable

In the version that I saw, the situation is drawn out with multiple people getting involved in the search before someone finally asks where the keys were lost.

Like many campfire skits, this one has been retold in various formats for centuries ( see the excellent article on Quote Investigator ). It always struck me as a lighthearted, cautionary tale. In most modern retellings the person searching is drunk, as if to emphasize the error in logic.

As a child I thought the takeaway was simply, "Don't be the stupid person". However, as I've grown up I've realized it's not so simple. It says something about people, and so we keep retelling it. People keep making this mistake, and they're not drunk or stupid.

The popularity of this fable seems to have taken hold in America in the 20's when several newspapers ran versions of the story as a joke police blotter. Sometime between the mid 40's and 60's the story began to be referred to as "the drunkard's search" or somewhat more professionally, "the streetlight effect". (Wikipedia)

( I think that the reference to 'streetlight' unfortunately plays into the fallacy. Generally a figurative 'light' exposes the truth, while in this situation the light distracts from the truth. )

Is the Moral About Bias?

Usually this tale is used to invoke something about cognitive bias in a process. Sometimes people talk about the Observer-Expectancy Effect, but I'm not sure this is exactly it. Anyhow, I'm not a researcher or scientist so I'm just going to avoid that whole subject.

The point is that our own biases (preferring to search in a well lit area), can dramatically interfere in our ability to solve a well defined problem.

But is this the only lesson?

I think there is also a warning about lending help to a process you don't fully understand. I've let other people recruit me into their "drunkard's searches", pulled in by my own good intentions and belief that I could be helpful. It's not a bad motivation, but there is no reason to lend a hand to a pointless effort.

The 'smart' person in the skit is the one that insists on building their own understanding of the problem from it's beginning. By asking one simple question this individual is able to identify the error in process, and presumably increase the probability that the problem will be solved.

Conclusion

I see this being very similar to the expression; "If the only tool you have is a hammer, everything looks like a nail". It's not just that it's easier to look where the light is better, but that's what we're accustomed to doing.

My advice as bullet points:
-Restate the Problem with as much detail as possible
-Don't hesitate to 'start from the beginning', several times if necessary.
-Remember to pursue the most logical solutions to a problem rather than the most familiar solutions