DRAFT

Picture yourself trekking through a blistering desert, your throat parched and your mind delirious, only to be teased by the illusion of an oasis shimmering on the horizon – a mirage in the desert. These illusions are just like the deceptive allure of statistical anomalies. Drinking from a fountain that doesn’t exist is not just akin to mistaking statistical noise for a meaningful pattern, but can also be a consequence of incomplete information.

In statistical analysis, surface-level findings are often misleading. Let us look at an example to illustrate this point further. Specifically, the following tables summarize data on the differential imposition of death sentences depending on race.

Upon first examination, it appears that there’s a significant discrepancy. Among white defendants, 12% received the death penalty compared to 10% of black defendants. At first glance, one could easily come to the conclusion that white individuals are treated more harshly by the justice system.

However, the plot thickens when we introduce the victim’s race into the equation, as depicted in the next contingency table.

Suddenly, the opposite picture emerges. When we consider both the defendant’s and the victim’s race, the disparity vanishes. In fact, when the defendant is white and the victim is black, there’s a stark contrast – none of these cases resulted in the death penalty. Moreover, cases in which the victim is white and the defendant is black resulted in the death penalty more often than cases in which both victim and defendant were white. Now it is clear that the justice system is not biased against white people, but in fact biased against black people.

Lurking Variables: The Silent Saboteurs of Data Analysis

Including just one additional variable, the victim’s skin color, completely turned the conclusion on its head. This is what statisticians call a lurking variable. They are big hidden influencers that are often overlooked or sometimes intentionally omitted. Somebody else takes a closer look at the same data: Lo and behold, they come to a different conclusion.

TODO: Another example (maybe hospitalization)

Such oversights are so prevalent that statisticians commonly emphasize: “Correlation and association do not imply causation!” It is also one of the roots for the common saying: “The only statistics you can trust are those you falsified yourself.” TODO: Explain that in terms of the previous example (or is it obvious?)

While the danger of lurking information is well-known in the field of statistics, this concept reaches far beyond the scope people usually confine it in. It is a phenomenon that you can spot in many aspects of everyday life, such as reading the newspaper. News stories never tell the whole story. They give a limited, summarized view. This is akin to summarizing raw data into a small number of variables. Some information loss is inevitable. But as we just saw, summarizing and thus omitting parts of the data can completely invert the conclusions we draw from it.

This applies to all kinds of information, including textual information, not just numeric data. Telling all intricate details and all perspectives of a news story is impossible. Therefore, you only get to hear a limited perspective, which means that some lurking information is almost certainly missing. Whether no one thought about including this vital information or it was omitted intentionally to push a narrative does not matter in the end. The point is, many news articles lack critical lurking information that might completely change the conclusions that readers draw from it was it included. TODO: This is sort of duplicated below… Consolidate?

TODO: Think of an example (it doesn’t have to be something that really happened, but it needs to be relatable.)

Consider this: “The only news you can trust are those you falsified yourself.” It is a harsh reality check on the information we consume, emphasizing the need for a critical eye even in supposedly trustworthy sources.

TODO: Link to the next topic

The Single-Story Syndrome: A Narrative Lurker

Apply that to the news, and you have got a lurking variable of epic proportions. News outlets often present a singular narrative, but it’s crucial to remember that other perspectives are lingering in the background. It is easy to read a newspaper article and instantly arrive at a conclusion. Upon further investigation, however, the issue at hand often turns out to be far more complicated. This more multifaceted understanding of what is perceived as the truth may be closer to the complete opposite of the original conclusion. TODO: This is sort of duplicated above. Consolidate?

These vastly different conclusions corresponding to different levels of informedness may seem contradictory, and that is because they are. Some are wrong and others are possibly wrong, in line with the philosophy of critical rationalism. Consider the death penalty example mentioned earlier, where the first conclusions built on a lack of information are completely provably false and for the rest of the conclusions we don’t know for sure. (TODO: Example with police criminality, where each case has to be treated separately to really analyze the data.)

TODO what to do with this? Take any controversial issue – say, the benefits of coffee consumption. One news report might highlight its positive effects on mental alertness, while another emphasizes potential health risks. The lurking information here is the diversity of opinions, studies, and experiences that shape the larger picture.

The Never-Ending Pursuit of Attention

Even though newspapers were once considered by many to be a reliable source of information, they have always been plagued by a design flaw that has been exacerbated over the past couple of decades. They are fueled by the pursuit of the world’s most valuable currency: Attention. Headlines and articles are specifically crafted to maximize reader attention and engagement, which conflicts with goals to deliver accurate and neutral information. Organizations seeking profit are particularly vulnerable to this trend, as their financial well-being fully depends on being noticed by readers and keeping them hooked.

Even organizations striving for neutrality can inadvertently overlook lurking information, as exemplified earlier. Of course it is important to recognize that there are sometimes incentives to purposefully filter the information that is being presented, but oftentimes it happens involuntarily, as a result of mere oversight rather than ill intent. Good authors try to improve the diversity of information and give multiple perspectives, but they can never cover all bases.

TODO what to do with this? The inclusion of a lurking variable could prompt a reassessment of the causes and consequences presented in the news.

Spotting Lurking Information in the News

As an avid reader and thinker, how can one avoid falling prey to a lack of lurking information? Unfortunately, this is extremely difficult, maybe practically impossible. However, there are a few guidelines that can lead to a more refined view of the world.

Cross-Validation Of All Sources: Find as many independent sources as possible, ideally a combination of primary, secondary and tertiary. Also include those that are controversial. Diverse perspectives help unveil the lurking variables hiding in biased narratives.

Timeline Analysis: Break down events chronologically. Lurking information may skulk in the temporal gaps between reported events, that others have deemed not newsworthy. TODO: Is this reasonable? It sounds plausible to me but I cannot think of an example off the top of my head.

Critical Thinking As An Algorithm: Develop your critical thinking algorithm. Identify and question assumptions, detect and avoid cognitive biases, free yourself of prejudice and stereotypes, never trust facile generalizations, and always be on the lookout for lurking information. TODO: Link to some other articles (on cognitive biases for instance).

Looking deeper: Even when you use many different sources, it is not uncommon that all sources overlook some vital aspects of the problem. Even when the authors are not copying each other, they may be in a related state of mind and have similar prejudices that lead to matching ideas reflected in their works. Therefore, to really understand a topic, one has to look far deeper, observing raw data themselves and trying to think outside the box to see what others have overlooked. This is the most difficult and time-consuming, but also the most rewarding option to gain a more truthful perspective of the world.

This leads to the following conclusions:

The picture that media paints is always deceptive. Every author has personal biases. Every for-profit news organization wants to hook your attention and deliver the most up-to-date breaking news. Sometimes quality and neutrality of information has to be sacrificed for that.

Give people and organizations the benefit of the doubt when you hear bad things and approach them with a healthy dose of skepticism when you hear good things about them. Many of those who are portrayed as dumb assholes are in fact very reasonable and likeable people once you get to know them better, while others who are portrayed as intelligent might lack a lot of information and only pretend to be experts. TODO: Should I avoid informal swear words? This applies to everyone, from your friend to your biggest enemy. Whether you hate certain politicians, businesspeople, activists, whatever, apply this advice.

Realize that you always lack critical (lurking) information. The human brain is incredible bad at recognizing ignorance, so try to avoid that mistake. Don’t be ashamed to admit you don’t know enough about a matter to comment on it. It is a trait of intelligent people to realize the incompleteness of their knowledge.

Realize that other people probably lack critical (lurking) information too. It is easy to see why those who are “street-smart”, while often very admirable and intelligent, lack a lot of information. But even academics can easily end up being trapped in a bubble and have trouble thinking outside the box. TODO: Should I add an example? Anchoring on a theory is so common in all scientific disciplins, it’s not difficult to think of one. But it might be slightly off-topic for this article. Do not use this advice as an excuse however to think your superficial knowledge about a subject is superior to the hard-earned expertise of someone else. Recognizing other people’s ignorance is typically easier than recognizing your own ignorance.

Do not get emotional and do not let others bring emotions into arguments. Emotion is the avenue through which news manipulates people. Propaganda has always played with fear, … TODO: Maybe an example from Joseph Goebbels or something less extreme?

In the end, being aware of lurking information isn’t about being paranoid; it’s about embracing the complexity of the world around us. Acknowledging the limitations of our knowledge and being receptive to diverse perspectives is necessary to navigate the realm of partial or inaccurate information. Gaining a more informed and nuanced understanding of the world around us has always been and will continue to be the best path forward for society. TODO: I never know how to end one of these articles. Maybe look at similar material for inspiration? But I do not know about any similar material… Scientific papers are too formal in their conclusions, that would not fit the concept of this website.

Sources

• [1] Radelet, M. L. (1981). Racial characteristics and the imposition of the death penalty. American Sociological Review, Volume 46 (6), 918-927.
• TODO the Stanford “Simpson’s paradox” link
• TODO Sources for other potential examples