In March 1951, the US intelligence community issued NIE 29-51, “Probability of an Invasion of Yugoslavia in 1951.” Sherman Kent, who sat on the CIA board that approved it, quotes its key judgment in his 1964 essay “Words of Estimative Probability”:

Although it is impossible to determine which course the Kremlin is likely to adopt, we believe that the extent of Satellite military and propaganda preparations indicates that an attack on Yugoslavia in 1951 should be considered a serious possibility.

The question that exposed the problem came from the chairman of the State Department’s Policy Planning Staff. Kent’s account:

Suddenly he said, “By the way, what did you people mean by the expression ‘serious possibility’? What kind of odds did you have in mind?” I told him that my personal estimate was on the dark side, namely that the odds were around 65 to 35 in favor of an attack. He was somewhat jolted by this; he and his colleagues had read “serious possibility” to mean odds very considerably lower. Understandably troubled by this want of communication, I began asking my own colleagues on the Board of National Estimates what odds they had had in mind when they agreed to that wording. It was another jolt to find that each Board member had had somewhat different odds in mind and the low man was thinking of about 20 to 80, the high of 80 to 20. The rest ranged in between.

Kent writes that he and a fellow board member, Max Foster, were “shaken perhaps more by the realization that Board members who had worked over the estimate had failed to communicate with each other than by the Board’s failure to communicate with its audience.”

I run an exercise in my workshops where I ask people to pick a vague word that often shows up in presentations, something like “strong performance.” Then everyone types in the number they would associate with it, usually a percent growth, and we all hit enter at the same time. Almost every time, the group laughs, because the answers are so wildly different.

You know what you mean, so you assume the reader does

Thomas Gilovich, Kenneth Savitsky, and Victoria Medvec studied this problem in a 1998 paper in the Journal of Personality and Social Psychology. The abstract describes the illusion of transparency as “a tendency for people to overestimate the extent to which others can discern their internal states,” and adds: “People often mistakenly believe that their internal states ‘leak out’ more than they really do.”

Their experiments tested lying and feelings like disgust. People telling a lie, for instance, thought it was easier to spot than it was.

A simpler demonstration, with information instead of feelings, is a 1990 Stanford study by Elizabeth Newton that Chip and Dan Heath describe in “The Curse of Knowledge,” their 2006 Harvard Business Review article: people tapping out songs on a table expected listeners to name half of them, and listeners named 2.5%. The Heaths explain the gap:

When a tapper taps, it is impossible for her to avoid hearing the tune playing along to her taps. Meanwhile, all the listener can hear is a kind of bizarre Morse code.

Fluffy language feels precise to the person who wrote it

I call words that mean a lot to the person who writes them and little to the person who reads them fluffy language. A few examples:

  • “Strong momentum in enterprise”
  • “Significant progress on onboarding”
  • “Pricing remains a key driver of churn”
  • “We believe the market is shifting”

When you write “strong momentum,” you know which deals closed, how many, and against what target, and none of that is on the page.

The Heaths open their article with the executive version of the same problem:

Many sensible strategies fail to drive action because executives formulate them in sweeping, general language. “Achieving customer delight!” “Becoming the most efficient manufacturer!” “Unlocking shareholder value!”

Their explanation is that leaders “are simply summarizing the wealth of concrete data in their heads. But frontline employees, who aren’t privy to the underlying meaning, hear only opaque phrases.”

Readers put “we believe” anywhere from 5% to 100%

Kent’s essay proposes a short scale to pin down probability words: “almost certain” means 93% give or take almost 6%, “probable” means 75% give or take about 12%, “chances about even” means 50% give or take about 10%, “probably not” means 30% give or take about 10%, and “almost certainly not” means 7% give or take about 5%.

In a footnote, Kent says writers attach “we believe” to a probability word as a softener:

Yet on many occasions a writer will feel uncomfortable and justifiably so-with a bare “It is likely that … ” Such a bald statement is seemingly more confident than the situation would warrant. The writer will feel something akin to a compulsion towards modesty and a drive to soften the “likely” by introducing it with a “we believe” or “we estimate.” Almost invariably he does not intend to change the odds associated with “likely.”

Richards Heuer reports how far apart readers land in Psychology of Intelligence Analysis, a book the CIA published in 1999. His chart comes from a 1977 decision-analysis handbook by Scott Barclay and colleagues. Each of 23 NATO officers saw one sentence repeated with a different probability word and put a percentage on every version. “We believe” is one of the phrases on the chart. Heuer’s summary:

While there was broad consensus about the meaning of “better than even,” there was a wide disparity in interpretation of other probability expressions.

In 2015, a Reddit user who posts as zonination ran a similar survey with 46 people on the r/samplesize forum and published the raw data. The middle half of answers put “we believe” between 60% and 80%. Individual answers ran from 5% to 100%.

What probability 46 people attached to six phrases: "we believe" spreads from 5% to 100%, with the middle half between 60% and 80%

“Better than even” is the tightest band on the chart. Half the answers sit between 55% and 60%, which matches Heuer’s NATO result.

Heuer also describes what happened when he and a colleague argued about a source:

I argued he was probably bona fide. My colleague contended that the source was probably under hostile control. After several months of periodic disagreement, I finally asked my colleague to put a number on it. He said there was at least a 51-percent chance of the source being under hostile control. I said there was at least a 51-percent chance of his being bona fide. Obviously, we agreed that there was a great deal of uncertainty. That stopped our disagreement. The problem was not a major difference of opinion, but the ambiguity of the term probable.

What to write instead

Say the thing you would say if someone asked “what do you mean?” If “strong momentum” means enterprise bookings grew 18% this quarter, write that. If “significant progress on onboarding” means setup time fell from three weeks to eight days, write that.

Put a number on uncertainty, or cut the hedge. “We believe the market is shifting” can become “there’s about a 70% chance the market shifts to annual contracts within a year,” or it can become “the market is shifting to annual contracts,” with the evidence underneath. Kent’s footnote suggests the “we believe” usually wasn’t adding odds in the first place.

Find out what the reader took away. You won’t catch all of it on your own, for the same reason the tappers couldn’t. Hand the page to someone without the context and ask them to tell you what it says. That’s the ten-second test, and the same check works on a memo.

Open your last deck and circle every “we believe,” “strong,” “significant,” and “key.” Next to each one, write the number or name you had in mind when you typed it.