Causal Inference And Missing Data

1:30 AM thoughts on how to learn causal inference The Right Way (tm)

Sep 04, 2026

A piece of a puzzle with a missing piece — Photo by Tanja Tepavac on Unsplash

Enclosed are some thoughts about causal inference that I had on September 4th, 2026, at approximately 1:30 a.m., Chicago time, the best time for clear thinking.

Every causal inference problem is about how you’re missing some data, and every solution is about ignoring that fact.

“Causal inference” is sexier than “missing data.” Missing data evokes “data cleaning,” the least sexy task in any corporation, a task best handled by someone else.

Causal inference evokes telling Other People that the chart they like so much isn’t causal. You can tell little stories like “this chart is like saying ice cream sales cause summer”—which is fun, and lets you embrace and nourish the natural smugness required to enter any scientific field.

[If the Youth just changed the construction “it’s giving [adjective]” to “it evokes [adjective]”, they, too, would sound like an urbane, sophisticated critic…]

But causal inference is no more than a special type of missing data problem. We know how the world works under one policy or customer decision, but we want to know how it’d look if they’d faced a different policy or made a different choice. We’re missing data on what could have been.

There are both infinite ways to attempt to establish causality and very few methods, like there are more novels than anyone could read in twenty lifetimes, but so few stories.

Say we have a treatment and we know which treatment each unit experienced (Z). We want to know how it changes the distribution of an outcome (Y). For each value of z, we can define the “potential value” Y(z), which indicates what Y would be if the unit had received a different treatment. We observe what actually happened, i.e., Y(Z) = sum 1(Z = z) Y(z). We only know what happened for one treatment.

Write down what you know.
Write down what you want to know.
Bridge the gap.

So, what do we know? We know the distribution of [Y(Z), Z]. We observe what actually happened for the treatment assigned to our units, and we know which treatments they were assigned.

What do we want to know? We want to know the distribution of Y(z) for any particular treatment z. We want to know what the world would look like if folks in our population made different decisions or faced a different environment.

Now, let’s bridge the gap:

We can write the distribution of Y(z) as:

P[Y(z)] = P[Y(z) | Z = z] Pr(Z=z) + P[Y(z) | Z != z] (1 - Pr(Z=z))

This is just a basic fact of probability distributions: the law of total probability. It doesn’t tell us anything. It cuts things into pieces, so we can see them clearly.

The key is to start building the bridge on the ground you are sure about. The thing we know is the distribution of Y for the treatments people actually received, so we’ve got:

Pr[Y(z) | Z = z],

because that is just the distribution of Y for units that actually faced the treatment z. So, we can just measure that from the data.

We also know which treatments the units received. So we know Pr(Z = z).

What we don’t know is just:

P[Y(z) | Z != z],

and that’s all of causal inference.

We don’t know this factor because we’re missing data on Y(z) for the population that doesn’t receive treatment z. All of causal inference is about coming up with different assumptions that tell us the value of this term, despite not having any data on it.

Of course, we have an alternative. We could refuse to make assumptions. Always consider the outside option. There is a way to restrict P[Y(z) | Z != z] that doesn’t depend on us taking any leaps of faith at all. We know:

0 <= P[Y(z) | Z != z] <= 1,

which tells us that:

P[Y(z)|Z=z] Pr(Z=z) <= P[Y(z)] <= P[Y(z)|Z=z] Pr(Z=z) + (1 - Pr(Z=z))

Which isn’t a lot, but it’s something. You have to start somewhere. In fact, if only a little data is missing, the gap between the bounds is fairly narrow. If most people in the population of interest receive treatment z, then we’ve got strong identification from the data alone. But if, say, 50% of the population doesn’t see it, then the gap between the lower and upper bounds is 50%—which is huge, when we’re talking about probability distributions.

The Assignment Assumptions impose restrictions on how treatments are assigned, allowing us to use the data we have for Y(z) from the Z=z population to translate it into a number for P[Y(z) | Z != z].

If you’ve ever run an experiment, what you’re relying on is the following Assignment Assumption.

Random Assignment. Suppose that treatment assignment is random in the sense that it is independent of Y(z).

If Random Assignment is true, then conditioning on Z=z or Z != z doesn’t change the distribution of Y(z) because it’s independent of assignment. So, we can write:

P[Y(z) | Z != z ] = Pr[Y(z) | Z = z]

And we can measure the right-hand side! It’s the same as the first factor and, in fact:

P[Y(z)] = P[Y(z) | Z = z]

When random assignment is true, correlation is causation.

A natural extension of Random Assignment is to suppose it holds conditional on some other variables.

Conditional Independence Assumption. This assumption says that, conditional on some X (which we observe), Y(z) and Z are independent, i.e., P[Y(z) | Z=z, X=x] = P[Y(z) | X=x].

This is what you’re relying on when you run a regression “controlling for X” or when you use inverse propensity score weighting or when you use a matching estimator or… It’s used a lot.

The assumption is, roughly: Well, yes, there’s selection, but it’s determined by something I know about. All we have to do is condition on a few things, and viola: we’re back to random assignment.

It isn’t a very credible assumption. The idea that there’s really nothing we don’t observe that plays a major role… I’ve never really heard a convincing argument for the strong version of this assumption. The best genre of justification is along the lines of: we know there’s one huge selection problem, and I can control for it. Are there other problems? Sure, but I’ve knocked out one big one. It’s not a bad argument as things go, despite the complexity of interpreting the conditional independence claim correctly as you add more and more X’s. And we shouldn’t discount the good—no, great—thing about this assumption is that you usually have the data to use it. You can always regress Y on X and Z, and it really does eliminate some forms of selection and mitigate others.

There are other assumptions in the Assignment category, I suppose, but the ones that give point identification are all—more or less—different varieties of these. We argue there’s some way to treat our data as an experiment, and then we go from there.

I wrote a few ’grafs about other approaches, but then saw myself veering off the tracks, and the post stopped being a blog post and veered dangerously close to something worse. A textbook. So I snipped it.

Besides, it’s all just variations on the theme.

Here’s a good exercise. If you’re learning causal inference and want to be able to write recipes instead of just reading the cookbook, answer the following question about any standard, workhorse methods you learn:

What does it say P[Y(z) | Z != z] is?
Does the method even estimate P[Y(z)] (or E[Y(z)])? One trick the “causal inference” methods use is to change the question to get their answer. Causation is hard. Sometimes you have to cheat. They switch to “treatment on the treated” or localize the treatment effect to a specific subset of the population, etc. It’s fine, really. Just watch out for the trick where these changes suddenly get reversed in the policy analysis, and we’re back to treating the estimate as a global treatment effect.
What does the data alone tell you about whatever parameter the method’s actually estimating?
How does the method fill in the gaps?

And so on. Answering these questions tells you what’s really going on.

The downside of grokking the methods is that it’ll make you critical. It’s very easy to spot problems in a causal analysis because it’s a hard problem. You’ll find flaws in every analysis.

The cure to this disease is being forced to do it yourself.

Like any positive theory of how to learn something about the world, causal inference has a nihilistic response. There are two species of nihilism in causal inference: