\section{Introduction}
A common task of observational studies is to infer the casual effect of an exposure on an outcome. Unlike well-controlled randomized experiments where experimenters use physical randomization to ensure the validity of their causal conclusions, observational studies are retrospective and suffer from many challenges that could compromise their causal conclusions. One major assumption researchers make is the treatment ignorability assumption (\citealp{rosenbaum1983central}), also known as the no unmeasured confounders assumption (NUCA) (\citealp{Robins1992NUCA}), which states that the exposure and control groups are comparable after adjusting for observed pre-exposure covariates. Unfortunately, concerns of bias from confounders that are not measured (e.g., lab results in the medical claims data) and cannot be measured  (e.g., motivation/personality in social science research) persist. 


How to address the unmeasured confounding bias? Broadly speaking, there are three types of strategies. First, one can conduct a sensitivity analysis that relaxes the NUCA and assess the effect under the posited sensitivity analysis model; see, e.g., \citet{rosenbaum2002observational,vanderweele2017sensitivity}, among many others. Methods in the second category often resort to a haphazard, natural experiment. Important examples include the instrumental variable (IV) methods (\citealp{AIR1996}) and regression discontinuity (RD) designs (\citealp{hahn2001identification}). Methods in the third category utilize the auxiliary information about the causal mechanism and the nature of the suspected unmeasured confounding, and aim to detect, \emph{quantify} and \emph{remove} the unmeasured confounding bias. One prominent example is the methodology leveraging negative control outcomes, i.e., outcome variables not affected by the treatment, and negative control exposures, i.e., exposure variables not affecting the outcome of interest (\citealp{rosenbaum1992detecting,lipsitch2010negative,Shi:2020ug}). The popular difference-in-differences (DID) estimation approach (\citealp{card1990impact}) can also be recast as a negative control outcome method \citep{sofer2016negative}. 


This article studies a method that utilizes the auxiliary information in a different way. The idea is to set aside a subset of data that is immune from the hypothesized cause-and-effect relationship, which we refer to as a \emph{placebo sample}, analyze the exposure-outcome association in the placebo sample, and formally integrate this knowledge into estimating the causal effect. Intuitively, presence of an exposure-outcome association in the placebo sample would raise serious concerns of the NUCA, while a null association \emph{corroborates} the causal conclusion.

Leveraging a placebo sample has been a popular bias detection strategy in empirical studies (see, e.g., \citet{hoynes2015income, peisakhin2018electoral}) and is mentioned in general discussions \citep{eggers2021placebo}. One prominent application scenario is policy evaluation, where individuals who are not eligible or minimally affected by the policy constitute a placebo sample. For instance, when studying the effect of minimum wages and the earned income tax credit (EITC) on deaths of despair -- deaths due to drug overdose, suicide, and alcohol-related causes, \cite{dow2020can} use college graduates as the placebo sample because they are unlikely ``to be exposed to minimum wage jobs or to be eligible for the EITC.'' Using a placebo sample for bias detection is a useful device; however, several limitations are evident. If we fail to detect bias, this cannot be equated with the absence of bias, as it could also be due to a lack of power, especially when the placebo sample size is small. Conversely, finding evidence of bias does not necessarily nullify the causal conclusion. The key is to formally incorporate the placebo sample in causal parameter identification, estimation and statistical inference. To the best of our knowledge, a formal statistical framework is currently lacking and this article aims to fill in this gap.











\section{Methods}

\subsection{Nonparametric identification}
\label{subsec: nonparametric identification}
Consider a binary exposure $A \in \{0, 1\}$ and potential outcomes $\{Y^{(0)}, Y^{(1)}\}$. Throughout the article, we assume the consistency assumption and Stable Unit Treatment Value Assumption (SUTVA) so that the observed outcome $Y$ satisfies $Y= AY ^{(1)} + (1-A) Y^{(0)}$ \citep{Rubin1980}. Suppose that we have a binary baseline covariate $S$ such that $S=0$ represents the \emph{placebo sample} that is unaffected by the exposure. We call subjects with $S=1$ the \emph{primary sample}.
\begin{assumption}[Placebo sample] \label{assump: placebo samples} For $S=0$, 
	$Y^{(1)} = Y^{(0)} = Y$  almost surely.
\end{assumption}
Figure \ref{fig: dag} illustrates a typical mechanism of a placebo sample using directed acyclic graphs (DAGs), where $A$'s (exposure) effect on $Y$ (outcome) is exclusively mediated by $M$ (mediator), and $U$ encodes the unmeasured confounders of the $A$-$Y$ relationship. For individuals in the placebo sample, $A$ does not affect $M$ and thus has no effect on $Y$. For instance, in the EITC study, $A$ encodes whether an individual's state of residence has enacted EITC laws, $M$ whether she claimed the EITC, and $Y$ whether she died of death of despair. A subset of state residents, e.g., college graduates, are unlikely to be eligible for the EITC and thus cannot claim the EITC with or without EITC laws, i.e., $A$ has no effect on $M$ (as illustrated in Figure \ref{fig: dag}b). 


Denote the other baseline covariates as $X$. Suppose that a random sample of $n$ subjects is obtained,  which is written as $\{ (Y_i, A_i, X_i, S_i): i=1,\dots, n\}$ and assumed to be independent and identically distributed according to the joint law of $(Y^{(A)}, A, X, S)$. Our target parameter is the average treatment effect of the treated in the primary sample
\begin{align}
	\theta_0 = E\{ Y^{(1)}- Y^{(0)} \mid S=1, A=1 \}. \label{eq: theta0}
\end{align}




\begin{figure}
	\centering
	\begin{tabular}[b]{cc}
		\begin{tabular}[b]{c}
			\begin{subfigure}[b]{0.4\columnwidth}
				\tikzset{
					-Latex,auto,node distance =1 cm and 1 cm,semithick,
					state/.style ={ellipse, draw, minimum width = 0.7 cm},
					state1/.style ={ draw, minimum width = 0.7 cm},
					point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
					bidirected/.style={Latex-Latex,dashed},
					el/.style = {inner sep=2pt, align=left, sloped}
				}
				\centering
				\resizebox{!}{0.32\textwidth}{\begin{tikzpicture}
						\node[state] (a) at (0,0) {${A}$};
						\node[state] (m) [right =of a] {${M}$};
						\node[state] (y) [right =of m] {${Y}$};
						\node[state] (u) [above =of m] {$U,X$};
					
						\path (a) edge node[above]{} (m);
						\path (m) edge node[above]{} (y);
						\path (u) edge node[above]{} (a);
						\path (u) edge node[above]{} (y);
						\path (u) edge node[above]{} (m);
				\end{tikzpicture}} 
				\caption{DAG for the primary sample ($S=1$).}
				\label{fig:S=1}
			\end{subfigure} \\
			\begin{subfigure}[b]{0.4\columnwidth}
				\tikzset{
					-Latex,auto,node distance =1 cm and 1 cm,semithick,
					state/.style ={ellipse, draw, minimum width = 0.7 cm},
					state1/.style ={ draw, minimum width = 0.7 cm},
					point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
					bidirected/.style={Latex-Latex,dashed},
					el/.style = {inner sep=2pt, align=left, sloped}
				}
				\centering
				\resizebox{!}{0.32\textwidth}{\begin{tikzpicture}
						\node[state] (a) at (0,0) {${A}$};
						\node[state] (m) [right =of a] {${M}$};
						\node[state] (y) [right =of m] {${Y}$};
						\node[state] (u) [above =of m] {$U,X$};
					
					
						\path (m) edge node[above]{} (y);
						\path (u) edge node[above]{} (a);
						\path (u) edge node[above]{} (y);
						\path (u) edge node[above]{} (m);
				\end{tikzpicture}} 
				\caption{DAG for the placebo sample ($S=0$).}
				\label{fig:S=0}
			\end{subfigure}
		\end{tabular}
		&
		\begin{subfigure}[b]{0.5\columnwidth}
			\centering
			\resizebox{!}{0.53\textwidth}{
				\begin{tikzpicture}
					
					\tikzstyle{dp}=[circle, inner sep=0pt, minimum size=7pt]
					
					\tikzstyle{trobs}=[dp, draw=black, fill=black]
					\tikzstyle{trlink}=[draw=black, thick]
					
					\tikzstyle{trcf}=[dp, draw=gray, fill=gray]
					\tikzstyle{trcflink}=[draw=gray,dashed, thick]
					
					\tikzstyle{uc}=[dp, draw=dblue, fill=dblue]
					\tikzstyle{uclink}=[draw=dblue, thick]
					
					\tikzstyle{lc}=[dp, draw=dorange, fill=dorange]
					\tikzstyle{lclink}=[draw=dorange, thick]
					
					\draw [->, thick] (0,0) -- (0,4.2) {};
					\draw [->, thick] (0,0) -- (7.8,0) {}; 
					\node at (0,4.5) {$E\{Y\mid S, A, X\}$};
					\draw[thick,-] (3,-0.1) -- (3,0.1) node[anchor=north,below=5pt] {$S=0$};
					\draw[thick,-] (6,-0.1) -- (6,0.1) node[anchor=north,below=5pt] {$S=1$};
					\node[uc] (trtpre) at (3,2) {};
					\node[trobs] (trtpost) at (6,4) {};
					\node[trcf] (trtcf) at (6,3) {};
					\draw [trcflink] (trtpre) -- (trtcf);
					\draw[decoration={brace,raise=7pt},decorate,thick] (6,3) -- node[left=10pt] {Trt. eff.} (6,4);
					\node[uc] (ucpre) at (3,0.5) {};
					\node[trobs] (ucpost) at (6,1.5) {};
					\draw [uclink] (ucpre) -- (trtpre);
					\draw [trlink] (ucpost) -- (trtpost);
					\draw [trcflink] (ucpre) -- (ucpost) ;
					\node[left= of trtpre] { $A=1$};
					\node[left= of ucpre] { $A=0$};
					\draw[decoration={brace,mirror,raise=7pt},decorate,thick,dblue] (3,0.5) -- node[right=10pt] {$\Delta_0(X)$} (3,2);
					\draw[decoration={brace,mirror,raise=7pt},decorate,thick] (6,1.5) -- node[right=10pt] {$\Delta_1(X)$} (6,4);
			\end{tikzpicture}}
			\caption{\label{fig: parallal} An illustration of Assumption \ref{assump: parallel}. }
		\end{subfigure}
	\end{tabular}
	\label{fig:ABC}
	\caption{(a)-(b): A typical mechanism of placebo sample. (c) An illustration of Assumption 2. \label{fig: dag}}
\end{figure}



Assumption \ref{assump: parallel} is our key identification assumption that links the subjects with $S=0$ and $S=1$. Figure \ref{fig: parallal} illustrates the idea. 

\begin{assumption} [Additive equi-confounding] \label{assump: parallel}
	$E\{ Y^{(0)}\mid S=1, A=1, X\} - E\{ Y^{(0)} \mid S=1, A=0, X\} = E\{ Y^{(0)} \mid S=0, A=1, X\} - E\{ Y^{(0)} \mid S=0, A=0, X\}$ almost surely.
\end{assumption}



The left-hand side of Assumption \ref{assump: parallel} encodes the level of confounding bias. Because the observed covariates $X$ may fail to render the exposure and control groups in the $S=1$ stratum comparable, $E\{ Y^{(0)}\mid S=1, A=1, X\} \neq E\{ Y^{(0)} \mid S=1, A=0, X\}$ in general. However, Assumption \ref{assump: parallel} states that the extent of residual confounding bias in the $S=1$ stratum is precisely equal to that in $S=0$, making it possible to debias using the placebo sample. 

A similar assumption also appears in the difference-in-differences (DID) and negative control outcome (NCO) literature. With repeated cross sectional data, DID assumes a parallel trends assumption which is a special case of Assumption \ref{assump: parallel} that uses the pre- and post-exposure time indicator as $S$. As such, DID with repeated cross sectional data can be recast as a placebo sample approach that uses a sample collected before the onset of the treatment as a placebo sample. For the NCO literature, the additive equi-confouding assumption says that the residual confounding bias for a NCO and the outcome is the same \citep{sofer2016negative}, compared to which Assumption \ref{assump: parallel} may be more reasonable as it is about the same outcome and is invariant to scaling of the outcome variable.  More discussion and sensitivity analysis of the assumptions are in  Supplement \S1.  



\begin{assumption}[Positivity] \label{assump: positive}
	For some $\epsilon>0$, $P(A= 1, S=1)>\epsilon$,  $P(S=1\mid X) <1 - \epsilon$, $\epsilon<P(A=1\mid S=0,X)<1-\epsilon$, and $P(A=1\mid S=1,X)<1-\epsilon$, with probability 1. 
\end{assumption}

We give some intuition before formally stating the identification results. For $s=0,1$, $a=0,1$, and every $x$, let $\mu_Y(s, a, x) = E\{Y\mid S=s, A=a, X=x\}$, and $ \Delta_s (x)  =\mu_Y(s, 1, x)  - \mu_Y(s, 0, x)$ be observed data functions. The causal parameter of interest can be decomposed into a contrast term $C(X)$ and a bias term $B(X)$ as follows:

\begin{equation*}
	\begin{split}
		E\{ Y^{(1)} - Y^{(0)}   \mid S = 1, A = 1, X\} 
		& = \underbrace{E\{Y^{(1)} \mid S = 1, A = 1, X\} - E\{Y^{(0)} \mid S=1, A=0, X\}}_{\text{contrast}~C(X)} \\ 
		& -\underbrace{\left[E\{Y^{(0)} \mid S=1, A=1, X\} - E\{Y^{(0)} \mid S=1, A=0, X\}\right]}_{\text{bias}~B(X)}.
	\end{split}
\end{equation*}
By the consistency assumption, the contrast function $C(X)$ equals $\Delta_1(X)$ and is identifiable from observed data. By the additive equi-confounding assumption, the bias function $B(X)$ equals $E\{Y^{(0)} \mid S=0, A=1, X\} - E\{Y^{(0)} \mid S=0, A=0, X\}$, which equals $\Delta_0(X)$ by the placebo sample assumption.  Put together, $ E\{ Y^{(1)}- Y^{(0)} \mid S=1, A=1, X \}  = \Delta_1 (X)  - \Delta_0(X) $, which is identifiable from observed data. Identification can also be from an inverse probability weighting  (IPW) approach that avoids directly modelling the outcome distribution. These two identification results are stated in Proposition \ref{prop: identification}. 




\begin{proposition}\label{prop: identification}
	(a)	Under Assumptions \ref{assump: placebo samples}-\ref{assump: parallel}, $\theta_0$ defined in \eqref{eq: theta0} is identified by 
	\[
	\theta_0=  E\left\{ \Delta_1 (X)  - \Delta_0(X)  \mid S=1, A=1 \right\}. 
	\]
	(b) Suppose that Assumptions \ref{assump: positive} also holds, then 
	\begin{align*}
		\theta_0 =  \frac{1}{E\{SA\}} E\left[ \frac{S - \pi_S(X) }{  1- \pi_S(X)  } \frac{  \pi_A(X, 1)  \{ A - \pi_A(X,S)\} }{\pi_A(X,S) \{1 - \pi_A(X,S)\}}  Y  \right] ,
	\end{align*}
	where $\pi_S(X) = P(S=1\mid X) $ and $ \pi_A(X, S) = P(A=1 \mid X, S)$. 
\end{proposition}

There are interesting connections among the DID, NCO, and our placebo sample method. Both NCO and placebo sample methods exploit known null effect: the former leverages a ``placebo'' outcome known not to be affected by the treatment, whereas the latter utilizes a ``placebo'' population known to be immune from the hypothesized cause-and-effect relationship. We view these two methods as complementary devices that can even be used in the same study. For instance, in the study of EITC on deaths of despair, \cite{dow2020can} use both devices, i.e., a cancer outcome as an NCO and college graduates as a placebo sample. In addition, the DID with longitudinal data can be interpreted as an NCO method that uses the pre-exposure outcome as an NCO, and the DID with repeated cross sectional data can be interpreted as a placebo sample method that uses a sample drawn before the exposure in place as the placebo sample. 


We note that Proposition \ref{prop: identification}(b) generalizes the IPW method proposed by \cite{Abadie:2005}, which is applicable only when $S$ is independent of all the other variables.


\subsection{Estimation and semiparametric inference}
\label{subsec:estimation}
Proposition \ref{prop: identification} suggests two estimators. The first one is a regression-based estimator 
\begin{align*}
	\hat\theta_{\rm reg}  = \frac{1}{n_{11}} \sum_{i=1}^n S_i  A_i \{ \Delta_1(X_i; \hat \beta) -  \Delta_0(X_i;  \hat \beta)\}, 
\end{align*}
where 
$n_{11} =\sum_{i=1}^n I_{(S_i=1, A_i=1)}$, $ \Delta_s(x; \hat \beta) =  \mu_Y (s,1,x; \hat\beta) -  \mu_Y (s,0,x; \hat\beta) $, $\mu_Y(s,a,x; \beta)$ is a parametric specification of $\mu_Y(s, a, x)$, and $\hat\beta$ an estimator of $\beta$. The second is an IPW estimator 
\begin{align*}
	\hat\theta_{\rm ipw}  = \frac{1}{n_{11}} \sum_{i=1}^n \frac{S_i - \pi_S(X_i; \hat\psi) }{1- \pi_S(X_i; \hat\psi)} \frac{ \pi_A(X_i, 1; \hat\alpha) \{ A_i -  \pi_A(X_i, S_i;  \hat\alpha)\} }{  \pi_A(X_i, S_i; \hat\alpha) \{1-  \pi_A (X_i, S_i; \hat\alpha)\}} Y_i ,
\end{align*}
where $\pi_S(x; \psi)$ and $\pi_A(x, s; \alpha)$ are respectively parametric models of $\pi_S(x)$ and $\pi_A(x)$, and $\hat\psi$ and $\hat \alpha$ are respectively  estimators of $\psi$ and $\alpha$. In practice, IPW-type estimators may be unstable when the (estimated) $\pi_A(\cdot)$ and $1-\pi_S(\cdot)$ are close to zero, and a common way to stabilize the weights is by normalization \citep{robins2007comment}; see the Supplement \S1.4 for details.




The reliability of the regression-based estimator and the IPW estimator depends on correct specification of different parts of the likelihood. The consistency of the former relies on $\mu_Y(s,a,x)$ being correctly specified by $\mu_Y(s,a,x; \beta)$, whereas the latter relies on $\pi_S(x)$ and $\pi_A(x, s)$ being correctly specified by $\pi_Y(x; \psi)$ and $\pi_A(x, s; \alpha)$. In practice, when we are uncertain about which models are correctly specified, it is of interest to develop a doubly robust estimator
that is guaranteed to be consistent and deliver valid inference about $\theta_0$ provided that either $\{\mu_Y\}$ or  $\{\pi_S, \pi_A\}$, but not necessarily both, are correctly specified. The next theorem derives the efficient influence function for $\theta$ \citep{Bickel:1993book, vanderVaart:2000book} in the nonparametric model, where no restrictions are placed on the distribution of observed data  $O= (Y, A, S, X)$. Theorem \ref{theo: EIF} also provides the basis of constructing a doubly-robust estimator. 


\begin{theorem}\label{theo: EIF}
	Under Assumptions \ref{assump: placebo samples}-\ref{assump: positive} and the nonparametric model, the efficient influence function for $\theta$ is 
	\begin{align*}
		& {\rm EIF} (O; \theta) =	\frac{SA}{ E\{SA\} } \{ Y -  \mu_Y(1,0,X) -  \mu_Y(0,1,X) + \mu_Y(0,0,X) - \theta\}   \\
		&  - \frac{S (1- A) \frac{\pi_A (X, 1)}{1- \pi_A(X,1)}}{E\{SA\}} \{ Y- \mu_Y(1, 0, X)\}  - \frac{ (1-S) A \frac{\pi_A (X, 1)}{\pi_A(X,0)}\frac{\pi_S(X)}{1-\pi_S(X)} }{E\{SA\}} \{ Y- \mu_Y(0, 1, X)\} \\
		&  + \frac{ (1-S) (1-A) \frac{\pi_A (X, 1)}{1- \pi_A(X,0)}\frac{\pi_S(X)}{1-\pi_S(X)} }{E\{SA\}} \{ Y- \mu_Y(0, 0, X)\}.
	\end{align*}
\end{theorem}


The efficient influence function gives an estimator $\hat\theta_{\rm dr}$ defined as the solution to $\sum_{i=1}^n \text{EIF} (O_i; \theta, \hat\eta) = 0$, where $\hat\eta= (\hat\mu_Y, \hat\pi_A, \hat\pi_S, \hat\lambda)$ denotes the collection of nuisance parameters, and $\hat\lambda = n_{11}/n$ estimates $E\{SA\}$. We prove in the Supplement \S2 that $\hat\theta_{\rm dr}$ is doubly robust. 


Next we derive the asymptotic property of $\hat\theta_{\rm dr}$. Let $\|f\|_2 =\{ \int f^2(o) dP(o)\}^{1/2}$ denote the $L_2(P)$ norm of any real-valued function $f$, and $\|f\|_2=\sum_{j=1}^\ell \|f_j\|_2$ for any collection of real-valued functions $f= (f_1,\dots, f_\ell)$, 
where $P$ denotes the distribution of $O$. Moreover, let $\eta_0 = (\mu_{Y0}, \pi_{A0}, \pi_{S0}, \lambda_0)$ denote the true values of the nuisance parameters. 

\begin{assumption} \label{assump: donsker}
	(a) $(\hat \theta_{\rm dr}, \hat\eta) \xrightarrow{P} (\theta_0, \bar\eta) $, where $\bar\eta = (\bar\mu_Y, \bar\pi_A, \bar\pi_S, \lambda_0)$ with either $\bar\mu_Y = \mu_{Y0}$ or $(\bar\pi_A, \bar\pi_S) =(\pi_{A0}, \pi_{S0})$.
	(b) For some $\epsilon>0$, $\hat\pi_S(X)<1-\epsilon, \epsilon < \hat\pi_A(0,X)<1-\epsilon,$ and $ \hat\pi_A(1,X)<1-\epsilon$ with probability 1. 
	(c) 
	For each $\theta$ in an open subset of the real line and each $\eta$ in a metric space, let $\text{EIF}(o; \theta, \eta)$ be a measurable function such that the class of functions $\{\text{EIF}(o; \theta, \eta): |\theta - \theta_0|<\epsilon, \|\eta- \bar\eta\|_2<\epsilon \}$ is Donsker for some $\epsilon>0$, and such that $E\{ \text{EIF}(O; \theta, \eta) - \text{EIF}(O; \theta_0, \bar\eta) \}^2 \rightarrow 0 $ as $(\theta, \eta) \rightarrow (\theta_0, \bar\eta)$.  
\end{assumption}

Assumption \ref{assump: donsker}(a) describes the double robustness of our proposed estimator. Assumption \ref{assump: donsker}(b) is standard for M-estimators \citep[Chapter 5.4]{vanderVaart:2000book}.

Theorem \ref{theo: asymptotics} below summarizes the doubly robust and locally efficient property of $\hat\theta_{\rm dr}$. 

\begin{theorem} \label{theo: asymptotics}
	Under Assumptions \ref{assump: placebo samples}-\ref{assump: donsker},   $\hat\theta_{\rm dr}$ satisfies
	\begin{align*}
		&  \hat\theta_{\rm dr} - \theta_0= O_P\left\{ n^{-1/2}+\big( \| \hat\pi_A - \pi_{A0} \|_2  +  \|\hat\pi_S - \pi_{S0} \|_2  \big) \| \hat\mu_Y -\mu_{Y0} \|_2 \right\},
	\end{align*}
	Suppose further that $ \big(  \| \hat\pi_A - \pi_{A0} \|_2  +  \|\hat\pi_S - \pi_{S0} \|_2  \big) \| \hat\mu_Y -\mu_{Y0} \|_2 = o_P(n^{-1/2})$, then 
	\begin{align}
		\sqrt{n} (\hat\theta_{\rm dr} - \theta_0 ) \xrightarrow{d} N\left(0, E \{ {\rm EIF} (O; \theta_0, \eta_0)^2\}\right), \label{eq: efficient}
	\end{align}
	and thus semiparametric efficient. 
\end{theorem}
The first part of Theorem \ref{theo: asymptotics} characterizes the convergence rate of $ \hat\theta_{\rm dr}$. The second part of Theorem \ref{theo: asymptotics} says that if the nuisance parameters are consistently estimated with fast rate, e.g., if they are estimated using parametric methods, then their variance contributions are negligible, and $\hat\theta_{\rm dr}$ achieves the semiparametric efficiency bound. The results in Theorems \ref{theo: EIF} and \ref{theo: asymptotics} are new, which generalize the results in \cite{sant2020doubly} to allow for $S$ being dependent on $A,X$.   


When \eqref{eq: efficient} holds, a plug-in variance estimator for $\sqrt{n} \hat\theta_{\rm dr}$ is $n^{-1}\sum_{i=1}^n {\rm EIF} (O_i; \hat\theta_{\rm dr}, \hat\eta)^2  $. Even if \eqref{eq: efficient}  does not hold, e.g., when only the model for $\{\mu_Y \}$ or the models for $\{ \pi_S, \pi_A\}$ are correctly specified, but all the nuisance parameters are finite-dimensional and in the form of M-estimators, $\sqrt{n} \hat\theta_{\rm dr}$ is still consistent and asymptotically normal from standard M-estimation theory \citep[Chapter 6]{newey1994_handbook}. Thus, a consistent variance estimator for  $\sqrt{n} \hat\theta_{\rm dr}$ can be constructed under the M-estimation framework; see details in the Supplement \S3. Alternatively, the nonparametric bootstrap is commonly used in practice.


Lastly, we remark that the Donsker condition in Assumption \ref{assump: donsker} can be relaxed by using sample splitting, which enables estimating the nuisance parameters using flexible data-driven or machine learning methods \citep{chernozhukov2017double}. In particular, \eqref{eq: efficient} holds as long as the nuisance parameters are estimated at faster than $n^{-1/4}$-rates. 




\section{Simulation study}
We compare the placebo sample approach to naive methods based on NUCA, and investigate the operating characteristics of various estimators proposed in Section \ref{subsec:estimation}. We simulate the full data according to the following data-generating process with sample size $n = 1000$:
\begin{enumerate}
	\item[(a)] $X = (X_1, X_2, X_3)$ where $X_j \sim N(0, 1)$ for $j = 1, \dots, 3$.
	\item[(b)] $S$ is Bernoulli with $P(S = 1 \mid X) = \text{expit}\{-X_1 - X_2 + 3X_3 - X_2 X_3\}$,
	\item[(c)] $A$ is Bernoulli with $P(A = 1 \mid X, S) = \text{expit}\{-X_1 - X_2 + X_3 + X_2 X_3 + 0.2S + 0.5\}$.
	\item[(d)] $U$ is Bernoulli with $P(U = 1 \mid X, S, A)$ satisfying (d1) $P(U = 1 \mid X, S, A) = 0.6A + 0.2$ and (d2) $P(U = 1 \mid X, S, A) = 0.6A + 0.2\times\text{sign}\{X_1 + X_2\} + 0.2$.
	\item[(e)] $Y(0) \mid X, U, S$ is Normal with unit variance and mean satisfying (e1)  $E\{Y(0) \mid X, U, S\} = -X_1 - X_2 + 0.5X_3 + 0.5X_2X_3 + 2U + 2$ and (e2) $E\{Y(0) \mid X, U, S\} = -X_1 - X_2 + (X_3 + 0.5X_2X_3)S + 2U + 2$.
	\item[(f)] For $S=0$, $Y(1) = Y(0)$. For $S=1$, (f1) $Y(1) - Y(0) = 1$ and (f2) $Y(1) - Y(0) \sim N(1, 0.5)$. The observed outcome $Y = AY(1) + (1-A)Y(0)$. The target parameter $\theta_0=1$. 
\end{enumerate}

The observed data are $ \{(X_i, S_i, A_i, Y_i), i=1,\dots, n\}$.   Four combinations of (d) and (e) all satisfy Assumption \ref{assump: parallel}. Our simulation study can be summarized by the factorial design below:
\begin{description}
	\item \textbf{Factor 1:} Data-generating process of $U\mid X, S, A$: (d1) and (d2);
	\item \textbf{Factor 2:} Data-generating process of $Y(0) \mid X, U, S$: (e1) and (e2);
	\item \textbf{Factor 3:} Treatment effect: (f1) and (f2);
	\item \textbf{Factor 4:} Estimator of ${\theta}_0$: (I) a naive regression estimator $\hat\theta_{\rm reg, naive}$ that regresses $Y$ on $A$ and $X$ in the $S = 1$ stratum, (II) a naive doubly robust (AIPW) estimator $\hat\theta_{\rm dr, naive}$ based on subjects in $S = 1$, and three placebo sample estimators proposed in Section \ref{subsec:estimation}: (III) regression-based estimator $\hat\theta_{\rm reg}$, (IV) stabilized IPW estimator $\hat\theta_{\rm ipw}$, and (V) doubly-robust estimator $\hat\theta_{\rm dr}$. 

\end{description}
Factors 1-3 specify $8$ scenarios, and Factor 4 specifies estimators of $\theta_0$. We further consider three placebo sample estimators under different model misspecification. All model misspecification refers to omitting an interaction term involving $X_2 X_3$ when fitting $\mu_Y$, $\pi_S$ and $\pi_A$. Table \ref{tbl: simu n = 1000 main} summarizes the simulation results for nine estimators in 3 scenarios. The remaining 5 scenarios can be found in the Supplement \S4. Estimators are evaluated in terms of their bias, median estimated standard error, and coverage of the $95\%$ confidence interval based on the nonparametric bootstrap using $2000$ bootstrap iterations. Both naive estimators ($\hat\theta_{\rm reg, naive}$ and $\hat\theta_{\rm dr, naive}$) are largely biased due to the unmeasured confounding. Among the three proposed estimators that leverage the placebo sample,
the regression-based estimator $\hat\theta_{\rm reg}$ has the smallest variance when $\pi_Y$ is correctly specified but becomes biased when $\pi_Y$ is misspecified. The IPW estimator $\hat\theta_{\rm ipw}$ has large finite-sample bias and poor coverage even when $\{\pi_S,\pi_A\}$ are correctly specified. The doubly robust estimator $\hat{\theta}_{\rm dr}$ is approximately unbiased in all three cases, and has smaller variance when all models are correctly specified compared to when only a subset of the models are correctly specified. We recommend using the doubly robust estimator $\hat{\theta}_{\rm dr}$ based on its robustness property and simulation results. In the Supplement \S3, we also discuss how to construct an empirical sandwich variance estimator, and provide \textsf{R} code implementing it.

\begin{table}[ht]
	\centering
	\caption{Simulation results for three scenarios. Scenario I: d = (d1), e = (e1), f = (f1). Scenario II: d = (d2), e = (e1), f = (f1). Scenario III: d = (d1), e = (e1), f = (f2). $\theta_0 = 1$ in all three scenarios. We trimmed the 1\% tail of the most extreme values of each estimator when reporting the bias.}
	\label{tbl: simu n = 1000 main}
	\resizebox{\textwidth}{!}{
		\begin{tabular}{lcccccccccc}
			\hline
			Estimator & \begin{tabular}{c}Model \\ Spec.\end{tabular} & Bias &\begin{tabular}{c}Median \\ Est. SE\end{tabular}& \begin{tabular}{c}Cov. \\ $95\%$ CI\end{tabular}& Bias &\begin{tabular}{c}Median \\ Est. SE\end{tabular}& \begin{tabular}{c}Cov. \\ $95\%$ CI\end{tabular}& Bias &\begin{tabular}{c}Median \\ Est. SE\end{tabular}& \begin{tabular}{c}Cov. \\ $95\%$ CI\end{tabular} \\ 
			\hline
			& &\multicolumn{3}{c}{Scenario I} &\multicolumn{3}{c}{Scenario II}&\multicolumn{3}{c}{Scenario III}\\
			$\hat\theta_{\rm reg, naive}$  & & 1.20 & 0.16 & 0.00\% &1.17 & 0.16 & 0.00\% &1.19 &0.16 &0.00\%\\ 
			$\hat\theta_{\rm dr, naive}$ & & 1.21 & 0.18 & 1.70\% &1.20 & 0.16 & 0.80\% &1.19 &0.18 &1.70\%\\ 
			$\hat\theta_{\rm reg}$ & \begin{tabular}{c}$\mu_Y$ \\ correct\end{tabular} & 0.00 & 0.19 & 94.0\% &0.01 & 0.19 & 94.1\% &0.00 &0.19 &95.7\%\\ 
			$\hat\theta_{\rm reg}$ & \begin{tabular}{c}$\mu_Y$ \\ incorrect\end{tabular} &  -0.36 & 0.20 & 53.5\% &-0.35 &0.20 &56.1\% &-0.36 &0.20 &54.4\%\\ 
			$\hat\theta_{\rm ipw}$ &\begin{tabular}{c}$(\pi_S, \pi_A)$ \\ correct\end{tabular} & -0.26 & 0.61 & 87.0\% &-0.19 &0.54 &89.3\% &-0.28 &0.61 &89.4\%\\ 
			$\hat\theta_{\rm ipw}$ &\begin{tabular}{c}$(\pi_S, \pi_A)$ \\ incorrect\end{tabular}& -0.52 & 0.63 & 81.5\% &-0.44 &0.54 &83.8\% &-0.50 &0.623 &82.8\%\\ 
			$\hat\theta_{\rm dr}$ & \begin{tabular}{c}All \\ correct\end{tabular} & 0.00 & 0.47 & 92.7\% &0.02 &0.43 &93.1\% &-0.07 &0.48 &94.5\%\\ 
			$\hat\theta_{\rm dr}$ &  \begin{tabular}{c}$\mu_Y$ \\ correct\end{tabular} & -0.01 & 0.61 & 92.0\% &-0.02 &0.53 &92.6\% &-0.01 &0.61 &94.3\%\\ 
			$\hat\theta_{\rm dr}$ & \begin{tabular}{c}$(\pi_S, \pi_A)$ \\ correct\end{tabular} &0.01 & 0.50 & 90.9\% &-0.01 &0.46 &92.2\% &-0.11 &0.51 &90.9\%\\ 
			\hline
	\end{tabular}}
\end{table}


\section{Application: The effect of EITC on infant health}
Following aspects of the study by \cite{hoynes2015income}, we apply our methods to study the effect of 1993 EITC reform on low birth weight among a ``high-impact sample'' that consists of single mothers age 18 and older with a high school education or less. The treated group is second- and higher order births and the control group is the first births, because firth births were exposed to relatively small EITC credit. Our placebo sample is the single mothers who are college graduates. The accompanying data in \cite{hoynes2015income} are collapsed to cells defined by state, year, parity of birth (first, second, third, fourth or greater birth to a mother), education of mother ($<$12, 12, 13-15, $>$16), race of the mother (white, black, other), ethnicity of the mother (Hispanic, non-Hispanic, missing), and age of mother (18-24, 25-35, 35+). Since the low birth weight is a binary outcome, we can recover individual-level data from the given size and fraction of low birth weight of each cell, which gives us 811,424 subjects in the high-impact sample and 53,131 subjects in the placebo sample. \textcolor{black}{As discussed in \S 1, as well as in \cite{hoynes2015income} and \cite{dow2020can}, the subgroup with high education is unlikely to be eligible for the EITC. This makes it appealing to consider the college graduates as a placebo sample. In this application, there is a concern of unmeasured confounding due to pre-exposure behaviors such as smoking and drinking, which may differ between the exposure and control groups, and can lead to well-established decreases in birth weight. As discussed in the Supplement \S1.1, a sufficient condition for Assumption \ref{assump: parallel} is that (i) within each socioeconomic cell defined above, the smoking and drinking rates are similar for single mothers in the placebo sample and the primary sample; and (ii) the effect of smoking and drinking on birth weight is not modified by the mother's education. The details of a sensitivity analysis are in the Supplement \S1.2.
}


For illustration, we calculate the proposed estimators based on effective tax year 1994 (in percentage points): the regression-based estimator $\hat\theta_{\rm reg} =-0.565$ (SE= 0.275), the IPW estimator $\hat\theta_{\rm ipw} =-0.384$ (SE= 1.101), and the doubly robust estimator $\hat\theta_{\rm dr} =-0.665$ (SE= 0.300). All estimators adjust for the aforementioned parity and demographic variables, as well as two-way interactions between the   demographic variables. The standard errors are calculated using the nonparametric bootstrap with 100 iterations. The relative performance of these three estimators is similar to what we have seen in simulation. All estimators indicate that, for second parity or higher births among the high-impact sample, the increased income through the EITC leads to reduced risk of low birth weight in the effective tax year 1994. Based on the regression-based estimator, the risk would be 0.565 (95\% CI: 0.026 to 1.104) percentage points lower (relative to the overall mean of 10.03 percent).





\bibliographystyle{apalike}
