CLT under Local Dependence: When Observations Are Connected
On this page
When observations share random inputs, the variance of their sum includes covariances between them. We study when dividing the centered sum by its standard deviation gives an approximately standard normal distribution. A dependency graph specifies which groups of observations must be independent. Under this condition, a bound from Stein's method gives sufficient conditions for a central limit theorem (CLT). We then compare it with a bound that directly controls the difference between distribution functions.
The starting result is Ross’s Theorem 3.5. We give the probabilistic argument in full under an explicit dependency-graph assumption. The later Chen–Shao theorem is stated with its conditions and applied to the same example; its longer concentration-inequality proof is not reproduced here.
1. The sum and its dependency graph
Let (X1,…,Xn)∼Pn be a real random vector with EPnXi=0 and finite fourth moments. The observations may be dependent, so Pn need not be a product of their marginal distributions. Throughout the graph proof, unmarked E, Var, Cov, and P use this joint distribution or its relevant marginal. The graph is fixed; we do not average over randomly generated graphs. Define
S=i=1∑nXi,σ2=Var(S)>0,W=σS.
Scroll to see the full equation →
A deterministic undirected graph G on {1,…,n} is a dependency graph if, for any disjoint vertex sets I,J with no edge between them,
(Xi)i∈I⊥⊥(Xj)j∈J.
Scroll to see the full equation →
This is independence of two random vectors, not merely pairwise independence. For example, if U,V are independent random signs, then U,V,UV are pairwise independent, but
U⋅V⋅(UV)=1almost surely.
Scroll to see the full equation →
The empty graph is therefore not a dependency graph for this triple.
Write
Ni={i}∪{j:{i,j} is an edge of G},D=imax∣Ni∣.
Scroll to see the full equation →
Thus D counts the vertex itself; it is one plus the maximum graph degree. Because the graph is undirected, j∈Ni if and only if i∈Nj. Nonneighbors are independent, so
Thus normalization must include covariances between distinct neighboring observations, even when each observation depends on only a small neighborhood.
Constructing the graph from independent inputs
Suppose {εv:v∈V} are mutually independent, and each observation has the form
Xi=gi((εv)v∈Bi),Bi⊆V.
Scroll to see the full equation →
Joining i and j whenever Bi∩Bj=∅ gives a dependency graph: two groups with no connecting edge are functions of disjoint sets of independent inputs.
For example, when Bi is the closed one-hop neighborhood of i in an underlying network, the dependency graph may require edges between vertices at network distance two. If the underlying maximum degree is d, then
D≤1+d+d(d−1)=1+d2.
Scroll to see the full equation →
This construction requires the independent-input representation. A social or spatial network by itself does not establish the required independence.
2. Why the normal distribution leads to Stein’s equation
Let Z∼N(0,1), with density φ(x)=(2π)−1/2e−x2/2 and distribution function Φ. Expressions Eh(Z) integrate only Z∼N(0,1); expressions Eh(W) integrate W=∑iXi/σ under Pn. The comparison requires no joint coupling of W and Z. Since φ′(x)=−xφ(x), integration by parts gives, for a bounded continuously differentiable f with bounded derivative,
where Lip(h)≤1 means ∣h(x)−h(y)∣≤∣x−y∣ for every x,y. These expectations exist because W and Z have finite first moments.
To use this distance, we need bounds on the solution of Stein's equation. The following Stein-factor lemma supplies them: for such h, the bounded solution satisfies
∥fh′∥∞≤π2,Lip(fh′)≤2.
Scroll to see the full equation →
For smooth h, the second inequality can be written ∥fh′′∥∞≤2; the Lipschitz formulation also covers nonsmooth h. We use these bounds from Ross, Lemma 2.5. They control the Taylor remainder below, leaving us to bound moments of local sums and count which products can be dependent.
3. A dependency-graph normal-approximation theorem
Theorem 1. Under the centering, fourth-moment, positive-variance, and dependency-graph assumptions above,
This is the bound in Ross, Theorem 3.5, used here with the full graph-independence condition. The proof separates two errors: a Taylor remainder and the fluctuation of a local quadratic sum. No identical-distribution assumption is needed.
Removing the neighborhood creates independence
First, define
ξi=σXi,ηi=j∈Ni∑ξj,Wi=W−ηi=j∈/Ni∑ξj.
Scroll to see the full equation →
The graph condition gives ξi⊥⊥Wi. Centering then implies
E[ξifh(Wi)]=EξiEfh(Wi)=0,
Scroll to see the full equation →
and hence
E[Wfh(W)]=i∑E[ξi{fh(W)−fh(Wi)}].
Scroll to see the full equation →
Next, expand around W. The fundamental theorem of calculus and the Lipschitz bound on fh′ give
The final inequality is Cauchy–Schwarz, using E(1−V)=0. Taking the supremum over h proves the intermediate bound
dW(W,Z)≤π2Var(V)+i∑E(∣ξi∣ηi2).
Scroll to see the full equation →
This intermediate bound retains the individual neighborhoods and covariances. When they are known, evaluating it directly can avoid replacing every local contribution by a worst-case maximum-degree bound.
Bounding the Taylor remainder
Expanding the neighborhood square and applying the triangle inequality gives
i∑E(∣ξi∣ηi2)≤σ31i∑j,k∈Ni∑E∣XiXjXk∣.
Scroll to see the full equation →
By the arithmetic–geometric mean inequality,
∣XiXjXk∣≤3∣Xi∣3+∣Xj∣3+∣Xk∣3.
Scroll to see the full equation →
To count each contribution, write ai=E∣Xi∣3. There are at most D2 choices of (j,k) for a fixed i, while symmetry of the neighborhoods gives
If {i,j} and {k,ℓ} are disjoint and no graph edge connects them, the corresponding products are independent and their covariance vanishes. Thus a potentially nonzero term must satisfy
k∈Ni∪Njorℓ∈Ni∪Nj.
Scroll to see the full equation →
Let Q be the set of ordered quadruples satisfying j∈Ni, ℓ∈Nk, and this last condition. For each fixed (i,j), the first alternative allows at most 2D⋅D pairs (k,ℓ). The second allows at most another 2D⋅D, since ℓ∈Nk is equivalent to k∈Nℓ. Consequently,
#{(j,k,ℓ):(i,j,k,ℓ)∈Q}≤D⋅4D2=4D3.
Scroll to see the full equation →
The same bound holds with any one of the four positions fixed: Q is invariant under interchanging the members of either pair and under interchanging the two pairs.
Write mi=EXi4. Cauchy–Schwarz, followed by Hölder and the arithmetic–geometric mean inequality, gives
Combining the two error bounds proves Theorem 1. The direct covariance count under our full graph assumption actually permits 8/π in place of 26/π; we retained the latter to display the source theorem's constant. Neither constant is asserted optimal.
4. When does the bound imply a CLT?
For a triangular array {Xni:1≤i≤n}, let Dn, σn, and Wn denote the corresponding quantities. Theorem 1 gives
For a simple sufficient condition, assume uniformly bounded third and fourth moments and σn2≥cn for some fixed c>0. Then
dW(Wn,Z)≤C(nDn2+nDn3/2),
Scroll to see the full equation →
where C depends only on the moment bounds and c. In particular,
Dn=o(n1/4)⟹Wn⇒N(0,1).
Scroll to see the full equation →
This is a sufficient condition from this bound, not a necessary degree threshold. The variance assumption must be verified separately; local dependence allows negative as well as positive covariances.
Example: Overlapping observations on a path
Let ε1,…,εn+1 be independent random signs, each taking −1 and 1 with equal probability, and set
Xi=εi+εi+1,1≤i≤n.
Scroll to see the full equation →
Only adjacent observations share an input. The path graph is a dependency graph, with D≤3. Direct calculation gives
The normalization can also be checked by collecting the independent inputs:
S=ε1+2i=2∑nεi+εn+1.
Scroll to see the full equation →
Ignoring the adjacent covariances would give variance 2n. By Slutsky's theorem, the resulting incorrectly normalized sum instead satisfies
2nS=2n4n−2Wn⇒N(0,2),
Scroll to see the full equation →
not a standard normal limit.
The sign of the shared input also matters. Replacing Xi by εi−εi+1 preserves the path dependency graph, but telescoping gives
S=ε1−εn+1,σ2=2,P(S/σ=0)=21.
Scroll to see the full equation →
This standardized distribution is the same for every n and is not normal. Bounded neighborhoods and bounded observations alone therefore do not suffice.
Example: Growing neighborhoods can still permit a CLT
Partition n=mnbn observations into mn independent blocks, each of size bn. Within block r, set every observation equal to a single random sign εr. A union of cliques is a dependency graph, and
Dn=bn,Sn=bnr=1∑mnεr,σn2=mnbn2=nbn.
Scroll to see the full equation →
All third absolute and fourth moments equal one. Substitution, without imposing σn2≍n, gives
σn3Dn2nσn2Dn3/2n=mn1,=mn1.
Scroll to see the full equation →
Therefore mn→∞ suffices, even when bn grows faster than n1/4. Indeed,
Wn=mn1r=1∑mnεr.
Scroll to see the full equation →
At the opposite extreme, one block gives Wn=ε1 for every n. Increasing the number of copies of the same observation does not create a normal limit.
5. From smooth test functions to distribution functions
For distribution-function error, define the Kolmogorov distance
dK(W,Z)=z∈Rsup∣P(W≤z)−Φ(z)∣.
Scroll to see the full equation →
The Wasserstein bound already implies convergence in this metric. To see this directly, let hz,δ equal one on (−∞,z], decline linearly to zero on [z,z+δ], and equal zero afterward. Its Lipschitz constant is 1/δ. Since φ≤M=(2π)−1/2,
A ramp on [z−δ,z] gives the same bound for the opposite difference. Minimizing over δ>0 yields
dK(W,Z)≤2MdW(W,Z)=(π2)1/4dW(W,Z).
Scroll to see the full equation →
If dW=0, take δ↓0. Otherwise the minimizing value is δ=2dW/M. For the path example this gives a Kolmogorov bound of order n−1/4, losing a square root relative to the Wasserstein rate.
6. Chen–Shao: A direct Berry–Esseen bound
The preceding smoothing argument turns a Wasserstein rate into a weaker distribution-function rate. To avoid that loss, we can work directly with hz(w)=1{w≤z}. Its Stein solution satisfies
fz′(w)−wfz(w)=1{w≤z}−Φ(z),
Scroll to see the full equation →
with the derivative at z defined by this equation. The derivative now has an indicator discontinuity, so the earlier global Lipschitz bound on fz′ cannot be used. In the integral remainder, the indicator part satisfies
∫0η{1{W−t≤z}−1{W≤z}}dt≤∣η∣1{∣W−z∣≤∣η∣}.
Scroll to see the full equation →
The new task is to control probability in a short interval around z, including intervals whose widths depend on the local variables. This is the role of the concentration inequalities in Chen and Shao (2004), Sections 3–4.
Nested local-dependence conditions
Continue to use normalized summands ξi=Xi/σ, so that Var(∑iξi)=1. For a set A, write ξA=(ξj)j∈A and let Ac denote its complement in {1,…,n}. The local-dependence conditions are
The conditions successively require independence for an observation, its first neighborhood, and a larger block. A dependency graph supplies these separations by taking
Ai=Ni,Bi=j∈Ai⋃Nj,Ci=j∈Bi⋃Nj.
Scroll to see the full equation →
For general local-dependence models, LD1 alone does not justify these larger-block independence statements.
Theorem 2: Chen–Shao’s uniform bound
Assume LD3 and finite pth absolute moments for some 2<p≤3. Define
Unlike the fourth-moment graph bound, this result controls distribution functions directly and permits p≤3. Its neighborhood parameter κ differs from D. It also counts how enlarged neighborhoods intersect and how many of them contain a given index, as specified above. We quote the theorem rather than reproduce its proof. The indicator-remainder calculation explains why the earlier Taylor argument alone is insufficient.
For the path example, choose Ai, Bi, and Ci to have radii one, two, and three. Then
Taking p=3, κ≤11, σ2=4n−2, and E∣Xi∣3=4 therefore gives
dK(4n−2S,Z)≤(4n−2)3/275⋅112⋅4n=O(n−1/2).
Scroll to see the full equation →
The rate improves on the n−1/4 obtained by smoothing our Wasserstein bound, although the displayed numerical constant is large. This is a comparison for this example, not a claim that every Chen–Shao bound is numerically sharper than every graph bound.
More generally, if κn is bounded, supn,iE∣Xni∣p<∞, and σn2≥cn, then
dK(Wn,Z)≤Cn1−p/2⟶0.
Scroll to see the full equation →
7. What is still needed for statistical inference?
The earlier doubly robust note separates a population identity from estimation error. A dependence CLT requires the same separation. For a target θn, suppose an estimator θn admits
n(θn−θn)=i=1∑nXni+rn,rn=oP(σn),
Scroll to see the full equation →
with centered leading summands satisfying one of the preceding CLTs. Then
σnn(θn−θn)=Wn+oP(1)⇒N(0,1).
Scroll to see the full equation →
If, in addition, a positive estimator σn of σn satisfies σn/σn→P1, Slutsky's theorem gives
σnn(θn−θn)⇒N(0,1).
Scroll to see the full equation →
The dependency graph must describe these leading summands, not just the observed network. Shared fitted nuisance functions can introduce dependence beyond graph neighbors; a separate expansion and remainder argument is then required. Likewise, the CLT does not itself prove consistency of σn.
For a random graph, the deterministic-graph theorem can instead be applied conditionally when its centering, independence, moment, and positive-variance conditions hold given that graph. The conditional approximation error must tend to zero, and the normalization must use the corresponding conditional variance. Randomness of the graph alone does not verify those conditions.
References
Nathan Ross (2011). Fundamentals of Stein’s method. Lemma 2.5 supplies the Stein factors; Section 3.2, Theorem 3.5, supplies the dependency-neighborhood bound.
Louis H. Y. Chen and Qi-Man Shao (2004). Normal approximation under local dependence. The Annals of Probability, 32(3A), 1985–2028. Section 2 defines LD1–LD3; Theorem 2.4 gives the uniform Berry–Esseen bound used here, with concentration inequalities and proofs in Sections 3–4.