Analyzing Matching-Based Parameter Estimators in Black-Box Data Generation Settings

Assume that, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\mathbf { X}}={\mathcal { S}}(\theta _{0},{\mathbf { Y}})$ </tex-math></inline-formula>, are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n \ i.i.d$ </tex-math></inline-formula>. observations in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbb {R}^{p}$ </tex-math></inline-formula>, with cdf <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F_{\theta _{0}}$ </tex-math></inline-formula> of unknown mathematical form; <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal { S}$ </tex-math></inline-formula> is a black-box, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p\ge 1$ </tex-math></inline-formula>. Parameter <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\theta _{0}$ </tex-math></inline-formula> is unknown, element of metric space <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(\Theta , d_{\Theta })$ </tex-math></inline-formula> which can be infinite dimensional. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</b> is random, with distribution independent of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\theta _{0}$ </tex-math></inline-formula>. For each <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\theta ^{*}$ </tex-math></inline-formula> in a discretization, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Theta ^{*}$ </tex-math></inline-formula>, of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Theta , \ F_{\theta ^{*}}$ </tex-math></inline-formula> is “learned” using independent <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\mathbf { X}}^{*}_{j}(\theta ^{*})={\mathcal { S}}(\theta ^{*}, {\mathbf { Y}}), 1\le j \le N_{rep}$ </tex-math></inline-formula>. Kolmogorov distance, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d_{K}$ </tex-math></inline-formula>, and the empirical cdfs of the observed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bf x$ </tex-math></inline-formula> and of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{rep}~{\mathbf { X}}^{*}_{j}(\theta ^{*}), \theta ^{*} \in \Theta ^{*}$ </tex-math></inline-formula>, are used to introduce for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\theta _{0}:~a$ </tex-math></inline-formula>) the minimum matching distance estimator (MMDE), and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</i>) the brand new, maximum matching support probability estimator (MMSPE). Under two mild assumptions, uniform consistency for both estimators is established. Their common upper error rate in probability depends on Kolmogorov’s entropy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Theta $ </tex-math></inline-formula>, is not affected by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> when <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Theta \subseteq \mathbb {R}^{m}$ </tex-math></inline-formula>, and establishes that MMSPE is competitive. Both estimators have theoretical advantages over competitors, and complement each other in numerical applications presented for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p=1, 2$ </tex-math></inline-formula>, because of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d_{K}$ </tex-math></inline-formula>’s computational cost for large <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i>, relaxed with recent algorithms. MMSPE obtained using, instead of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d_{K}$ </tex-math></inline-formula>, the Euclidean sup-norm distance of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bf x$ </tex-math></inline-formula> from each of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{rep}~{\mathbf { X}}^{*}_{j} (\theta ^{*}), \theta ^{*} \in \Theta ^{*}$ </tex-math></inline-formula>, provides an approximation for the MLE when <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{rep}$ </tex-math></inline-formula> is large, indicating MLE’s limitations for large <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i>. MMDE and MMSPE obtained using Wolfowitz’s half-spaces distance and matching tool the empirical measure instead of the empirical cdf, have a computational advantage for large <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i>.