Finite-Sample Recovery of Predictive History Quotients from Independent POMDP Trajectories

We study a modest but identifiable abstraction problem for partially observed control: under one fixed data-collection policy, which bounded observation contexts have the same finite-horizon continuation law? We define the *episode-position predictive quotient*, an explicitly policy- and position-dependent estimand. Our estimator first selects contexts on one half of independently reset trajectories, then takes at most one prespecified continuation block from each trajectory in the other half. It compares fixed- alphabet histograms by empirical total variation and abstains whenever a pair lacks the samples required by its own concentration bound. Under an $`\varepsilon`$-gap, a per-context visitation floor, and a selection threshold no larger than the inference threshold, we prove exact recovery with probability $`1-\delta`$ from $`O((d+\log(K/\delta))/(\mu_{\min}\varepsilon^2))`$ trajectories, where $`d`$ is the continuation-alphabet size and $`K`$ the selected context count. A two-point construction shows the dependence on $`\mu_{\min}`$ and $`\varepsilon`$ is unavoidable; the dependence on $`d`$ is not matched and is known to be improvable by optimal closeness testers. The guarantee is non-asymptotic: it does not treat overlapping windows as independent, does not rely on chi-square calibration, and conditions each context only on its own count, so nested candidate contexts are covered. Implementation checks on three analytically solvable environments—each with $`\varepsilon`$ set below its known gap—recover the exact quotients on all of 20 seeds, including an environment whose quotient requires merging; a budget sweep shows the power gate degrading through abstention with no incorrect merge at any budget; and a position-nonstationary chain shows the overlapping-window mode systematically merging contexts the episode-position quotient separates.

Authors

Publication Details

Published
2026-11-14
DOI
https://doi.org/10.68852/pf-261114.000001
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Finite-Sample Recovery of Predictive History Quotients from Independent POMDP Trajectories

Nissan Pow
Reinforcement Learning in Robotics
article

Finite-Sample Recovery of Predictive History Quotients from Independent POMDP Trajectories

Nissan Pow
article en

Abstract

We study a modest but identifiable abstraction problem for partially observed control: under one fixed data-collection policy, which bounded observation contexts have the same finite-horizon continuation law? We define the *episode-position predictive quotient*, an explicitly policy- and position-dependent estimand. Our estimator first selects contexts on one half of independently reset trajectories, then takes at most one prespecified continuation block from each trajectory in the other half. It compares fixed- alphabet histograms by empirical total variation and abstains whenever a pair lacks the samples required by its own concentration bound. Under an $`\varepsilon`$-gap, a per-context visitation floor, and a selection threshold no larger than the inference threshold, we prove exact recovery with probability $`1-\delta`$ from $`O((d+\log(K/\delta))/(\mu_{\min}\varepsilon^2))`$ trajectories, where $`d`$ is the continuation-alphabet size and $`K`$ the selected context count. A two-point construction shows the dependence on $`\mu_{\min}`$ and $`\varepsilon`$ is unavoidable; the dependence on $`d`$ is not matched and is known to be improvable by optimal closeness testers. The guarantee is non-asymptotic: it does not treat overlapping windows as independent, does not rely on chi-square calibration, and conditions each context only on its own count, so nested candidate contexts are covered. Implementation checks on three analytically solvable environments—each with $`\varepsilon`$ set below its known gap—recover the exact quotients on all of 20 seeds, including an environment whose quotient requires merging; a budget sweep shows the power gate degrading through abstention with no incorrect merge at any budget; and a position-nonstationary chain shows the overlapping-window mode systematically merging contexts the episode-position quotient separates.

Vol. 1
Fundação para a Ciência e a Tecnologia
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon