Efficient learning from disjunction agreement pairs: an algorithm and proof for the formulation in Blum–Balcan (2007)

We give a polynomial-time algorithm and proof for the problem "Co-training with disjunctions" from A. Blum and M.-F. Balcan, "Open Problems in Efficient Semi-Supervised PAC Learning" (COLT 2007), Section 3.2. For an arbitrary joint distribution D on pairs (x1, x2), with no independence assumption between views, and a target disjunction that is perfectly compatible (h*(x1) = h*(x2) almost surely), an exclusion-closure (Horn-closure) algorithm computes the greatest disjunction consistent with the labelled examples and with exact agreement on the unlabelled pairs. The resulting learner uses O((ln|C_{D,chi}(tau)| + ln(1/delta))/epsilon) labelled pairs and O((n + ln(1/delta))/tau) unlabelled pairs, and runs in time linear in the input size. Negated literals are handled by feature lifting. A fully specified stopping rule removes the need to know the size of the compatible class. The record contains the paper (PDF, LaTeX source) and the full repository archive: a Python reference implementation, verification scripts, and their results. This note does not claim that the argument is new or that the problem remained open; those questions are left to the authors of the original note. Contact: [email protected]