Tuning Social Indexes on Training-Data

Abstract To help a (perhaps artificial) policy decider compare the performance of its (perhaps artificial) social agents, a type of abstract evaluation machine is presented that could be used in various concrete operative situations. Designed on a sequence of progressively more challenging objectives on the chain $$O_{0},O_{1},...,O_{l}\ldots .,O_{L}$$ , this machine, applied to any social agent A belonging to a given class, evaluates A ’s performance taking into account the state x , provided that the gain scores $$\omega _{l|x}:=Val_{|x}(O_{l})-Val_{|x}(O_{l-1})$$ which regulate the machine have been assigned. A criterion is proposed that interprets gain scores in a probabilistic context, by invoking an intuitive principle of “intrinsic worthiness”. The specific question of interest concerns a structured, possibly automatable, approach to extract meaningful standardized gain scores $$\omega _{l|x}$$ , from the “reference training-data set” associated to the choice of reference agent $$A^{*}$$ (eg a “ good practice”), conditional also on assumptions on modeling. An advanced (pseudo-Bayesian) approach is outlined in a Dirichlet-multinomial framework. A subspace $$\mathcal {\mathcal {M}}_{0}$$ of Dirichlet distributions (viewed as a geometric submanifold) is modeled by assuming that hidden processes, related to progress along the goal chain, are driven by hidden transition probabilities dependent on state x, governed by a set of hyper-hyperparameters. Applying an orthogonality principle, it is searched for that specific configuration of the hyperparameters set such that a certain (pseudo-Bayesian) vector-operator of updating, given the current training data set, is orthogonal to the submanifold $$\mathcal {\mathcal {M}}_{0}$$ ; thus tuning hyper-hyperparameters on the training-data set. Then, hidden gains scores could be backward recovered which enter evaluation machine