An investigation to improve a nonlinear mixed-effects approach for EC50 estimation based on multi-donor dose–response data

Dose-response relationships are important in assessing the efficacy and potency of compounds, which can usually be characterized by a 4-parameter logistic (4-PL) model estimating EC50, slope factor, lower asymptote, and upper asymptote. EC50, the concentration of a compound that induces a response halfway between the baseline and maximum, is a key quantity to evaluate compound potency. For multi-donor dose-response data, it is often of interest to estimate the overall EC50 (i.e. the average EC50 of the population of donors) and its 95% confidence interval (CI). A few multi-donor EC50 estimation methods have been proposed in the literature. Jiang and Kopp-Schneider (2014) systematically compared the meta-analysis approach and the nonlinear mixed-effects approach and concluded that the meta-analysis approach is simple and robust to summarize EC50 estimates from multiple experiments, especially suited in the case of a small number of experiments, while the nonlinear mixed-effects approach has the issue of convergence failures probably due to overparameterization. In this article, we propose a modification of the nonlinear mixed-effects approach by using the stochastic approximation expectation-maximization (SAEM) algorithm to estimate model parameters and using multiple starting points to search for globally optimal values, which can substantially alleviate the issue of convergence failures even for small number of donors (e.g. <i>n</i> = 3), and achieve a smaller absolute median bias and better coverage probability of 95% confidence interval than the meta-analysis approach when the number of donors is not too small (e.g. <i>n</i> ≥ 7).