Hybrid method benchmark dose estimation with a heterogeneous variance structure

For several decades, the benchmark dose (BMD) methodology has been recommended for estimating safe exposure levels to toxic substances. When the response variable is continuous, the BMD and its lower confidence limit (BMDL) are often estimated using the hybrid method, which assumes a normal distribution to model the probability of an adverse response. Typically, this approach relies on a dose-response model with the assumption of constant standard deviation across all doses. However, when this assumption is violated, it can lead to biased estimates, and current implementations of the hybrid method do not account for this. In this paper, we introduce an extended class of dose-response models that allows for variation in the standard deviation across doses and adapt the hybrid method accordingly. We illustrate the proposed method using two data sets with two types of heteroscedasticity and show, through simulation, that addressing variance heterogeneity reduces bias and results in BMDL estimates with coverage closer to the nominal level.