Sequential experimental design and response optimisation

Overview & Synopsis

We consider the situation where one wants to maximise a function f(theta,x) with respect to x, with theta unknown and estimated from observations y_k. This may correspond to the case of a regression model, where one observes y_k = f( theta,x_k) + epsilon _k, with epsilon_ k some random error, or to the Bernoulli case where y_k in {0,1}, with Pr[y_k = 1|theta, x_k] = f(theta,x_k). Special attention is given to sequences given by x_k+1 = arg max_x f(theta^ k,x) + alpha_ k d_k(x), with theta^ k an estimated value of theta obtained from (x_1, y_1), ... , (x_k, y_k) and d_k(x) a penalty for poor estimation. Approximately optimal rules are suggested in the linear regression case with a fi nite horizon, where one wants to maximize sum_{i=1}^N w_i f(theta,x_i) with {w_i} a weighting sequence. Various examples are presented, with a comparison with a Polya urn design and an up-and-down method for a binary response problem.