Bayesian Neural Network for Nonparametric Regression

With neural networks, the main difficult in improving the model generalization capability is controlling the complexity of the model. This paper investigates a Bayesian neural network learning for nonparametric regression. Prior knowledge about the model parameters can be incorporated within Bayesian inference and combined with training data to control complexity of different parts of the model. A Markov chain Monte Carlo algorithm is used to optimize model control parameters and obtain the predictive distribution. We show that the complexity of the models adapts to the complexity of the data and produces good results on five noisy test functions in two dimension. The performance and advantage of this approach are compared with conventional neural network methods.