Scalable Gaussian Process Classification via Expectation Propagation

Posted: 2015-07-17 in paper of the day, research
Tags: , ,

Daniel Hernandez-Lobato and Jose Miguel Hernandez-Lobato:
Scalable Gaussian Process Classification via Expectation Propagation
arXiv

The paper is about large-scale Gaussian process classification. Unlike many others, the authors use Expectation Propagation (and not Variational Inference) for approximate inference. An approximate marginal likelihood expression is derived that factorizes over the data instances, which allows for distributed inference and training. Training is additionally sped up by using mini-batches of data.

An interesting part of the paper is that the approximate factors (EP) and the GP’s hyper-parameters are trained jointly: After one parallel update of the factors, a gradient step is taken to update the hyper-parameters. Although EP has not converged and the moment-approximation is incorrect, the authors find that the required correction terms are negligible and omit them therefore. This makes the entire procedure really fast.

Really strong experimental results in terms of scaling to large data sets (\gg 10^6) are still missing, but the approach seems to have quite a bit of potential.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s