AI Insight
This paper introduces Flow-Transformed Implicit Processes (FTIP), a new method for Bayesian inference over functions that uses normalizing flows to create more flexible posterior distributions. Unlike existing approaches that use restrictive Gaussian distributions over combination weights when approximating function priors, FTIP can capture asymmetric, heavy-tailed, and multimodal posterior shapes. Experiments demonstrate that FTIP better represents complex posterior uncertainty structures in function space compared to standard Gaussian approximations.
Why it matters
This advancement could improve uncertainty quantification in machine learning applications where capturing the full range of possible outcomes is critical, such as medical diagnosis, climate modeling, and risk assessment. The method's ability to represent complex posterior distributions may lead to more reliable predictions in domains where assumptions of symmetric, unimodal uncertainty are unrealistic.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box {alpha} objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.
Source: Flow-Transformed Implicit Processes for Function-Space Variational Inference