My Weekly Reading Progress Report

All members of the exponential family have conjugate priors
if the posterior distributions p(θ | x) are in the same probability distribution family as the prior probability distribution p(θ), the prior and posterior are then called conjugate distributions, and the prior is called a conjugate prior for the likelihood function p(x | θ)

o
another distribution q(w|D) with some variational parameters θ.
P(w|D) (our posterior from above)
P(x) is the evidence
P(x|θ) which is the likelihood
P(θ) which is our prior
take a variational approximation rather than a Monte Carlo scheme to find the approximate Bayesian posterior distribution
only point estimates of the weights are achieved in the network. As a result, these networks make overconfident predictions and do not account for uncertainty in the parameters
Aleatoric uncertainty, on the other hand, is modelled by placing a distribution over the output of the model.
Epistemic uncertainty is modelled by placing a prior distribution over a model’s weights and then trying to capture how much these weights vary given some data
heteroscedastic uncertainty which depends on the inputs to the model
homoscedastic uncertainty, the uncertainty which stays constant for different inputs
Epistemic uncertainty represents the uncertainty caused by the model itself
Aleatoric uncertainty measures the noise inherent in the observations
A prior probability distribution is defined before observing the data, the learning happens and the distribution transforms into posterior distributions once the data is observed