API
GPLikelihoods.BernoulliLikelihoodGPLikelihoods.BijectiveSimplexLinkGPLikelihoods.CategoricalLikelihoodGPLikelihoods.ChainLinkGPLikelihoods.ExpLinkGPLikelihoods.ExponentialLikelihoodGPLikelihoods.GammaLikelihoodGPLikelihoods.GaussianLikelihoodGPLikelihoods.HeteroscedasticGaussianLikelihoodGPLikelihoods.InvLinkGPLikelihoods.LinkGPLikelihoods.LogLinkGPLikelihoods.LogisticLinkGPLikelihoods.LogitLinkGPLikelihoods.NBParamFailureGPLikelihoods.NBParamIGPLikelihoods.NBParamIIGPLikelihoods.NBParamPowerGPLikelihoods.NBParamSuccessGPLikelihoods.NegativeBinomialLikelihoodGPLikelihoods.NormalCDFLinkGPLikelihoods.PoissonLikelihoodGPLikelihoods.ProbitLinkGPLikelihoods.SoftMaxLinkGPLikelihoods.SqrtLinkGPLikelihoods.SquareLinkGPLikelihoods.expected_loglikelihood
Likelihoods
GPLikelihoods.BernoulliLikelihood — TypeBernoulliLikelihood(l=logistic)Bernoulli likelihood is to be used if we assume that the uncertainity associated with the data follows a Bernoulli distribution. The link l needs to transform the input f to the domain [0, 1]
\[ p(y|f) = \operatorname{Bernoulli}(y | l(f))\]
On calling, this would return a Bernoulli distribution with l(f) probability of true.
GPLikelihoods.CategoricalLikelihood — TypeCategoricalLikelihood(l=BijectiveSimplexLink(softmax))Categorical likelihood is to be used if we assume that the uncertainty associated with the data follows a Categorical distribution.
Assuming a distribution with n categories:
n-1 inputs (bijective link)
One can work with a bijective transformation by wrapping a link (like softmax) into a BijectiveSimplexLink and only needs n-1 inputs:
\[ p(y|f_1, f_2, \dots, f_{n-1}) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_{n-1}, 0))\]
The default constructor is a bijective link around softmax.
n inputs (non-bijective link)
One can also pass directly the inputs without concatenating a 0:
\[ p(y|f_1, f_2, \dots, f_n) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_n))\]
This variant is over-parametrized, as there are n-1 independent parameters embedded in a n dimensional parameter space. For more details, see the end of the section of this Wikipedia link where it corresponds to Variant 1 and 2.
GPLikelihoods.ExponentialLikelihood — TypeExponentialLikelihood(l=exp)Exponential likelihood with scale given by l(f).
\[ p(y|f) = \operatorname{Exponential}(y | l(f))\]
GPLikelihoods.GammaLikelihood — TypeGammaLikelihood(α::Real=1.0, l=exp)Gamma likelihood with fixed shape α.
\[ p(y|f) = \operatorname{Gamma}(y | α, l(f))\]
On calling, this returns a Gamma distribution with shape α and scale invlink(f).
GPLikelihoods.GaussianLikelihood — TypeGaussianLikelihood(σ²)Gaussian likelihood with σ² variance. This is to be used if we assume that the uncertainity associated with the data follows a Gaussian distribution.
\[ p(y|f) = \operatorname{Normal}(y | f, σ²)\]
On calling, this would return a normal distribution with mean f and variance σ².
GPLikelihoods.HeteroscedasticGaussianLikelihood — TypeHeteroscedasticGaussianLikelihood(l=exp)Heteroscedastic Gaussian likelihood. This is a Gaussian likelihood whose mean and variance are functions of latent processes.
\[ p(y|[f, g]) = \operatorname{Normal}(y | f, sqrt(l(g)))\]
On calling, this would return a normal distribution with mean f and variance l(g). Where l is link going from R to R^+
GPLikelihoods.PoissonLikelihood — TypePoissonLikelihood(l=exp)Poisson likelihood with rate defined as l(f).
\[ p(y|f) = \operatorname{Poisson}(y | θ=l(f))\]
This is to be used if we assume that the uncertainity associated with the data follows a Poisson distribution.
Negative Binomial
GPLikelihoods.NegativeBinomialLikelihood — TypeNegativeBinomialLikelihood(param::NBParam, invlink::Union{Function,Link})There are many possible parametrizations for the Negative Binomial likelihood. We follow the convention laid out in p.137 of [^Hilbe'11] and provide some common parametrizations. The NegativeBinomialLikelihood has a special structure; the first type parameter NBParam defines in what parametrization the latent function is used, and contains the other (scalar) parameters. NBParam itself has two subtypes:
NBParamProbfor parametrizations wheref -> p, the probability of success of a Bernoulli eventNBParamMeanfor parametrizations wheref -> μ, the expected number of events
NBParam predefined types
NBParamProb types with p = invlink(f) the probability of success or failure
NBParamSuccess: Herep = invlink(f)is the probability of success. This is the definition used inDistributions.jl.NBParamFailure: Herep = invlink(f)is the probability of a failure
NBParamMean types with μ = invlink(f) the mean/expected number of events
NBParamI: Mean is linked tofand variance is given byμ(1 + α)NBParamII: Mean is linked tofand variance is given byμ(1 + α * μ)NBParamPower: Mean is linked tofand variance is given byμ(1 + α * μ^ρ)
To create a new parametrization, you need to:
- create a new type
struct MyNBParam{T} <: NBParam; myparams::T; end; - dispatch
(l::NegativeBinomialLikelihood{<:MyNBParam})(f::Real), which must return aNegativeBinomialfromDistributions.jl.
NegativeBinomial follows the parametrization of NBParamSuccess, i.e. the first argument is the number of successes and the second argument is the probability of success.
Examples
julia> NegativeBinomialLikelihood(NBParamSuccess(10), logistic)(2.0)
+API · GPLikelihoods.jl API
GPLikelihoods.BernoulliLikelihoodGPLikelihoods.BijectiveSimplexLinkGPLikelihoods.CategoricalLikelihoodGPLikelihoods.ChainLinkGPLikelihoods.ExpLinkGPLikelihoods.ExponentialLikelihoodGPLikelihoods.GammaLikelihoodGPLikelihoods.GaussianLikelihoodGPLikelihoods.HeteroscedasticGaussianLikelihoodGPLikelihoods.InvLinkGPLikelihoods.LinkGPLikelihoods.LogLinkGPLikelihoods.LogisticLinkGPLikelihoods.LogitLinkGPLikelihoods.NBParamFailureGPLikelihoods.NBParamIGPLikelihoods.NBParamIIGPLikelihoods.NBParamPowerGPLikelihoods.NBParamSuccessGPLikelihoods.NegativeBinomialLikelihoodGPLikelihoods.NormalCDFLinkGPLikelihoods.PoissonLikelihoodGPLikelihoods.ProbitLinkGPLikelihoods.SoftMaxLinkGPLikelihoods.SqrtLinkGPLikelihoods.SquareLinkGPLikelihoods.expected_loglikelihood
Likelihoods
GPLikelihoods.BernoulliLikelihood — TypeBernoulliLikelihood(l=logistic)
Bernoulli likelihood is to be used if we assume that the uncertainity associated with the data follows a Bernoulli distribution. The link l needs to transform the input f to the domain [0, 1]
\[ p(y|f) = \operatorname{Bernoulli}(y | l(f))\]
On calling, this would return a Bernoulli distribution with l(f) probability of true.
sourceGPLikelihoods.CategoricalLikelihood — TypeCategoricalLikelihood(l=BijectiveSimplexLink(softmax))
Categorical likelihood is to be used if we assume that the uncertainty associated with the data follows a Categorical distribution.
Assuming a distribution with n categories:
n-1 inputs (bijective link)
One can work with a bijective transformation by wrapping a link (like softmax) into a BijectiveSimplexLink and only needs n-1 inputs:
\[ p(y|f_1, f_2, \dots, f_{n-1}) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_{n-1}, 0))\]
The default constructor is a bijective link around softmax.
n inputs (non-bijective link)
One can also pass directly the inputs without concatenating a 0:
\[ p(y|f_1, f_2, \dots, f_n) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_n))\]
This variant is over-parametrized, as there are n-1 independent parameters embedded in a n dimensional parameter space. For more details, see the end of the section of this Wikipedia link where it corresponds to Variant 1 and 2.
sourceGPLikelihoods.ExponentialLikelihood — TypeExponentialLikelihood(l=exp)
Exponential likelihood with scale given by l(f).
\[ p(y|f) = \operatorname{Exponential}(y | l(f))\]
sourceGPLikelihoods.GammaLikelihood — TypeGammaLikelihood(α::Real=1.0, l=exp)
Gamma likelihood with fixed shape α.
\[ p(y|f) = \operatorname{Gamma}(y | α, l(f))\]
On calling, this returns a Gamma distribution with shape α and scale invlink(f).
sourceGPLikelihoods.GaussianLikelihood — TypeGaussianLikelihood(σ²)
Gaussian likelihood with σ² variance. This is to be used if we assume that the uncertainity associated with the data follows a Gaussian distribution.
\[ p(y|f) = \operatorname{Normal}(y | f, σ²)\]
On calling, this would return a normal distribution with mean f and variance σ².
sourceGPLikelihoods.HeteroscedasticGaussianLikelihood — TypeHeteroscedasticGaussianLikelihood(l=exp)
Heteroscedastic Gaussian likelihood. This is a Gaussian likelihood whose mean and variance are functions of latent processes.
\[ p(y|[f, g]) = \operatorname{Normal}(y | f, sqrt(l(g)))\]
On calling, this would return a normal distribution with mean f and variance l(g). Where l is link going from R to R^+
sourceGPLikelihoods.PoissonLikelihood — TypePoissonLikelihood(l=exp)
Poisson likelihood with rate defined as l(f).
\[ p(y|f) = \operatorname{Poisson}(y | θ=l(f))\]
This is to be used if we assume that the uncertainity associated with the data follows a Poisson distribution.
sourceNegative Binomial
GPLikelihoods.NegativeBinomialLikelihood — TypeNegativeBinomialLikelihood(param::NBParam, invlink::Union{Function,Link})
There are many possible parametrizations for the Negative Binomial likelihood. We follow the convention laid out in p.137 of [^Hilbe'11] and provide some common parametrizations. The NegativeBinomialLikelihood has a special structure; the first type parameter NBParam defines in what parametrization the latent function is used, and contains the other (scalar) parameters. NBParam itself has two subtypes:
NBParamProb for parametrizations where f -> p, the probability of success of a Bernoulli eventNBParamMean for parametrizations where f -> μ, the expected number of events
NBParam predefined types
NBParamProb types with p = invlink(f) the probability of success or failure
NBParamSuccess: Here p = invlink(f) is the probability of success. This is the definition used in Distributions.jl.NBParamFailure: Here p = invlink(f) is the probability of a failure
NBParamMean types with μ = invlink(f) the mean/expected number of events
NBParamI: Mean is linked to f and variance is given by μ(1 + α)NBParamII: Mean is linked to f and variance is given by μ(1 + α * μ)NBParamPower: Mean is linked to f and variance is given by μ(1 + α * μ^ρ)
To create a new parametrization, you need to:
- create a new type
struct MyNBParam{T} <: NBParam; myparams::T; end; - dispatch
(l::NegativeBinomialLikelihood{<:MyNBParam})(f::Real), which must return a NegativeBinomial from Distributions.jl.
NegativeBinomial follows the parametrization of NBParamSuccess, i.e. the first argument is the number of successes and the second argument is the probability of success.
Examples
julia> NegativeBinomialLikelihood(NBParamSuccess(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.8807970779778824)
julia> NegativeBinomialLikelihood(NBParamFailure(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.11920292202211757)
@@ -9,9 +9,9 @@
julia> mean(d) ≈ exp(2.0)
true
julia> var(d) ≈ exp(2.0) * (1 + 3.0)
-true
[^Hilbe'11]: Hilbe, Joseph M. Negative binomial regression. Cambridge University Press, 2011.
sourceGPLikelihoods.NBParamSuccess — TypeNBParamSuccess(successes)
Negative Binomial parametrization with successes the number of successes and invlink(f) the probability of success. This corresponds to the definition used by Distributions.jl.
\[ p(y|\text{successes}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{successes})}{y! \Gamma(\text{successes})} p^\text{successes} (1 - p)^y\]
sourceGPLikelihoods.NBParamFailure — TypeNBParamFailure(failures)
Negative Binomial parametrization with failures the number of failures and invlink(f) the probability of success. This corresponds to the definition used by Wikipedia.
\[ p(y|\text{failures}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{failures})}{y! \Gamma(\text{failures})} p^y (1 - p)^{\text{failures}}\]
sourceGPLikelihoods.NBParamI — TypeNBParamI(α)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α) where α > 0.
sourceGPLikelihoods.NBParamII — TypeNBParamII(α)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ) where α > 0.
sourceGPLikelihoods.NBParamPower — TypeNBParamPower(α, ρ)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ^ρ) where α > 0 and ρ > 0.
sourceLinks
GPLikelihoods.Link — TypeLink(f)
General construction for a link with a function f.
sourceGPLikelihoods.ChainLink — TypeChainLink(links)
Create a composed chain of different links.
sourceGPLikelihoods.BijectiveSimplexLink — TypeBijectiveSimplexLink(link)
Wrapper to preprocess the inputs by adding a 0 at the end before passing it to the link link. This is a necessary step to work with simplices. For example with the SoftMaxLink, to obtain a n-simplex leading to n+1 categories for the CategoricalLikelihood, one needs to pass n+1 latent GP. However, by wrapping the link into a BijectiveSimplexLink, only n latent are needed.
sourceThe rest of the links ExpLink, LogisticLink, etc., are aliases for the corresponding wrapped functions in a Link. For example ExpLink == Link{typeof(exp)}.
When passing a Link to a likelihood object, this link corresponds to the transformation p=link(f) while, as mentioned in the Constrained parameters section, the statistics literature usually uses the denomination inverse link or mean function for it.
GPLikelihoods.LogLink — TypeLogLink()
log link, f:ℝ⁺->ℝ . Its inverse is the ExpLink.
sourceGPLikelihoods.ExpLink — TypeExpLink()
exp link, f:ℝ->ℝ⁺. Its inverse is the LogLink.
sourceGPLikelihoods.InvLink — TypeInvLink()
inv link, f:ℝ/{0}->ℝ/{0}. It is its own inverse.
sourceGPLikelihoods.SqrtLink — TypeSqrtLink()
sqrt link, f:ℝ⁺∪{0}->ℝ⁺∪{0}. Its inverse is the SquareLink.
sourceGPLikelihoods.SquareLink — TypeSquareLink()
^2 link, f:ℝ->ℝ⁺∪{0}. Its inverse is the SqrtLink.
sourceGPLikelihoods.LogitLink — TypeLogitLink()
log(x/(1-x)) link, f:[0,1]->ℝ. Its inverse is the LogisticLink.
sourceGPLikelihoods.LogisticLink — TypeLogisticLink()
1/(1+exp(-x)) link. f:ℝ->[0,1]. Its inverse is the LogitLink.
sourceGPLikelihoods.ProbitLink — TypeProbitLink()
ϕ⁻¹(y) link, where ϕ⁻¹ is the invcdf of a Normal distribution, f:[0,1]->ℝ. Its inverse is the NormalCDFLink.
sourceGPLikelihoods.NormalCDFLink — TypeNormalCDFLink()
ϕ(y) link, where ϕ is the cdf of a Normal distribution, f:ℝ->[0,1]. Its inverse is the ProbitLink.
sourceGPLikelihoods.SoftMaxLink — TypeSoftMaxLink()
softmax link, i.e f(x)ᵢ = exp(xᵢ)/∑ⱼexp(xⱼ). f:ℝⁿ->Sⁿ⁻¹, where Sⁿ⁻¹ is an (n-1)-simplex It has no defined inverse
sourceExpectations
GPLikelihoods.expected_loglikelihood — Functionexpected_loglikelihood(
+true
[^Hilbe'11]: Hilbe, Joseph M. Negative binomial regression. Cambridge University Press, 2011.
sourceGPLikelihoods.NBParamSuccess — TypeNBParamSuccess(successes)
Negative Binomial parametrization with successes the number of successes and invlink(f) the probability of success. This corresponds to the definition used by Distributions.jl.
\[ p(y|\text{successes}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{successes})}{y! \Gamma(\text{successes})} p^\text{successes} (1 - p)^y\]
sourceGPLikelihoods.NBParamFailure — TypeNBParamFailure(failures)
Negative Binomial parametrization with failures the number of failures and invlink(f) the probability of success. This corresponds to the definition used by Wikipedia.
\[ p(y|\text{failures}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{failures})}{y! \Gamma(\text{failures})} p^y (1 - p)^{\text{failures}}\]
sourceGPLikelihoods.NBParamI — TypeNBParamI(α)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α) where α > 0.
sourceGPLikelihoods.NBParamII — TypeNBParamII(α)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ) where α > 0.
sourceGPLikelihoods.NBParamPower — TypeNBParamPower(α, ρ)
Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ^ρ) where α > 0 and ρ > 0.
sourceLinks
GPLikelihoods.Link — TypeLink(f)
General construction for a link with a function f.
sourceGPLikelihoods.ChainLink — TypeChainLink(links)
Create a composed chain of different links.
sourceGPLikelihoods.BijectiveSimplexLink — TypeBijectiveSimplexLink(link)
Wrapper to preprocess the inputs by adding a 0 at the end before passing it to the link link. This is a necessary step to work with simplices. For example with the SoftMaxLink, to obtain a n-simplex leading to n+1 categories for the CategoricalLikelihood, one needs to pass n+1 latent GP. However, by wrapping the link into a BijectiveSimplexLink, only n latent are needed.
sourceThe rest of the links ExpLink, LogisticLink, etc., are aliases for the corresponding wrapped functions in a Link. For example ExpLink == Link{typeof(exp)}.
When passing a Link to a likelihood object, this link corresponds to the transformation p=link(f) while, as mentioned in the Constrained parameters section, the statistics literature usually uses the denomination inverse link or mean function for it.
GPLikelihoods.LogLink — TypeLogLink()
log link, f:ℝ⁺->ℝ . Its inverse is the ExpLink.
sourceGPLikelihoods.ExpLink — TypeExpLink()
exp link, f:ℝ->ℝ⁺. Its inverse is the LogLink.
sourceGPLikelihoods.InvLink — TypeInvLink()
inv link, f:ℝ/{0}->ℝ/{0}. It is its own inverse.
sourceGPLikelihoods.SqrtLink — TypeSqrtLink()
sqrt link, f:ℝ⁺∪{0}->ℝ⁺∪{0}. Its inverse is the SquareLink.
sourceGPLikelihoods.SquareLink — TypeSquareLink()
^2 link, f:ℝ->ℝ⁺∪{0}. Its inverse is the SqrtLink.
sourceGPLikelihoods.LogitLink — TypeLogitLink()
log(x/(1-x)) link, f:[0,1]->ℝ. Its inverse is the LogisticLink.
sourceGPLikelihoods.LogisticLink — TypeLogisticLink()
1/(1+exp(-x)) link. f:ℝ->[0,1]. Its inverse is the LogitLink.
sourceGPLikelihoods.ProbitLink — TypeProbitLink()
ϕ⁻¹(y) link, where ϕ⁻¹ is the invcdf of a Normal distribution, f:[0,1]->ℝ. Its inverse is the NormalCDFLink.
sourceGPLikelihoods.NormalCDFLink — TypeNormalCDFLink()
ϕ(y) link, where ϕ is the cdf of a Normal distribution, f:ℝ->[0,1]. Its inverse is the ProbitLink.
sourceGPLikelihoods.SoftMaxLink — TypeSoftMaxLink()
softmax link, i.e f(x)ᵢ = exp(xᵢ)/∑ⱼexp(xⱼ). f:ℝⁿ->Sⁿ⁻¹, where Sⁿ⁻¹ is an (n-1)-simplex It has no defined inverse
sourceExpectations
GPLikelihoods.expected_loglikelihood — Functionexpected_loglikelihood(
quadrature,
lik,
q_f::AbstractVector{<:Normal},
y::AbstractVector,
-)
This function computes the expected log likelihood:
\[ ∫ q(f) log p(y | f) df\]
where p(y | f) is the process likelihood. This is described by lik, which should be a callable that takes f as input and returns a Distribution over y that supports loglikelihood(lik(f), y).
q(f) is an approximation to the latent function values f given by:
\[ q(f) = ∫ p(f | u) q(u) du\]
where q(u) is the variational distribution over inducing points. The marginal distributions of q(f) are given by q_f.
quadrature determines which method is used to calculate the expected log likelihood.
Extended help
q(f) is assumed to be an MvNormal distribution and p(y | f) is assumed to have independent marginals such that only the marginals of q(f) are required.
sourceexpected_loglikelihood(::DefaultExpectationMethod, lik, q_f::AbstractVector{<:Normal}, y::AbstractVector)
The expected log likelihood, using the default quadrature method for the given likelihood. (The default quadrature method is defined by default_expectation_method(lik), and should be the closed form solution if it exists, but otherwise defaults to Gauss-Hermite quadrature.)
sourceSettings
This document was generated with Documenter.jl version 0.27.25 on Sunday 10 March 2024. Using Julia version 1.10.2.
This function computes the expected log likelihood:
\[ ∫ q(f) log p(y | f) df\]
where p(y | f) is the process likelihood. This is described by lik, which should be a callable that takes f as input and returns a Distribution over y that supports loglikelihood(lik(f), y).
q(f) is an approximation to the latent function values f given by:
\[ q(f) = ∫ p(f | u) q(u) du\]
where q(u) is the variational distribution over inducing points. The marginal distributions of q(f) are given by q_f.
quadrature determines which method is used to calculate the expected log likelihood.
Extended help
q(f) is assumed to be an MvNormal distribution and p(y | f) is assumed to have independent marginals such that only the marginals of q(f) are required.
expected_loglikelihood(::DefaultExpectationMethod, lik, q_f::AbstractVector{<:Normal}, y::AbstractVector)The expected log likelihood, using the default quadrature method for the given likelihood. (The default quadrature method is defined by default_expectation_method(lik), and should be the closed form solution if it exists, but otherwise defaults to Gauss-Hermite quadrature.)