log-likelihood function generated by scipy.stats.rv_continuous.fit

Question

The method scipy.stats.rv_continuous.fit finds the parameters that maximise a log likelihood function which is determined by the input data and the specification of the distribution rv_continuous. For instance, this could be normal or gamma.

The documentation for scipy.stats.rv_continuous.fit doesn't explain how the log-likelihood function is generated and I would like to know how. I need it so I can calculate the value of the log-likelihood at the parameters estimated by fit (i.e. the maximum value).

Answer 1

Log-likelihood is the logarithm of the probability that a given set of observations is observed given a probability distribution. You can access the value of a probability density function at a point x for your scipy.stats.rv_continuous member using scipy.stats.rv_continuous.pdf(x,params). You would take the product of these values for each member of your data, then take the log of that. For instance:

import numpy as np
from scipy.stats import norm

data = [1,2,3,4,5]
m,s = norm.fit(data)
log_likelihood = np.log(np.product(norm.pdf(data,m,s)))

Answer 2

While Mark's answer is technically correct, I can only re-iterate nikosd's concern from the comment - taking the product first and then logging it, will in many practical scenarios make your results unusable. If have many (thousands/millions) observations in data, each has a probability of <=1, so your product np.product(norm.pdf(data,m,s)) will be very small and often smaller than numerical precision, making your results unstable/wrong.

Thus the better way - and the reason why log-likelihood is used in the first place - is to first log the individual probabilities np.log(norm.pdf(data,m,s)) and then sum over the resulting vector.

import numpy as np
from scipy.stats import norm

data = [1,2,3,4,5]
m,s = norm.fit(data)
log_likelihood = np.sum(np.log(norm.pdf(data,m,s)))

I thought this was important enough to warrant a separate answer.