Home > Technology peripherals > AI > body text

13 probability distributions that must be mastered in deep learning

WBOY
Release: 2023-04-11 21:58:12
forward
1610 people have browsed it

13 probability distributions that must be mastered in deep learning

1. Overview of probability distribution

13 probability distributions that must be mastered in deep learning

  • Conjugate means that it has the relationship of a conjugate distribution.
  • In Bayesian probability theory, if the posterior distribution p(θx) and the prior probability distribution p(θ) are in the same probability distribution family, the prior and posterior are called conjugate distributions, The prior is called the conjugate prior of the likelihood function. Conjugate prior Wikipedia is here (https://en.wikipedia.org/wiki/Conjugate_prior).
  • Multiple classification means that the random variance is greater than 2.
  • n times means that we also consider the prior probability p(x).
  • To learn more about probability, I recommend reading [pattern recognition and machine learning, Bishop 2006].

2. Distribution probability and characteristics

1. Uniform distribution (continuous)

Code: https://github.com/graykode/distribution-is- all-you-need/blob/master/uniform.py

The uniform distribution has the same probability value on [a, b] and is a simple probability distribution.

13 probability distributions that must be mastered in deep learning

2. Bernoulli distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you- need/blob/master/bernoulli.py

  • The prior probability p(x) does not consider the Bernoulli distribution. Therefore, if we optimize for maximum likelihood, we can easily become overfitted.
  • Use binary cross-entropy to classify binomial classification. Its form is the same as the negative logarithm of the Bernoulli distribution. Picture

3. Binomial distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you-need/blob/master/ binomial.py

  • The binomial distribution with parameters n and p is a discrete probability distribution of the number of successes in a series of n independent experiments.
  • The binomial distribution is a distribution that takes into account the prior probability by specifying the quantity to be picked in advance.

13 probability distributions that must be mastered in deep learning

4. Multi-Bernoulli distribution, categorical distribution (discrete)

Code: https://github.com/graykode/distribution -is-all-you-need/blob/master/categorical.py

  • Multiple Bernoulli is called a categorical distribution.
  • Cross entropy has the same form as the multi-Bernoulli distribution taking the negative logarithm.

13 probability distributions that must be mastered in deep learning

5. Polynomial distribution (discrete)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/multinomial.py

The relationship between polynomial distribution and categorical distribution is the same as the relationship between Bernoul distribution and binomial distribution.

13 probability distributions that must be mastered in deep learning

6.β distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need/ blob/master/beta.py

  • The beta distribution is conjugate to the binomial distribution and the Bernoulli distribution.
  • Using conjugation, the posterior distribution can be more easily obtained by using the known prior distribution.
  • When the β distribution satisfies the special case (α=1, β=1), the uniform distribution is the same.

13 probability distributions that must be mastered in deep learning

7.Dirichlet distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/dirichlet.py

  • dirichlet distribution and polynomial distribution are conjugate.
  • If k=2, it is beta distribution.

13 probability distributions that must be mastered in deep learning

8. Gamma distribution (continuous)

Code: https://github.com/graykode/distribution-is-all- you-need/blob/master/gamma.py

  • If gamma(a,1)/gamma(a,1) gamma(b,1) is the same as beta(a,b), then The gamma distribution is the beta distribution.
  • The exponential distribution and the chi-square distribution are special cases of the gamma distribution.

13 probability distributions that must be mastered in deep learning

9. Exponential distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you -need/blob/master/exponential.py

The exponential distribution is a special case of the γ distribution when α is 1.

13 probability distributions that must be mastered in deep learning

10. Gaussian distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need/ blob/master/gaussian.py

The Gaussian distribution is a very common continuous probability distribution.

13 probability distributions that must be mastered in deep learning

11. Normal distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need /blob/master/normal.py

The normal distribution is the standard Gaussian distribution, with a mean of 0 and a standard deviation of 1.

13 probability distributions that must be mastered in deep learning

12. Chi-square distribution (continuous)

Code: https://github.com/graykode/distribution-is-all-you-need /blob/master/chi-squared.py

  • The chi-square distribution with k degrees of freedom is the distribution of the sum of the squares of k independent standard normal random variables.
  • Chi-square distribution is a special case of beta distribution

13 probability distributions that must be mastered in deep learning

13.t distribution (continuous)

Code: https:/ /github.com/graykode/distribution-is-all-you-need/blob/master/student-t.py

t The distribution is a symmetrical bell-shaped distribution, similar to the normal distribution, but with a longer tail Heavy, which means it's more likely to produce values ​​well below average.

13 probability distributions that must be mastered in deep learning

via:https://github.com/graykode/distribution-is-all-you-needa

The above is the detailed content of 13 probability distributions that must be mastered in deep learning. For more information, please follow other related articles on the PHP Chinese website!

source:51cto.com
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template