Kernel density estimation

In statistics, kernel density estimation (KDE) is a non-parametric way to estimate the probability density function of a random variable. Kernel density estimation is a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form.[1][2] One of the famous applications of kernel density estimation is in estimating the class-conditional marginal densities of data when using a naive Bayes classifier,[3][4] which can improve its prediction accuracy.[3]

. . . Kernel density estimation . . .

Let (x₁, x₂, …, x_n) be independent and identically distributed samples drawn from some univariate distribution with an unknown densityƒ at any given point x. We are interested in estimating the shape of this function ƒ. Its kernel density estimator is

${displaystyle {widehat {f}}_{h}(x)={frac {1}{n}}sum _{i=1}^{n}K_{h}(x-x_{i})={frac {1}{nh}}sum _{i=1}^{n}K{Big (}{frac {x-x_{i}}{h}}{Big )},}$

where K is the kernel — a non-negative function — and h > 0 is a smoothing parameter called the bandwidth. A kernel with subscript h is called the scaled kernel and defined as K_h(x) = 1/h K(x/h). Intuitively one wants to choose h as small as the data will allow; however, there is always a trade-off between the bias of the estimator and its variance. The choice of bandwidth is discussed in more detail below.

A range of kernel functions are commonly used: uniform, triangular, biweight, triweight, Epanechnikov, normal, and others. The Epanechnikov kernel is optimal in a mean square error sense,[5] though the loss of efficiency is small for the kernels listed previously.[6] Due to its convenient mathematical properties, the normal kernel is often used, which means K(x) = ϕ(x), where ϕ is the standard normal density function.

The construction of a kernel density estimate finds interpretations in fields outside of density estimation.[7] For example, in thermodynamics, this is equivalent to the amount of heat generated when heat kernels (the fundamental solution to the heat equation) are placed at each data point locations x_i. Similar methods are used to construct discrete Laplace operators on point clouds for manifold learning (e.g. diffusion map).

. . . Kernel density estimation . . .

. . . Kernel density estimation . . .