These notes give an introduction to the theory of reproducing kernel Hilbert spaces. We take a somewhat algebraic view of some of his results. In later chapters we consider generalizations of this theory to the
vector-valued setting. We will use the paper of V. I. Paulson in this short note of reproducing kernel Hilbert spaces. We have tried our best to translate it in simple words so that the beginner understand some of the facts in simple language.

Reproducing kernel Hilbert spaces arise in a number of areas, including
approximation theory, statistics, machine learning theory, group representation theory, operator theory and operator algebra and various areas of complex analysis.

We will consider Hilbert spaces over either the field of real numbers, R,
or of complex numbers, C. We will use F to denote either R or C, so that
when we wish to state a definition or result that is true for either the real
or complex numbers, we will use F.
Given a set X, if we equip the set of all functions from X to F
with the usual operations of addition, (f + g)(x) = f(x) + g(x), and scalar
multiplication, (λ · f)(x) = λ · (f(x)), then it is a vector space over F.

Definition

Definition 2.1. Given a set X, we will say that H is a reproducing
kernel Hilbert space(RKHS) on X over F, provided that:
(i) H is a vector vector space,
(ii) H is endowed with an inner product, h, i, making it into a Hilbert
space,
(ii) for every y ∈ X, the linear evaluation functional, E : H → F,
defined by E(f) = f(y), is bounded.

Remark

In sequel, we will use k for K(x, y) where x and y are in X.

Definition

Let H be a reproducing kernel Hilbert space on X. We say
that H separates points provided that for x is not equal to y there exists f ∈ H such that f(x) is not equal to f(y).

We have the following very important problem which can be solved by using above definition.

The following problem shows that if H is a reproducing kernel Hilbert space(RKHS) and if there is a closed subspace of H then the closed subspace of H is also a reproducing kernel Hilbert space by using the orthogonal projection from H onto that closed subspace.

An Example of Reproducing Kernel Hilbert Space

Hardy-Hilbert space of the unit disc H²(D) is an example of reproducing kernel Hilbert space. The norm on H²(D) is given by

where a_n are the Taylor coefficients of f.

The map L : H2(D) →l² (N) defined by L(f) = (a_0, a_1,a_2 . . .) is a linear inner product preserving
isomorphism and hence we see that H² (D) can be identified with the Hilbert
space, l² (N), where N= {0, 1, 2, . . .} the natural numbers with 0, and hence is itself a Hilbert space. Thus, we see that (ii) of the
above definition is met.

Next we show that every power series in H2(D), converges to define a
function on the disk. To see this note that if z ∈ D, then

Thus the evaluation functional E_z is bounded.

Thus, each power series defines a function on D and the vector space operations on formal power series, clearly agrees with their vector space operations as functions on D, and so (i) is met. Hence all the three condition of the above definition satisfies and hence H²(D) is reproducing Kernel Hilbert space.

Problem:

Sobolev space on [0,1] is reproducing kernel Hilbert space.

Complexification of Reproducing Kernel Hilbert spaces

Let H be a RKHS of real-valued functions on the set X with reproducing
kernel, k. Let W = {f_1 + i f_2 : f_1, f_2 ∈ H}, which is a vector space of
complex-valued functions on X. If we set,

then it is easily checked that this defines an inner product on W, with
corresponding norm,

Hence, W is a Hilbert space and since

we have that W equipped with this inner product is a RKHS of complex valued functions on X with reproducing kernel, k.

We call W the complexification of H. Since every real-valued RKHS
can be complexified in a way that still preserves the reproducing kernel, we
shall from this point on, only consider the case of complex-valued reproducing kernel Hilbert spaces.

Let X be any set and let H be a RKHS on X with kernel K. In this section
we will show that “k” completely determines the space H and characterize
the functions that are the kernel functions of some RKHS.

The following Proposition shows that the set of reproducing kernels is dense in H.

Problem

Generalize the above scalar version to the vector valued version.