Functional analysis is a branch of mathematics that deals with the study of vector spaces, particularly those that are infinite-dimensional. It has applications in various fields, including mathematical physics, differential equations, and optimization.
Preliminaries
Before delving into the intricacies of functional analysis, it is essential to lay the groundwork with some key concepts from linear algebra. These preliminaries include, but not limited to:
- Vector spaces: A vector space is a collection of vectors that can be added and multiplied by scalars, satisfying certain algebraic properties.
- Subspaces: A subspace is a subset of a vector space that itself forms a vector space under the same operations.
- Linear transformations: A linear transformation is a function between vector spaces that preserves vector addition and scalar multiplication.
SPACES OF FUNCTIONS, SETS, SEQUENCES
Continuous Functions
A function \(f(x)\) is continuous at a point \(x = a\) if the following three conditions are met:
- The function \(f(x)\) is defined at the point \(a\): \(f(a)\) is defined.
- The limit of \(f(x)\) as \(x\) approaches \(a\) from the left exists: \(\lim_{x\to a-} f(x)\) exists.
- The limit of \(f(x)\) as \(x\) approaches \(a\) from the right exists, and the two limits are equal to \(f(a)\): \(\lim_{x \to a+} f(x)\) exists, and \(\lim_{x \to a-} f(x) = \lim_{x \to a+} f(x) = f(a)\).
Example
Show that the function \(f(x) = \frac{1}{x}\) is continuous as \(x\) approaches positive or negative infinity.
Solution
Is \(f(x)\) defined at positive infinity and negative infinity?
To determine if the function \(f(x) = \frac{1}{x}\) is continuous as \(x\) approaches positive or negative infinity, we need to examine the limit of the function at the point of interest.
Let's look at the limit as \(x\) approaches positive infinity: \(\lim_{x\to\infty} f(x) = \lim_{x\to\infty} \frac{1}{x}\).
To evaluate this limit, we can use the property that the limit of a reciprocal function is the reciprocal of the limit of the function, that is
\(\lim_{x\to\infty} \frac{1}{x} = \frac{1}{ \lim_{x\to\infty} x}\).
We know that as \(x\) approaches positive infinity, the value of \(x\) itself approaches positive infinity,
\(\lim_{x\to\infty} x = +\infty\).
Thus \(\lim_{x\to\infty} \frac{1}{x} = \frac{1}{ \lim_{x\to\infty} x} = 0\).
With similar argument, the limit as \(x\) approaches negative infinity is
\(\lim_{x\to-\infty} \frac{1}{2} = \frac{1}{ \lim_{x\to-\infty} x} = 0\)
We see that the limit of \(f(x) = \frac{1}{x}\) as \(x\) approaches both positive and negative infinity is \(0\).
Is the function continuous as \(x\) approaches positive or negative infinity?
Open Balls
Consider a pair ( \(X, d\)) called metric space, where \(X\) is the set and \(d\) a metric.
If we fix a point \(x\) on a metric space, we can find other points which are of the same distance as \(x\). In the common geometry of planes, this would result to a circle in 2-D space or a sphere in 3-D space.
How do we then generalize this notion of a ball for an abstract metric space?
Definition
An open ball of radius \(\epsilon > 0\) centered at \(x\), \(B_{\epsilon}(x) \) is defined as
\begin{align}
B_{\epsilon}(x) = \{ y\in X | d(x,y) < \epsilon\}.
\end{align}
An open ball is never empty because at least a fixed point \(x\) lies in the set.
Plot the following using GeoGebra or Desmos Calculator:
- \(x^2 + y^2 <a\)
- \( x^2 + y^2 + z^2 -ax -by - cz = 0 \)
Open Sets
Consider the following metric space (set) containing an arbitrary set \(A\) and an arbitrary point \(x\).
Open set \(A\) descriptively means if you are inside the set \(A\), you should never see the boundary of set \(A\).
Definition
\(A \subseteq X\) is called open if for each \(x \in A \) there is an open ball \(B_{\epsilon}(x) \subseteq A\).
Boundary Points
Consider \(A \subseteq X\).
Our open ball contains points that are in \(A\) and others are outside \(A\).
Definition
A point \(x \in X\) is called a boundary point for \(A\) if for all \(\epsilon > 0 \), \(B_{\epsilon} \cap A \neq \emptyset \) and \(B_{\epsilon} \cap A^{c} \neq \emptyset \), where \(A^{c} := X \backslash A \).
In this case, every open ball around \(x\) contains points from \(A\) and points from the compliment of \(A\).
Boundary points, \(\partial A = \{ x \in X| x \ \rm{is \ a \ boundary \ point \ for } \ A \}\).
Therefore:
- \(A\) is open set if all the boundary points, \(\partial A \) are outside it.
- \(A\) is open \( \iff \) \(A \cap \partial A = \emptyset \), that's all boundary points are outside \(A\).
- \(A\) is closed if all boundary points belongs to \(A\). That's, \(A\) is closed \( \iff \) \(A \cup \partial A = A \).
- \(A \subseteq X\) is closed if \(A^{c} := X \backslash A \) is open.
Closure of \(A\), \(\bar{A} := A \cup \partial A \) (always closed).
We will further introduce the concepts of sequences, limits and closed sets, convergence and Cauchy sequences, completeness and boundedness.
THE CONCEPT OF GEOMETRICAL DIMENSIONS IN MATHEMATICS
Dimension is the number of independent coordinates/components required to specify/describe the location of a point or the size/shape of a mathematical object. Dimensions can be geometric, vector space, topological, or fractal, depending on the context.
Geometrical dimension is the concept used to describe the number of spatial coordinate axes needed to specify a point and its direction in a space.
In the context of vector spaces, the dimension refers to the number of distinct linearly independent vectors needed to span the entire vector space.
Basic geometric dimensions in mathematics include:
- 0-Dimensional (0-D) object: A point, which has no measurable length, height, or depth. It is a location in space that has no spatial extent or direction, only a position.
- 1-Dimensional (1-D) object: A line or line segment or a ray, which has length but no other dimensions.
- 2-Dimensional (2-D) object: A plane or polygon, which has length and height but no depth. Examples are circles, planes and polygons.
- 3-Dimensional (3-D) object: Has length, height, and depth. Examples are cubes and spheres.
In physics, time is often regarded as the 4th dimension. But, mathematicians can work with higher-dimensional objects that do not exist in 3 spatial dimensions like the 4-D hypercube or hypersphere regardless of their physical existence in our 3-D universe.
Vectors in 1-D, 2-D, and 3-D Spaces
- 1-D Vectors: Symbolically represented as $(x)$, where $x$ is a scalar representing the magnitude and direction along a single axis. A 1-D vector has one component and represents movement or change in a specific direction in one-dimensional space.
- 2-D Vectors: Symbolically represented as $(x, y)$, where $x$ and $y$ represent the magnitudes and directions along the $x$ and $y$ axes, respectively.
- 3-D Vectors: Symbolically represented as $(x, y, z)$, where $x, y$, and $z$ represent the magnitudes and directions along the \(x, \ y\), and \(z\) axes, respectively.
Example:
- A 1-D vector \((2)\) represents a magnitude of \(2\) along a single axis.
- A 2-D vector \((3, 4)\) represents a magnitude of \(5\) (using the Pythagorean theorem) and a direction at an angle of \(53.13^{\deg}\) from the positive x-axis.
- A 3-D vector \((1, 2, 3)\) represents a magnitude of \(\sqrt{14}\) (using the Pythagorean theorem) and a direction pointing in the \(x\)-axis, \(y\)-axis, and \(z\)-axis directions.
- Points: Both 1-D and 2-D objects can be represented by points, which are 0-D vectors.
- Lines: 1-D objects can be represented by vectors that define their direction and length.
- Planes: 2-D objects can be represented by vectors that form their bounding edges.
- Solids: 3-D objects can be represented by vectors that form their vertices and edges.
Vector Representations
A vector can be represented in several ways, including:
- Geometrically - using an arrow or line segment to represent the vector's magnitude and direction.
- Algebraically (Component form): The vector \(v = (3, -2, 5)\) can be represented as an ordered tuple of its components. This means that the vector has components along the \(x, \ y\), and \(z\) axes respectively. In this case, the \(x\)-component of the vector is \(3\), the \(y\)-component is \(-2\), and the \(z\)-component is \(5\).
- Symbolically - The vector \(v = (a, b, c)\) can be represented using variables like \(a, \ b\), and \(c\) to denote the components. This allows for generalizing the representation of the vector without assigning specific numerical values to the components. The vector \(v\) in this form can represent any vector in three-dimensional space with components represented by variables \(a, \ b\), and \(c\).
- Matrix form - representing the vector as a column matrix: \(v = \begin{bmatrix} 2\\ -1 \\ 3 \end{bmatrix}\)
- Unit vector representation - The vector \(v = (4, -3, 0)\) can be expressed in terms of unit vectors as \(v = 4i - 3j\). Here, \(i, \ j\), and \(k\) are the unit vectors along the \(x,\ y\), and \(z\) axes respectively. This representation breaks down the vector into its components along the different axes, making it easier to understand the direction and magnitude of the vector relative to the coordinate system.
As previously stated, vectors like other objects go beyond the basic 3 dimensions. For example, a \(10\)-dimensional vector has \(10\) components, which cannot be easily visualized, but is useful for representing complex multi-variable systems or data in higher-dimensional spaces.
Basis Vectors
- Basis vectors are a set/family of linearly independent vectors that span a given vector space (an \(n\)-dimensional vector space). That's, any vector in the \(n\)-dimensional space can be expressed as a unique linear combination of the \(n\) basis vectors.
- They form the fundamental building blocks that can be used to represent any vector in that vector space.
- In an \(n\)-dimensional vector space, there are \(n\) basis vectors, denoted as \(e_1, \ e_2, \ e_3, \ ..., \ e_n\).
- The basis vectors are independent of each other and cannot be expressed as a linear combination of the other basis vectors.
In a 3-D Cartesian coordinate system, the standard basis vectors are:
- i = (1, 0, 0)
- j = (0, 1, 0)
- k = (0, 0, 1)
Let's now explain with examples some of the terms that emerged above.
Linear Combination
- Any vector in 3-D space, like \((2, -3, 4)\), can be written as a linear combination of the basis vectors (above):
\((2, \ -3, \ 4) = 2i + (-3)j + 4k\). - The coefficients \(2, -3\), and \(4\) determine how much of each basis vector is used to construct the vector in question.
- Any arbitrary vector \((a, \ b, \ c)\) in 3-D space can be expressed as a linear combination of these three basis vectors.
- This means the basis vectors "span" the entire 3-D vector space - they can be used to represent any possible vector in that space.
In linear algebra, a set of distinct vectors is said to be linearly independent if none of the vectors in the set can be expressed as a linear combination of the other vectors in the set.
Mathematically, a set of distinct vectors \({v_1, \ v_2, \ ..., \ v_n}\) is linearly independent if the equation:
$$c_1v_1 + c_2v_2 + ... + c_nv_n = 0 \ \text{(zero vector)}$$
has only the trivial solution where all coefficients \((c_1, \ c_2, \ ..., \ c_n)\) are zero.
Linearly Dependent Vectors
A set of vectors \({v_1, v_2, ..., v_n}\) in an \(n\) vector space is said to be linearly dependent if there exist scalars \(c_1, \ c_2, \ ..., \ c_n\), not all zero, such that:
$$c_1v_1 + c_2v_2 + ... + c_nv_n = 0$$
where \(0\) is the zero vector.
- If we had a fourth vector, say \(v = (1, \ 1, \ 1)\), this would be linearly dependent on the basis vectors.
- That's because \(v\) can be expressed as a linear combination of basis vectors \(i, \ j\), and \(k\): \(v = i + j + k\).
- Unit vectors are basis vectors that have been normalized to have a length or magnitude of \(1\).
- A unit vector points in a specific direction but has no magnitude - they indicate direction only, without any magnitude information.
- Unit vectors are denoted with a "hat" symbol, such as \(\hat{i}, \ \hat{j}, \hat{k}\).
- In an \(n\)-dimensional space, the unit vectors are:
- \(\hat{i} = (1, 0, 0, ..., 0)\)
- \(\hat{j} = (0, 1, 0, ..., 0)\)
- \(\hat{i} = (0, 0, 1, ..., 0)\)
- ...
- n'th unit vector \(= \ (0, 0, 0, ..., 1)\)
- Unit vectors are useful for defining direction without the influence of magnitude.
Vector Spaces
A vector space \(V\) is a collection of objects with a vector addition and scalar multiplication defined (and closed under both operations) over a field \(\mathbb{F}\) and which in addition satisfies the following axioms:
- \((\alpha + \beta)v = \alpha v + \beta v\) for all \(v \in V\) and \(\alpha, \ \beta \in \mathbb{F} \).
- \( \alpha(\beta v) = (\alpha \beta)v \), \(v \in V\) and \(\alpha, \ \beta \in \mathbb{F} \).
- \(u + v = v + u \) for all \(u, \ v \in V \).
- \( u + (v + w) = (u + v) + w \) for all \( u, \ v, \ w \in V \).
- \( \alpha(u+v) = \alpha u + \alpha v \) for all \(u, \ v \in V \) and \(\alpha \in \mathbb{F} \).
- There exists \(0 \in V\) such that \( 0 + v = v \); \(0\) (zero vector) is usually called the origin.
- \(0 v = 0\) for all \(v \in V\).
- For every \(v \in V \), there exists \(-v \in V \), called the additive inverse of \(v\), such that \(v + (-v) = 0\)
- \(Iv = v\), where \(I\) denotes the multiplicative identity (unit) in \(\mathbb{F}\).
A vector in mathematics is a quantity that has both magnitude and direction (e.g., along \(x\) or \(y\) axis) but no position; for example, velocity and acceleration.
Scalar
A scalar, is a one-dimensional quantity/object such as a real numbers \(5\) and \(-2\). Scalars have magnitude but no direction, and they can be considered as single values representing quantities like temperature, distance, or time.
Scalar Field \(\mathbb{F} \)
Examples of scalar fields \((\mathbb{F})\) is the set of real numbers \((\mathbb{R})\) and the set of complex numbers \((\mathbb{C})\).
Linear Mapping
Given vector spaces \(V_1\) and \(V_2\), a mapping \(L: V_1 \rightarrow V_2\) is called linear if
- \(L(u+v) = L(u) + L(v) \);
- \(L(\alpha v) = \alpha L(v) \)
Linear Operator
A linear operator, \(L\), on a vector space \(V\) over a field \(\mathbb{F}\) is a map from \(V\) to itself (\(L: V \rightarrow V\)) that preserves the linear structure of \(V\), that's forn any \(u, \ v \in V \) and any
\(\alpha \in \mathbb{F} \);
- \(L(u+v) = L(u) + L(v) \); and
- \(L(\alpha v) = \alpha L(v) \).
Identity Operator
The identity operator (Let's denote it by \(I\)) is the operator on \(V\) such that \(Iv = v\) for all \(v\in V\).
Eigenvalues and Eigenvectors
Given an operator, \(L\), any vector, \(v\), such that \(Lv = \lambda v\) for some \(\lambda \in \mathbb{F}\) is called an eigenvector of \(L\), and \(\lambda\) is its associated eigenvalue.
Examples of Linear Operators
A function \(f\) is called a linear operator if it has the following properties:
- \(f(x+y) = f(x) + f(y) \) for all \(x\) and \(y\).
- \(f(cx) = cf(x) \) for all \(x\) and all constants \(c\).
The most common examples of linear operators are differentiation and integration, where the above rule looks like:
- \( \frac{\rm{d}}{\rm{d}x}\left(au + bv\right) = a\frac{\rm{d}u}{dx} + b\frac{\rm{d}v}{\rm{d}x}\);
- \(\int_{r}^{s}(au + bv)\rm{d}x = a\int_r^s udx + b\int_r^s v\rm{d}x \)
Note
\(x: = y\) means the item on the left hand side is being defined to be what is on the right hand side (\(x\) is defined to \(y\)); while
\(1 = \rm{sin}^2(\theta) + \rm{cos}^2(\theta) \) means the two sides are equal.
\(\mathbb{F}^{n} \) - the \(n\)-tuples Space
A vector \(v\) is an object defined by the set of \(n\)-tuples, (\(v_1, v_2, ..., v_n\)). The vector space \(\mathbb{R}^3\) contains triples \(v = (x_1, x_2, x_3\)) of real numbers with points in three-dimensional Euclidean space. In \(\mathbb{R}^3\), \(3\) defines the dimension of the vector space, and \(\mathbb{R}\) indicates the field \(\mathbb{F}\) where the components that constitute vectors belong to.
We can call \(\mathbb{R}^3\) a three-dimensional vector space over the real field.
Subspaces
Let \(V\) be a vector space over the field \(\mathbb{F}\). A subspace of \(V\) is a subset \(W\) of \(V\) which is itself a vector space over \(\mathbb{F}\) with the closed operations of the vector addition and scalar multiplication on \(V\) such that
- for any \(u, v \in W\), then \(u+v \in W\);
- \(0 \in W\);
- For each \(u \in W \), then \((-u) \in W \);
- For each \(u \in W\) and each scalar \(c\in \mathbb{F} \), then \(cu \in W \).
A non-empty subset \(W\) of \(V\) is a subspace of \(V\) if and only if for each pair of vectors \(u, v \in W\) and each scalar \(c \in \mathbb{F} \), the vector \(cu + v \in W\).
Spanning in Terms of Vector Spaces
Given a vector space \(V\) over a field \(\mathbb{F}\), then span of a set \(S\) of vectors is defined to be the intersection of all subspaces of \(V\) that contain the set \(S\).
Functional Analysis
David Hilbert, a German mathematician, and Stefan Banach a Polish mathematician are both well known for laying down the foundation of what is today called Functional Analysis. Functional Analysis is a coupling of algebraic operations on one side and, Real and Complex analysis on one side. Hilbert realized that some analytical notions/ideas such as norm and the inner product could be extended from vectors which are finite sequences of real numbers to infinite sequences of real numbers. If it is possible to define the convergence of the infinite sequence of numbers, then the Hilbert space is complete. Completeness of Hilbert space means every Cauchy sequence converges to a finite value.
Many problems in mathematical analysis study are not primarily concerned with a single object such as function, a measure or an operator.
Most of the interesting classes that occur in this way turn out to be vector spaces, either with real scalars or with complex ones -- they deal with large class of objects.
A Hilbert space is a vector space \(H\) (real or complex) with an inner product \(<f,g>\) such that the norm defined by an inner product, where \( f, g \in H\) turns \(H\) into a complete metric space. If the metric defined by the norm is not complete, then \(H\) is instead known as an inner product space.
Banach extended Hilbert's ideas considerably. Banach space is a vector space with a norm but not necessarily given by an inner product. It turns out that, if we consider the infinite-dimensional space in an algebraic sense and we want to apply some analysis on it, then we have what is called Functional Analysis. The functional analysis considers spaces consisting of functions, sequences, and linear maps between them.
1. METRIC SPACES
The knowledge of metric spaces is a building block towards the study of functional analysis. Metric spaces illustrate the notion of distance between any two points on an arbitrary set \(X\) by generalizing the three properties outlined below.
Definition
Let \(X\) be a nonempty set (we can think of this set as a collection of points). A real-valued function \(d\) defined on \(X \times X\), that is a map \(X \times X \rightarrow [0, \infty)\) is said to be a metric on \(X\) if it satisfies the conditions:
- The distance between any two arbitrary points \(x\) and \(y\) is always positive, and it vanishes if and only if the two points lie on the same position,
\begin{align}\nonumber
d(x,y)\ge 0 \ \text{for all} \ x, y \in X, \ \text{and} \ d(x,y) = 0 \ \text{if and only if} \ x = y;
\end{align} - The distance between two points \(x\) and \(y\) is the same regardless of whichever point it is measured from,
\begin{align}\nonumber
d(x,y) = d(y,x) \ \text{for all} \ x, y \in X;
\end{align} - The distance from one point to another cannot exceed the sum of the distances from these points to an intermediate point,
\begin{align}\nonumber
d(x,y) \le d(x,z) + d(z,y) \ \text{for all} \ x, y \in X.
\end{align}
\(d\) is the generalization of the distance function, and \(d(x,y)\) is interpreted as the distance between two points/elements \(x\) and \(y\) of the set \(X\).
The set \(X\) together with a given metric \(d\) on \(X\) is called a metric space and it is denoted by \((X,d)\). Sometimes the metric space \((X, d)\) is simply denoted by \(X\).
Properties of the metric d
- For any \(x, y, z_1, z_2, \dots, z_n \in X\), from the triangle inequality, it follows by induction that
\begin{align}\nonumber
\begin{split}
& d(x,y)\le d(x,z_1) + d(z_1, y)\\
& \le d(x, z_1) + d(z_1, z_2) + d(z_2, y)\\
& \le d(x, z_1) + d(z_1, z_2) + d(z_2, z_3) + d(z_3, y)\\
& \dots \dots \dots \dots \\
& \le d(x,z_1) + d(z_1, z_2) + \dots + d(z_n,y).
\end{split}
\end{align} - For any \(x, y, z \in X \) we have
\begin{align}\nonumber
|d(x,z) - d(y,z)|\le d(x,y).
\end{align}
Proof:
Let \( x, y, z \in X\). By the triangle inequality we have
\begin{align}\nonumber
\begin{split}
& d(x,z) \le d(x,y) + d(y,z)\\
& d(x,z) - d(y,z) \le d(x,y).
\end{split}
\end{align}
Again, by the triangle inequality, we have
\begin{align}\nonumber
\begin{split}
& d(y,z)\le d(y,x) + d(x,z)\\
& -(d(x,z) - d(y,z)) \le d(y,x) = d(x,y) \ \text{(symmetry property)}\\
& d(x,z) - d(y, z) \ge -d(x,y)\\
& -d(x,y) \le d(x,z) - d(y,z)
\end{split}
\end{align}
We finally have
\begin{align}\nonumber
\begin{split}
& -d(x,y)\le d(x,z)-d(y,z) \le d(x,y)\\
& |d(x,z) - d(y,z)|\le d(x,y).
\end{split}
\end{align}
Example 1: Let \(\mathbb{F} \in \{\mathbb{R}, \mathbb{C} \}\) i.e., the set of all complex or real numbers. For any \(x, y \in \mathbb{F} \), define
\begin{align}\nonumber
d(x,y) = |x-y|.
\end{align}
Then \(\mathbb{F}, d\) is a metric space and the metric \(d\) is called the usual metric on \(\mathbb{R}/ \mathbb{C} \).
Proof: For all \(x, y, z\in \mathbb{F}\) we have ( we use the fact \(|a+b|\le |a|+|b| \) for real or complex numbers \(a\) and \(b\) for the triangle einequlaity):
-
\begin{align}\nonumber
d(x,y)\ge 0;
\end{align} - \begin{align}\nonumber
d(x,y) = |x-y| = 0 \ \text{if and only if and only if} \ x = y;
\end{align} - \begin{align}\nonumber
d(x,y) = |x-y| = |-(x-y)| = |y-x| = d(y,x);
\end{align} -
\begin{align}\nonumber
\begin{split}
& d(x,y) = |x-y| = |(x-z) + (z-y)|\\
& \le |x-z| + |z-y|\\
&= d(x,z) + d(z,y).
\end{split}
\end{align}
\begin{align}\nonumber
d(x,y) = \begin{cases}
0 \ \text{if} \ x = y \\
1 \ \text{if} \ x \ne y.
\end{cases}
\end{align}
Then \((X, d)\) is a metric space. The metric \(d\) is called discrete metric and the space \((X, d)\) is called discrete metric space.
This example shows that on each nonempty set we can always define at least one metric -- the discrete metric space.
Example 3: Let \(X = \mathbb{R}^2\), the set of all points in the coordinate plane. For any \(x = (x_1, x_2), y = (y_1, y_2) \) in \(X\), define
\begin{align}\nonumber
d(x,y) = \sqrt{(x_1-y_1)^2 + (x_2 - y_2)^2}.
\end{align}
Then \((X, d)\) is a metric space, and \(d(x,y)\) is the natural distance between two points in a plane. \(d\) is known as the usual metric on \(\mathbb{R}^2\).
Let's verify this. For any \( x = (x_1,x_2), y = (y_1, y_2), z = (z_1, z_2) \) in \(X\):
- It is obvious that \(d(x,y) \ge 0\).
- \begin{align}\nonumber
\begin{split}
d(x,y) = 0 & \Longleftrightarrow \sqrt{(x_1-y_1)^2 + (x_2-y_2)^2} = 0\\
& \Longleftrightarrow (x_1-y_1)^2 = 0 \ \text{and} \ (x_2-y_2)^2 = 0\\
& \Longleftrightarrow x_1 = y_1, x_2 = y_2 \\
& \Longleftrightarrow x = y.
\end{split}
\end{align} - \begin{align}\nonumber
\begin{split}
d(x,y) & = \sqrt{(x_1-y_1)^2 + (x_2-y_2)^2}\\
& = \sqrt{(y_1-x_1)^2 + (y_2-x_2)^2}\\
& = d(y,x).
\end{split}
\end{align} - \begin{align}\nonumber
\begin{split}
\left[d(x,y)\right]^2 & = (x_1-y_1)^2 + (x_2-y_2)^2\\
& = \left[(x_1-z_1)+(z_1-y_1)\right]^2 + \left[(x_2-z_2) + (z_2-y_2)\right]^2\\
& = (x_1-z_1)^2 + (z_1-y_1)^2\\
& + 2\left[\underbrace{(x_1-z_1)}_{a}\underbrace{(z_1-y_1)}_{b} + \underbrace{(x_2 - z_2)}_{c}\underbrace{(z_2-y_2)}_{d} \right]\\
& + (x_2-z_2)^2 + (z_2-y_2)^2
\end{split}
\end{align}
Let \(x_1-z_1 = a, z_1-y_1 = b, x_2 - z_2 = c, \ \text{and} \ z_2 - y_2 = d \), and apply the inequality \((ab+ cd)\le \sqrt{(a^2 + c^2)(b^2 + d^2)} \) to have
\begin{align}\nonumber
\begin{split}
\left[d(x,y)\right]^2 & \le (x_1-z_1)^2 + (x_2-z_2)^2\\
& + 2\sqrt{\underbrace{(x_1-z_1)^2}_{a^2} + \underbrace{(x_2-z_2)^2}_{c^2}}\sqrt{\underbrace{(z_1-y_1)^2}_{b^2} + \underbrace{(z_2-y_2)^2}_{d^2}}\\
& + (z_1-y_1)^2 + (z_2-y_2)^2\\
& = \left[d(x,z)\right]^2 + 2d(x,z)d(z,y) + \left[d(z,y)\right]^2\\
& = \left[d(x,z) + d(z,y)\right]^2\\
\implies d(x,y) & \le d(x,z) + d(z,y).
\end{split}
\end{align}
Let \(X = \mathbb{R}^2 \). For any \(x = (x_1, x_2), y = (y_1, y_2) \) in \(X\), define
\begin{align}\nonumber
d(x,y) = |x_1-y_1| + |x_2-y_2|.
\end{align}
Then \(d\) is a metric on \(X\) and \((X, d)\) is a metric space.
Proof: Conditions (1 - 4) follows from Example 1. Let's verify the triangle inequality. For any \( x = (x_1, x_2), y =(y_1, y_2), \ \text{and} \ z = (z_1, z_2) \) in \(X\):
\begin{align}\nonumber
\begin{split}
d(x,y) & = |x_1-y_1| + |x_2-y_2|\\
& = |(x_1-z_1) + (z_1-y_1)| + |(x_2-z_2) + (z_2-y_2)|\\
& \le |x_1-z_1| + |z_1-y_1| + |x_2-z_2| + |z_2-y_2|\\
& = |x_1-z_1| + |x_2-z_2| + |z_1-y_1| + |z_2-y_2|\\
& = d(x,z) + d(z,y).
\end{split}
\end{align}
2. NORMED LINEAR SPACES
Definition
A normed linear space (NLS) is a vector space equipped with a norm which is a function that assigns a non-negative (scalar value) to each vector in the space.
Mathematically, a NLS is a mathematical object that consists of a vector space $V$ over a field $\mathbb{F}$ \((\mathbb{R} \ \text{or} \ \mathbb{C})\), together with a function $||.||: V \rightarrow \mathbb{R}^{+} \cup \{0 \} \ ([0,\infty))$ , called a norm, that satisfies the following properties:
- $||x|| \le 0$ for all $x\in V$, and $||x|| = 0$ if and only if $x =0$.
- $||\alpha x|| = |\alpha|.||x||$ for all $\alpha \in \mathbb{F}$ and $x \in V$.
- $||x + y|| \le ||x|| + ||y||$ for all $x, \ y \in V$ (the triangle inequality).
- The first property says that the norm is non-negative, and only zero if the vector itself is the zero vector.
- The second property says that the norm scales linearly with a scalar multiplication by a field element.
- The third property says that the norm satisfies the triangle inequality, which is a statement about the length of the shortest path between two points.
- Distance: In a NLS, we can define distance between two vectors $x$ and $y$ as $||x - y||. This is simply the norm of the difference between the two vectors. Distance measures how close two vectors are to each other in the NLS.
- Convergence: In a NLS, a sequence $\{x_n \}$ of vectors is said to converge to a vector $x$ if for any $\epsilon > 0$, there exists an $N\in \mathbb{N}$ such that $||x_n - x|| < \epsilon$ for all $n > N$.
This means that the sequence gets arbitrarily close to $x$ as $n$ becomes large enough.
Convergence is a crucial concept in an analysis and leads to notions such as limit, continuity, and compactness. - Continuity: A function $f$ from a NLS $X$ to another NLS $Y$ is continuous at a point $x \in X$ if for any $\epsilon >0$, there exists a $\delta >0$ such that $||x-y||< \epsilon$ implies $||f(x) - f(y)|| \epsilon$.
This means that, small perturbations of the input $x$ leads to small perturbations of the output $f(x)$.
Continuity is a fundamental concept in analysis, as it captures the notion of smoothness and allows us to extend various algebraic and topological properties from the space \(X\) to space \(Y\). - Completeness: A NLS \(X\) is complete if every Cauchy sequence in it converges to a limit in \(X\).
Completeness is an important concept in analysis, as it helps us to establish the existence of certain objects that might not exist in an incomplete space. For example, in an incomplete space, a sequence might converge to something that is not itself a limit point of the space. - Boundedness: A subset \(A\) of a NLS \(X\) is said to be bounded if there exists a real number \(M\) such that \(||x|| \le M \) for all \(x \in A\).
This means that the set \(A\) does not contain vectors that are arbitrarily far apart from each other in the NLS.
Boundedness is a useful concept in analysis since it allows us to reason about convergence and continuity of functions defined on the set \(A\). For example, if a function is continuous and \(A\) is a bounded set, then its image under the function will also be a bounded set.
Arbitrarily close: The phrase arbitrarily close in mathematics means that something can be made as close as desired to something else.
For example, in a NLS, a sequence \(x_n\) is said to converge to a limit \(x\) if for any real number \(\epsilon\) greater than \(0\), there exists a positive integer \(N\) such that the distance between \(x_n\) and \(x\) is less than \(\epsilon \) whenever \(n\) is greater than \(N\). In this context, arbitrarily close means that given any tolerance level \(\epsilon\), we can always find a point in the sequence that is close enough to the limit, such that the distance between them is less than \(\epsilon\).
The concept of arbitrarily close is important in many areas of mathematics such as analysis, topology and geometry.
Limit: In a NLS, a limit of a sequence \(\{x_n\}\) of vectors is a vector \(x\) in the space such that for any \(\epsilon>0\), there exists and integer \(N\in \mathbb{N}\) such that \(||x_n - x|| < \epsilon\) for all \(n\ge N\).
This means that, we can get arbitrarily close to \(x\) by choosing a term in the sequence at or after some fixed index \(N\). In other words, we can make the distance between \(x_n\) and \(x\) as small as we like by choosing \(n\) large enough.
In a more concrete mathematical terms, we say that a sequence \(\{x_n\}\) converges to \(x\) if
\begin{align}
\lim_{n\to \infty}(||x_n - x||) = 0,
\end{align}
where \(\lim\) is the limit operator and \(||.||\) is the norm on the space.
The concepts of convergence and limits are coupled together. We can alternatively define these concepts as follows:
- A sequence \(\{x_n\}\) in a NLS \(X\) converges to a limit \(x\) written \(x_n \to x\) as \(n \to \infty\), when for \(\forall \epsilon>0\), \(\exists N\), such that \(n\ge N\).
- A sequence \(\{x_n\}\) in \(X\) converges to \(x \in X\) if \(||x_n - x|| \to 0\) as \(n\to \infty\).
A sequence \(\{x_n\}\) in \(X\) is Cauchy if \(||x_n - x_m|| \to 0\) as \(n, \ m \to \infty\).
Cauchy sequence is a generalization of the convergent sequence.
Sequences
In mathematics, a sequence is an ordered list of elements, typically numbers, that are arranged in a specific order.
The elements of a sequence are usually indexed by natural numbers, starting from \(1\) or \(0\), and denoted by \(a_n\), where \(n\) is the index.
Each element in the sequence is called a term. The terms in a sequence are usually denoted by the subscripts \(1\), \(2 \), \(3\), ..., so that the first term is denoted by \(a_1\), the second term is denoted by \(a_2\), the third term is denoted by \(a_3\), and so on.
A sequence can be finite or infinite. A finite sequence has a fixed number of terms, while an infinite sequence has an unbounded number of terms.
For example, the following is a sequence of five numbers: \(a_1 =1\), \(a_2 = 3\), \(a_3 = 5\), \(a_4 = 7\), \(a_5 = 9\).
In a NLS \((X, ||.||)\), the elements of a sequence can be vectors, functions or other objects.
Compactness
A set of functions or operators is said to be compact if any sequence in the set has a convergent subsequence.
Examples of NLS
- Let \(X = \mathbb{R}^{n}\) and define
\begin{align}\nonumber
||x|| = \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}},
\end{align}
where \(x\in \mathbb{R}^{n} \). This is a norm, called the Euclidean norm, and has the interpretation of the length of the vector \(x\). - The space of all real-valued continuous functions on the interval \([0,1]\), that is \(X = C([0,1]) \), can be equipped with many norms, for example,
the \(l^{p}\)-norms:
\begin{align}\nonumber
||f||_{p} = \left(\int_{0}^{1}|f(x)|^{p}dx\right)^{\frac{1}{2}},
\end{align}
where \(f\in X\), \(1\le p < \infty\),
and the sup-norm \(p = \infty\):
\begin{align}\nonumber
||f||_{\infty} = \text{sup}_{x \in [0,1]}|f(x)|,
\end{align}
where \(f \in X\). - \(X = l^{p} \) has norm given by
\begin{align}\nonumber
||x|| = \left(\sum_{[i=1}^{\infty}|x_i|^{p} \right)^{\frac{1}{p}},
\end{align}
where \(x \in X\).
We prove the first example and leave the rest for the reader.
Let \(X = \mathbb{R}^{n}\) and define
\begin{align}
||x|| = \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}},
\end{align}
where \(x\in \mathbb{R}^{n} \). Let's prove that this is indeed a norm.
Proof
- \( ||x|| = \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}}\) = \((|x_1|^2 + |x_2|^2 + \dots + |x_n|^2)^{\frac{1}{2}}\).
Since \(|x_i|\ge 0 \implies ||x||\ge 0\).
Suppose \(||x|| = 0\). This implies \(|x_i| = 0 \implies x_i = 0, \ \text{and thus} \ x = 0 \).
Conversely, suppose \(x = 0\), then \(|x_i| = 0 \implies x_i = 0 \) and hence \(||x|| = 0\).
We have shown that \(||x|| \ge 0\) and \(||x|| = 0\) if and only if \(x = 0\). - for all \(x\in X\) and any \(\alpha \in \mathbb{R}\),
\begin{align}\nonumber
\begin{split}
||\alpha x|| & = \left(\sum_{i=1}^{n}|\alpha x_i|^2\right)^{\frac{1}{2}}\\
& = (|\alpha x_1|^2 + |\alpha x_2|^2 + \dots + |\alpha x_n|^2)^{\frac{1}{2}}\\
& = (|\alpha|^2 |x_1|^2 + |\alpha|^2 |x_2|^2 + \dots + |\alpha|^2 |x_n|^2)^{\frac{1}{2}}\\
& = (|\alpha|^2 (|x_1|^2 + |x_2|^2 + \dots + |x_n|^2))^{\frac{1}{2}}\\
& = |\alpha|(|x_1|^2 + |x_2|^2 + \dots + |x_n|^2)^{\frac{1}{2}}\\
& = |\alpha|\left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}}\\
& = |\alpha| \lVert x \rVert
\end{split}
\end{align} - Recall the Cauchy-Schwarz inequality: If \(x, y \in \mathbb{R}^{n}\), then
\begin{align}
\sum_{i=1}^{n} |x_iy_i| \le \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}} \left(\sum_{i=1}^{n}|y_i|^2\right)^{\frac{1}{2}}.
\end{align}
which also implies
\begin{align}
\sum_{i=1}^{n} |x_i||y_i| \le \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}} \left(\sum_{i=1}^{n}|y_i|^2\right)^{\frac{1}{2}}.
\end{align}
Note:
The Cauchy inequality leads to another inequality called Minkowski: If \(x, y \in \mathbb{R}^{n}\), then
\begin{align}
\left(\sum_{i=1}^{n} |x_i + y_i|^2 \right)^{\frac{1}{2}} \le \left(\sum_{i=1}^{n}|x_i|^2\right)^{\frac{1}{2}} + \left(\sum_{i=1}^{n}|y_i|^2\right)^{\frac{1}{2}}.
\end{align}
We use the Cauchy-Schwarz inequality to prove the triangle inequality, \(||x + y|| \le ||x|| + ||y|| \).
We know that
\begin{align}\nonumber
\begin{split}
||x+y|| & = \sqrt{(x_1 + y_1)^2 + (x_2 + y_2)^2 + \dots + (x_n + y_n)^2}\\
& = \left(\sum_{i=1}^{n} |x_i + y_i|^2 \right)^{\frac{1}{2}}\\
& \implies ||x+y||^2 = \sum_{i=1}^{n} |x_i + y_i|^2\\
& = \sum_{i=1}^{n} |x_i + y_i||x_i + y_i|\\
& \le \sum_{i=1}^{n} |x_i + y_i|(|x_i| + |y_i|)\\
& = \sum_{i=1}^{n} |x_i + y_i||x_i| + \sum_{i=1}^{n} |x_i + y_i||y_i| \\
& \le \left(\sum_{i=1}^{n} |x_i + y_i|^2 \right)^{\frac{1}{2}} \left(\sum_{i=1}^{n} |x_i|^2 \right)^{\frac{1}{2}}\\
& + \left(\sum_{i=1}^{n} |x_i + y_i|^2 \right)^{\frac{1}{2}} \left(\sum_{i=1}^{n} |y_i|^2 \right)^{\frac{1}{2}} \ \text{(Here, we applied the Cauchy-Schwarz inequality)} \\
& = \left(\sum_{i=1}^{n} |x_i + y_i|^2 \right)^{\frac{1}{2}} \left( \left(\sum_{i=1}^{n} |x_i|^2 \right)^{\frac{1}{2}} + \left(\sum_{i=1}^{n} |y_i|^2 \right)^{\frac{1}{2}} \right)
\end{split}
\end{align}
It follows that
\begin{align}\label{eqnC}
& ||x+y||^2 \le ||x+y|| \left(||x|| + ||y|| \right)
\end{align}
Suppose, \(||x+y||^2 = 0, \ \text{this implies} \ ||x+y|| = 0 \implies ||x+y|| \le ||x|| + ||y||\).
Suppose \(||x+y||^2 \ne 0\), then dividing \((\ref{eqnC})\) by \(||x+y||\) both sides, we have
\begin{align}\nonumber
||x+y|| \le ||x|| + ||y||
\end{align}
The real linear space \(\mathbb{R}\) is a normed linear space (NLS) with norm \(||.||: \mathbb{R} \to \mathbb{R}\) defined by \(||x|| = |x|\); for all \(x \in \mathbb{R}\).
Proof
- For any \(x\in \mathbb{R}\), we have \(||x|| = |x|\ge 0; \ \implies ||x|| \ge 0\).
Suppose \(||x|| = 0\); this implies \(|x| = 0 \implies x = 0\).
Conversely, suppose \(x = 0\), this implies \(|x| = 0 \implies ||x|| = 0\).
Therefore, \(||x|| \ge 0\), and \(||x|| = 0\) if and only if \(x = 0\). - For any scalar \(\alpha\) and \(x \in \mathbb{R}\), we have
\(||\alpha x|| = |\alpha x| = |\alpha||x| = |\alpha|||x||\) - For all \(x_1, x_2 \in \mathbb{R}\), we have
\begin{align}\nonumber
\begin{split}
||x_1 + x_2|| & = |x_1 + x_2|\\
& \le |x_1| + |x_2|\\
& = ||x_1|| + ||x_2||\\
& \implies ||x_1 + x_2 || \le ||x_1|| + ||x_2||
\end{split}
\end{align}
The complex linear space \(\mathbb{C}\) is a normed linear space with the norm defined by \(||z|| = |z|; \ \forall z \in \mathbb{C}\). Prove (left for the reader).
Example 4
Let \(X\) be the vector space of real-valued functions, well defined and continuous on \([-1,1]\). Show that the following three maps, where \(f\in X\) are normed linear spaces:
- \(||f||_{1} = \int_{-1}^{+1}|f(x)|dx \).
- \(||f||_{2} = \left(\int_{-1}^{+1}f^2(x)dx\right)^{\frac{1}{2}} \).
- \(||f||_{\infty} = \text{sup}_{x\in[-1,1]}\{|f(x)|\} \) (left for reader to prove).
We check for the three properties that define a normed linear space:
-
- \begin{align}\nonumber
||f||_1 = \int_{-1}^{+1}|f(x)|\text{d}x \ge 0, \ \forall \ f \in X.
\end{align}
Suppose
\begin{align}\nonumber
||f||_1 = \int_{-1}^{+1}|f(x)|\text{d}x = 0, \ \implies \ f(x) = 0.
\end{align}
Conversely, suppose $f(x) = 0$, then
\begin{align}\nonumber
||f||_1 = \int_{-1}^{+1}|0|\text{d}x = \int_{-1}^{+1}0\text{d}x = [0x]_{-1}^{+1} = 0(1) - 0(-1) = 0.
\end{align}
Therefore, $||f||_1 \ge 0$, and $||f||_1 = 0$ if and only if $f=0$. -
\begin{align}\nonumber
\begin{split}
||\alpha f||_1 & = \int_{-1}^{+1}|\alpha f(x)|\text{d}x\\
& = \int_{-1}^{+1}|\alpha||f(x)|\text{d}x = |\alpha|\int_{-1}^{+1}|f(x)|\text{d}x\\
& = |\alpha|||f||_1, \ \forall \ f\in X \ \text{and} \ \alpha \in \mathbb{R}.
\end{split}
\end{align} - Suppose \(f, g \in X\), that's, they are continuous on $[-1,+1]$, then
\begin{align}\nonumber
\begin{split}
||f+g||_1 & = \int_{-1}^{+1}|(f+g)(x)|\text{d}x\ = \int_{-1}^{+1}|f(x) + g(x)|\text{d}x\\
& \le \int_{-1}^{+1}(|f(x)| + |g(x)|)\text{d}x\\
& = \int_{-1}^{+1}|f(x)|\text{d}x + \int_{-1}^{+1}|g(x)|\text{d}x\\
& = ||f||_1 + ||g||_1\\
& \implies ||f+g||_1 \le ||f||_1 + ||g||_1.
\end{split}
\end{align}
Since all the three properties satisfied, therefore \(X, ||.||\) is a normed linear vector space.
- \begin{align}\nonumber
-
- \(||f||_{2} = \left(\int_{-1}^{+1}f^2(x)dx\right)^{\frac{1}{2}} \geq 0\) for all \(f \in X\).
This is because, the integrand \(f^2(x) \geq 0 \ \text{on} \ [-1,1]\) - the square of a real number is always non-negative, and the integral of a non-negative function on the interval where the given function is non-negative is non-negative. \(f^2(x)\) will always be a quadratic function raised to some power \(n\) depending on the degree of the original function \(f(x)\).
Suppose that the \(L^2\)-norm defined by \(||f||_2 = \left(\int_{-1}^{+1}f^2(x)dx\right)^{\frac{1}{2}} = 0\). Then, this is true only if
\(f^2(x) = 0\) which implies \(f(x) = 0\).
Conversely, suppose \(f(x) = 0\). Then
\(||f||_2 = \left(\int_{-1}^{+1} 0^2 dx\right)^{\frac{1}{2}} = \left(\int_{-1}^{+1} 0 dx\right)^{\frac{1}{2}} = 0 \).
Hence, on \([-1,1]\), \(||f||_2 \geq 0\) and \(||f||_2 = 0\) if and only if \(f(x) =0\). - For all \(f \in X\) and any \(\alpha \in \mathbb{R}\), we have
\begin{align}\nonumber
\begin{split}
||\alpha f||_2 & = \left(\int_{-1}^{+1} (\alpha f(x))^2 dx\right)^{\frac{1}{2}}\\
& = \left(\int_{-1}^{+1} \alpha^2 f^2(x) dx\right)^{\frac{1}{2}}\\
& = \left(\alpha^2\left(\int_{-1}^{+1} f^2(x) dx\right)\right)^{\frac{1}{2}}
& = |\alpha|\left(\int_{-1}^{+1} f^2(x) dx\right)^{\frac{1}{2}}\\
& = |\alpha|||f||_2
\end{split}
\end{align} - Suppose \(f, g \in X\), then
\begin{align}\nonumber
\begin{split}
||f+g||_{2}^2 & = \int_{-1}^{+1}((f+g)(x))^2dx \\
& = \int_{-1}^{+1}(f(x) + g(x))^2dx\\
& = \int_{-1}^{+1}(f^2(x) + 2f(x)g(x) + g^2(x)) dx\\
& \le \int_{-1}^{+1}(|f(x)|^2 + 2|f(x)||g(x)| + |f(x)|^2) dx\\
& = \int_{-1}^{+1}|f(x)|^2dx + 2\int_{-1}^{+1}|f(x)||g(x)| dx +\int_{-1}^{+1} |f(x)|^2 dx \\
& = ||f||_2^2 + 2\int_{-1}^{+1}|f(x)||g(x)|dx + ||g||_2^2
\end{split}
\end{align}
By the Cauchy-Schwarz inequality,
\begin{align}\nonumber
\begin{split}
\int_{-1}^{+1}|f(x)||g(x)|dx & \le \left(\int_{-1}^{+1}|f(x)|^2dx \right)^{\frac{1}{2}} \left(\int_{-1}^{+1}|g(x)|^2dx \right)^{\frac{1}{2}}\\
& = ||f||_2||g||_2
\end{split}
\end{align}
Therefore,
\begin{align}\nonumber
\begin{split}
||f+g||_2^2 & \le ||f||_2^2 + 2||f||_2||g||_2 + ||g||_2^2\\
& \implies ||f+g||_2^2 \le \left(||f||_2 + ||g||_2 \right)^2
\end{split}
\end{align}
Taking square roots both sides, we have
\begin{align}\nonumber
||f+g||_2 \le ||f||_2 + ||g||_2
\end{align}
Alternatively, we could prove the triangle inequality using the Minkowski inequality:
The \( L^2 \) norm is defined as:
\[
\|f\|_2 = \left( \int_{-1}^{+1} f^2(x) \, dx \right)^{\frac{1}{2}}
\]
By the Minkowski inequality for integrals, we have:
\[
\left( \int_{-1}^{+1} |f(x) + g(x)|^2 \, dx \right)^{\frac{1}{2}} \leq \left( \int_{-1}^{+1} |f(x)|^2 \, dx \right)^{\frac{1}{2}} + \left( \int_{-1}^{+1} |g(x)|^2 \, dx \right)^{\frac{1}{2}}
\]
Since \( |f(x)|^2 = f^2(x) \) and \( |g(x)|^2 = g^2(x) \), this simplifies to:
\[
\left( \int_{-1}^{+1} (f(x) + g(x))^2 \, dx \right)^{\frac{1}{2}} \leq \left( \int_{-1}^{+1} f^2(x) \, dx \right)^{\frac{1}{2}} + \left( \int_{-1}^{+1} g^2(x) \, dx \right)^{\frac{1}{2}}
\]
Therefore:
\[
\|f + g\|_2 \leq \|f\|_2 + \|g\|_2
\]
All the three properties are satisfied, and therefore, the map \(||f||_2 = \left(\int_{-1}^{+1}f^2(x)dx \right)^{\frac{1}{2}}\) is a normed linear vector space on \(X\).
- \(||f||_{2} = \left(\int_{-1}^{+1}f^2(x)dx\right)^{\frac{1}{2}} \geq 0\) for all \(f \in X\).
Define $C[0,1]$ to be the space of all continuous complex-valued functions on the interval $[0,1]$, that is,
$C[0,1] = \{f: [0,1] \rightarrow \mathbb{C}| \ f \ \text{is continuous}\}$. Prove that $\left(C[0,1], ||.||_{\infty}\right)$ with $||f||_{\infty} = \underset{x\in [0,1]}{\text{sup}}|f(x)|$ is a normed linear space.
Proof
We check for the three properties that define a normed linear space:
- $||f||_{\infty} = \underset{x\in [0,1]}{\text{sup}}|f(x)| \ge 0$ since $|f(x)| \ge 0$ for all $x\in [0,1]$ and all \( f \in C[0,1] \).
$||f||_{\infty} = 0 \Longleftrightarrow \underset{x \in [0,1]}{\text{sup}} |f(x)| = 0:$
Suppose $||f||_{\infty} = 0$, then by definition of supremum, we have $\forall \ x\in [0,1]$ and all \(f \in C[0,1]\),
$|f(x)|\le \underset{x\in[0,1]}{\text{sup}}|f(x)| = 0$. But we also have $|f(x)|\ge 0$ for all $x \in [0,1]$, and thus the only possibility is $f(x) = 0.$
Conversely,
If $f(x) =0$, it is trivial that $||f||_{\infty} = \underset{x\in[0,1]}{\text{sup}}|0| = 0.$
Thus \(||f||_{\infty} \geq 0\), and \(||f||_{\infty} = 0 \) if and only \(f(x) = 0\). - for all \(f \in C[0,1]\) and any complex scalar \(z \in \mathbb{C}\),
$||zf||_{\infty} = \underset{x\in[0,1]}{\text{sup}}|zf(x)| = \underset{x\in[0,1]}{\text{sup}}|z||f(x)| = |z|\underset{x\in[0,1]}{\text{sup}}|f(x)| = |z|||f||_{\infty}.$ - for all \(f,g \in C[0,1]\),
$||f+g||_{\infty} = \underset{x\in[0,1]}{\text{sup}}|(f+g)(x)|\\
= \underset{x\in[0,1]}{\text{sup}}|f(x)+g(x)| \le \underset{x\in[0,1]}{\text{sup}}\left(|f(x)| + |g(x)|\right)$
$= \underset{x\in[0,1]}{\text{sup}}|f(x)| + \underset{x\in[0,1]}{\text{sup}}|g(x)| = ||f||_{\infty} + ||g||_{\infty}$,
thus $||f+g||_{\infty} \le ||f||_{\infty} + ||g||_{\infty}$.
Since all the three properties are met, therefore $\left(C[0,1], ||.||_{\infty}\right)$ is a normed linear space.
Suppose $X$ is a normed linear space, then prove the following statements.
- If the limit of a sequence $x_n$ exists, then it is unique.
- Every convergent sequence in $X$ is Cauchy, but the converse is not true, in general.
- A Cauchy sequence is convergent if and only if it has a convergent subsequence.
- Every Cauchy sequence in $X$ is bounded.
- $X$ is a Banach space if and only if every absolute convergent series in $X$ is convergent.
Problem 1
Suppose $X$ is a normed linear space, then prove that if the limit of a sequence $x_n$ exists, then it is unique.
Proof
Suppoes $x$ and $y$ are two distinct limits of the sequence $x_n$, that is $x \ne y$.
Then for any $\epsilon > 0$, $\exists \ N\in \mathbb{N}$ such that for all $n \ge N$
$$||x_n - x|| < \frac{\epsilon}{2}.$$.
Similarly, for any $\epsilon > 0$, $\exists \ N_1\in \mathbb{N}$ such that for all $n \ge N_1$
$$||x_n - y|| < \frac{\epsilon}{2}.$$.
Let's choose $\epsilon \le ||x-y||$, and let $N_0 = \text{max}\{N, N_1\}$, therefore,
\begin{align}
\begin{split}
||x-y|| & = ||x - x_n + x_n - y|| \le ||x - x_n|| + ||x_n - y|| = ||x_n - x|| + ||x_n - y||\\
& < \frac{\epsilon}{2} + \frac{\epsilon}{2} \le 2. \frac{||x-y||}{2} = ||x-y||
\end{split}
\end{align}
for all $n \ge N_0$.
This is a contradiction, and hence the limit is unique, that is $x =y$.
Problem 2.
Suppose $X$ is a normed linear space. Prove that every convergent sequence in $X$ is Cauchy, but the converse is not true, in general.
Proof
To prove that every convergent sequence in a normed linear space $X$ is Cauchy, suppose $\{x_n\}$ is a convergent sequence in $X$ with limit $x$. Then, for any $\epsilon > 0$, there exists an integer $N$ such that for all $n \ge N$, $||x_n - x||< \frac{\epsilon}{2}$.
Now, if $m, n \ge N$, we have:
\begin{align}\nonumber
||x_n - x_m|| = ||(x_n - x) + (x - x_m)|| \le ||x_n - x|| + ||x_m - x|| < \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon.
\end{align}
Thus, $\{x_n\}$ is a Cauchy sequence.
To show that the converse is not true in general, consider the sequence $\{x_n\}$ defined by $x_n = \frac{1}{n}$ in the normed linear space $X = (0,1)$ with the usual absolute value norm, $(X, |.|)$.
This sequence is Cauchy, since for any $\epsilon > 0$, we can choose $N = \frac{1}{\epsilon}$ and note that if $m, n \ge N$, then
$|x_n - x_m| = |\frac{1}{n} - \frac{1}{m}| < \frac{1}{N} = \epsilon$.
However, this sequence does not converge in $X$, since its limit as $n \to \infty$ would have to be 0, which is not in $X$. We see that every Cauchy sequence is not convergent in $X$.
Therefore, the converse of the statement is not true in general.
Problem 3.
Suppose $X$ is a normed linear space. Prove that a Cauchy sequence in \(X\) is convergent if and only if it has a convergent sub-sequence.
Proof
To prove that a Cauchy sequence in a normed linear space $X$ is convergent if and only if it has a convergent subsequence, we will prove two statements:
1. If a Cauchy sequence in $X$ is convergent, then it has a convergent subsequence.
2. If a Cauchy sequence in $X$ has a convergent subsequence, then it is also convergent.
Statement 1:
Suppose $\{x_n\}$ is a Cauchy sequence in $X$ that converges to $x$. We want to show that $\{x_n\}$ has a convergent subsequence.
Since $\{x_n\}$ converges to $x$, for any $\epsilon > 0$, there exists an integer $N$ such that for all $n \ge N$, $||x_n - x|| < \frac{\epsilon}{2}$. Since $\{x_n\}$ is Cauchy, there exists an integer $N_1$ such that for all $n, n_k \ge N_1$, $||x_n - x_{{n}_{k}}|| < \frac{\epsilon}{2}$.
Now, let $N_0 = \text{max}\{N, N_1 \}$. Then, for all $n_{k} \ge N_0$, we have:
$$||x_{{n}_{k}} - x|| = ||x_{{n}_{k}} - x_n + x_n -x|| \le ||x_n - x_{{n}_{k}}|| + ||x_n -x|| < \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon.$$
Thus, the sequence $\{x_n\}$ has a subsequence $(x_{{n}_{k}}\}$ such that $||x_{{n}_{k}} - x|| < \epsilon$ for all $n_k \ge N_0$. This implies that $\{x_{{n}_{k}}\}$ converges to $x$, and the proof of statement 1 is complete.
Statement 2:
Suppose $\{x_n\}$ is a Cauchy sequence in $X$ that has a convergent subsequence $\{x_{{n}_{k}}\}$ with limit $x$. We want to show that the entire Cauchy sequence $\{x_n\}$ converges to $X$ as well.
Since $\{x_{{n}_{k}}\}$ converges to $x$, then for any $\epsilon > 0$, there exists an integer $N$ such that for all $n_k \ge N$, $||x_{{n}_{k}} - x|| < \frac{\epsilon}{2}$.
However, since $\{x_n\}$ is a Cauchy sequence, there exists an integer $N_1$ such that for all $n, n_k \ge N_1$, $||x_n - x_{{n}_{k}}|| < \frac{\epsilon}{2}$.
Now, let $N_0$ = $\text{max}\{N_1, N \}$. Then, for all $n, n_k \ge N_0$, we have:
$$||x_n - x|| = ||x_n - x_{{n}_{k}} + x_{{n}_{k}} - x|| \le ||x_n - x_{{n}_{k}}|| + ||x_{{n}_{k}} - x|| < \frac{\epsilon}{2} + \frac{\epsilon}{2} = \epsilon.$$
This implies $\{x_n\}$ converges to $x$, completing the proof of statement 2.
Therefore, combining both implications, we can conclude that a Cauchy sequence in a normed linear space X is convergent if and only if it has a convergent subsequence.
Problem 4
Suppose $X$ is a normed linear space. Prove that every Cauchy sequence in X is bounded.
Let $\{x_n\}$ be a Cauchy sequence in $X$. Then for every $\epsilon>0$, there exists $N\in\mathbb{N}$ such that for all $m,n \ge N$, $\|x_m-x_n\|<\epsilon.$
In particular, for $n=N$ and $m=N+1$, we have $\|x_{N+1}-x_N\|<\epsilon$. Therefore,
$$||x_{N+1} || = ||x_{N+1} - x_{N} + x_{N}||,$$
which implies $$\|x_{N+1}\| \leq \|x_{N+1}-x_N\|+\|x_N\| <\epsilon +\|x_N\|$$
Again, for $n=N+1$ and $m=N+2$, we have $\|x_{N+2}-x_{N+1}\|<\epsilon$. So
$$\|x_{N+2}\| \leq \|x_{N+2}-x_{N+1}\|+\|x_{N+1}\| <\epsilon + (\epsilon +\|x_N\|)$$
Continuing this process, we see that for any $n\ge N$,
$$\|x_n\|<\epsilon(n-N) +\|x_N\|$$
Since $\epsilon$ and $N$ are fixed, the right hand side is a constant, so the sequence $\{x_n\}$ is bounded.
See this proof too. and this doc.
Problem 5
Suppose $X$ is a normed linear space, then prove that $X$ is a Banach space if and only if every absolute convergent series in $X$ is convergent.
Proof
To prove that a normed linear space $X$ is a Banach space if and only if every absolutely convergent series in $X$ is convergent, we must prove two statements:
1. If $X$ is a Banach space, then every absolutely convergent series in $X$ is convergent.
2. If every absolutely convergent series in $X$ is convergent, then $X$ is a Banach space.
Statement 1:
Suppose $X$ is a Banach space, and let $\sum x_n$ be an absolutely convergent series in $X$, i.e., the series $\sum ||x_n||$ converges. We want to show that $\sum x_n$ converges in $X$ as well.
Since $\sum||x_n||$ converges , it implies that $\sum ||x_n|| < \infty$.
Then for any $\epsilon >0$, $\exists \ N \in \mathbb{N}$ such that for all $n,m \in N$, $n \ge m \ge N$ we have:
$$\sum_{k=m+1}^{n}||x_k|| = \left| \sum_{k=1}^{n}||x_k|| - \sum_{k=1}^{m} ||x_k|| \right| < \epsilon$$
since the sequence of partial sums, i.e., the series $\sum_{k=m+1}^{n}||x_k||$ is Cauchy, that is, converges in $X$.
Similarly, for $n \ge m \ge N$, we also have
$$\left|\left|\sum_{k=1}^{n}x_k - \sum_{k=1}^{m}x_k \right|\right| = \left|\left|\sum_{k=m+1}^{n}x_k\right|\right| \le \sum_{k=m+1}^{n}||x_k|| < \epsilon$$.
This implies $$\left|\left|\sum_{k=1}^{n}x_k - \sum_{k=1}^{m}x_k\right|\right| < \epsilon$$
and hence without loss of generality, the series $\sum_{n=1}^{\infty}x_n$ $(\text{the sequence of partial sums of} \ \sum_{n=1}^{\infty}x_n)$ is Cauchy in $X$.
Since $X$ is a Banach space, it implies that $\sum_{n=1}^{\infty}x_n$ or simply $\sum x_n$ converges in $X$.
It turns out that an absolutely convergent series is a Cauchy series.
Statement 2
Facts:
Every convergent sequence in $X$ is Cauchy.
A Cauchy sequence is convergent if and only if it has a convergent subsequence.
If a Cauchy sequence in $X$ has a convergent subsequence, then it is also convergent.
A Banach space is a complete normed vector space. For $X$ to be a Banach Space, we need to show that every Cauchy sequence converges to a limit that is also in the normed vector space $X$.
And we can then use the fact: If a Cauchy sequence in $X$ has a convergent subsequence, then it is also convergent to show that $X$ is Banach.
Thus, it suffices to show that the Cauchy Sequence $\{x_n\}$ has a convergent subsequence $\{x_{{n}_{k}}\}$ (the Cauchy sequence converges to the same limit as its subsequence).
Further example problems:
Suppose that the space $l^{\infty}$, that is, the space of all bounded sequences with the norm $||x||{\infty} = \text{sup}|x_i|$ for $i \in \mathbb{N}$ is a normed linear space. Prove that $l^{\infty}$ is complete in the induced metric defined by $d(x,y) = \text{sup}|\xi{i} - \eta_{i}|$.
Proof
To prove that $l^{\infty}$ is complete in the induced metric $d(x,y) = \sup|\xi_i - \eta_i|$, we need to show that every Cauchy sequence in $l^{\infty}$ converges to an element in $l^{\infty}$.
Let $\{x_{m}\}_{m=1}^{\infty}$ be a Cauchy sequence in $l^{\infty}$. By definition of the induced metric, for every $\varepsilon > 0$, there exists $N \in \mathbb{N}$ such that
$$d(x_{m}, x_{n}) = \sup|\xi_i^{(m)} - \xi_i^{(n)}| < \varepsilon$$
all $m, n \geq N.$
Now, fix $i \in \mathbb{N}$. Since the sequence of real or complex numbers $\{\xi_i^{(m)}\}_{m=1}^{\infty}$ is Cauchy in $\mathbb{R}$ or $\mathbb{C}$, it converges to some real/complex number $\xi_i$. Define the sequence $x = \{\xi_i\}_{i=1}^{\infty}$. We claim that $x \in l^{\infty}$.
Since $\{\xi_i^{(m)}\}_{m=1}^{\infty}$ is bounded for every $i \in \mathbb{N}$, there exists a constant $M_i$ such that $|\xi_i^{(m)}| \leq M_i$ for all $m \in \mathbb{N}$. Let $M = \sup|M_i|_{i=1}^{\infty}$. Then, for every $i \in \mathbb{N}$ and $m \in \mathbb{N}$, we have $|\xi_i^{(m)}| \leq M$, which implies that $|\xi_i| \leq M$ for all $i \in \mathbb{N}$. Hence, $x \in l^{\infty}$.
To show that $x_{m}$ $(\text{or} \ \{x_m\} )$ converges to $x$ in the induced metric, let $\varepsilon > 0$. We choose $N \in \mathbb{N}$ such that for all $m, n \geq N$, we have $d(x_{m}, x_{n}) = \sup|\xi_i^{(m)} - \xi_i^{(n)}| < \varepsilon$ for each $i \in \mathbb{N}$.
Taking the limit as $n \to \infty$, we obtain $\sup|\xi_i^{(m)} - \xi_i| \leq \varepsilon$ for all $m \geq N$.
Since $\varepsilon$ is arbitrary, this implies that $d(x_{m}, x) \to 0$ as $m \to \infty$. Therefore, $x_{m}$ converges to $x$ in the induced metric.
Hence, every Cauchy sequence in $l^{\infty}$ converges to an element in $l^{\infty}$. Therefore, $l^{\infty}$ is complete in the induced metric.
NB
We sometimes write $d(x^{(m)}, x^{(n)})$ instead of $d(x_m, x_n)$ and also often denote the sequence $\{x_m \}$ by $\{x_{m}\}_{m=1}^{\infty}$ or $\{x^{(m)}\}$ or $\{x^{(m)}\}_{m=1}^{\infty}$ or $(x^{(m)})_{m=1}^{\infty}$.
Bounded Linear Maps
A linear map or a linear operator $T$ between real (or complex) linear spaces $X$ and $Y$ is a function $T: X \rightarrow Y$ such that
\begin{align}
T(\lambda x + \mu y) = \lambda Tx + \mu Ty
\end{align}
for all $\lambda, \mu \in \mathbb{R} \ (\text{or} \ \mathbb{C})$ and for all $x, y\in X$.
- A linear map $T: X \rightarrow X$ is called a linear transformation of $X$, or linear operator on $X$.
- If $T: X \rightarrow Y$ is one-to-one and onto, then we say that $T$ is non-singular or invertible and define the inverse map
\begin{align}
T^{-1}: Y \rightarrow X
\end{align}
by $T^{-1}y = x$ if and only if $Tx = y$. This means $TT^{-1} = T^{-1}T = I$. Here, the linearity of $T$ implies the linearity of $T^{-1}$.
Therefore, if $X$ and $Y$ are normed linear spaces, we can define the notion of a bounded linear map. The boundedness of a linear map is equivalent to its continuity.
Definition: Let $X$ and $Y$ be two normed linear spaces. We denote both $X$ and $Y$ norms by $||.||$. A linear map $T: X \rightarrow Y$ is said to be bounded if there exists a constant $M\ge )$ such that
\begin{align}
||Tx||\le M||x|| \ \text{for all} \ x\in X.
\end{align} - If no such constant exists, then we say that $T$ is unbounded.
- If $T: X \rightarrow Y$ is unbounded linear map, then we define the operator norm or uniform norm $||T||$ of $T$ by
\begin{align}
||T|| = \text{inf}\{M: ||T|| \le M||x|| \ {\rm for} \ {\rm all} \ x\in X\}.
\end{align} - We denote the set of all linear maps $T: X \rightarrow Y$ by $\mathcal{L}(X,Y)$, and the set of all bounded linear maps $T: X \rightarrow Y$ by $\mathcal{B}(X,Y)$.
- When the domain and range of spaces are the same, we write $\mathcal{L}(X,X) = \mathcal{L}(X)$ and $\mathcal{B}(X,X) = \mathcal{B}(X)$.
- Linear maps on infinite-dimensional space need not to be bounded.
\begin{align}
||\{x_n\}|| = \text{sup}\{|x_n|\}
\end{align}
is complete in the induced metric, i.e., $\text{d}(x,y) = \text{sup}|x_i - y_i|, i \in \mathbb{N}$.
Theorem
Let $X$ and $Y$ be normed linear spaces and let $T$ be a continuous linear operator on $X$ into $Y$. Then there exists a uniquely determined continuous linear operator $\hat{T}$ of $\hat{X}$ into $\hat{Y}$ such that
\begin{align}
\hat{T}x = Tx \ \text{if} \ x\in X \ \text{and} \ ||\hat{T}|| = ||T||.
\end{align}
Theorem
A normed linear space $X$ is a Banach space if and only if every absolute convergent series in $X$ is convergent.
Remarks
We say that the series $\sum_{i=1}^{\infty}x_i$ is absolutely convergent or absolutely summable if $\sum_{i=1}^{\infty}||x_i||$ is convergent, i.e., $\sum_{i=1}^{\infty}||x_i|| < \infty$.
3. INNER PRODUCT SPACES
An inner product space $X$ is a vector space with an inner product $<, >$ defined on $X$, where an inner product on $X$ is a mapping $<,>: X \times
X \rightarrow \mathbb{F}$, and $\mathbb{F}$ is a scalar field, i.e., $\mathbb{F} = \mathbb{R} \ \text{or} \ \mathbb{C}$; such that $\forall x,y,z \in X$, and $\alpha \in \mathbb{F}$:
- $<x+y,z> = <x,z> + <y,z>$,
- $<\alpha x, y> = \alpha<x,y>$,
- $<x,y> = \overline{<y,x>}$ (complex conjugate),
- $<x,x> \ge 0$ and $<x,x> = 0 \Leftrightarrow x = 0$, i.e., $<x,x> = 0$ if and only if $x=0$.
If $x = (x_1, x_2, \dots, x_n)$, and $y =(y_1, y_2, \dots, y_n)$, i.e., $x,y \in \mathbb{R}^n$, then
\begin{align}
<x,y > = <(x_1, x_2, \dots, x_n), (y_1, y_2, \dots, y_n)> = \sum_{i=1}^{\infty}x_iy_i,
\end{align}
which is the dot product on $\mathbb{R}^n$.
An inner product defines a norm on $X$ given by
\begin{align}
||x|| = \sqrt{<x,x>},
\end{align}
that is,
\begin{align}
||x||^2 = <x,x> = \sum_{i=1}^{n} x_i^2 \ge 0.
\end{align}
A metric on $X$ is given by
\begin{align}
\text{d}(x,y) = ||x-y|| = <x-y, x-y>^{\frac{1}{2}},
\end{align}
so that $||x-y||^2 = <x-y, x-y>$.
Question: Is $||x|| = \sqrt{<x,x>}$ a NLS?
Proof (N_1 - N_4)
For all $x,y \in X$ and any $\alpha \in \mathbb{F}$,
- $<x.x> \ge 0 \implies ||x||^2 \ge 0 \implies ||x|| \ge 0$.
- $<x,x> = 0 \implies ||x||^2 = 0 \implies ||x|| = 0 \implies x = 0$.
If $x=0$, $\implies$ $<0,0> = ||0||^2 = ||0|| = 0$. - $<\alpha x, y> = \alpha<x,y> \implies <\alpha x,x> = \alpha<x,x>$, and
so $||\alpha x||^2 = <\alpha x, \alpha x> = \alpha \bar{\alpha}<x,x> = |\alpha|^2||x||^2$
This implies $||\alpha x||^2 = |\alpha|^2||x||^2 \implies ||\alpha x|| = |\alpha|||x||$. -
\begin{align}
\begin{split}
||x+y||^2 & = <x+y, x+y> = <x,x> + <x,y> + <y,x> + <y,y>\\
& = <x,x> + <y,y> + <x,y> + \overline{<y,y>} \\
& \le <x,x> + <y,y> + 2|<x,y>|\\
& = ||x||^2 + ||y||^2 + 2|<x,y>|\\
& \le ||x||^2 + ||y||^2 + 2||x||||y|| \ (\text{Note that} \ |<x,y>| \le ||x||||y||)\\
& \implies ||x+y||^2 \le ||x||^2 + ||y||^2 + 2||x||||y|| = \left(||x|| + ||y|| \right)^2 \\
& \implies ||x+y|| \le ||x|| + ||y||
\end{split}
\end{align}
\begin{align}
\sum_{i=1}^{n}|x_iy_i| \le \left(\sum_{i=1}^{n}|x_i|^2\right)^2 \left(\sum_{i=1}^{n}|y_i|^2\right)^2,
\end{align}
which implies
\begin{align}
\sum_{i=1}^{n}|x_i| \le \left(\sum_{i=1}^{n}|x_i|^2\right)^2.
\end{align}
This follows from the Cauchy-Schwarz inequality,
\begin{align}
|u.v|\le ||u||||v|| \implies u.v \le ||u||||v||.
\end{align}
In an inner product space, we have
\begin{align}\label{cs1}
|<u,v>|^2 \le <u,u>.<v,v>.
\end{align}
Taking square roots from both sides, $(\ref{cs1})$ implies
\begin{align}
|<u,v>| \le ||u||||v||,
\end{align}
since $||u||^2 = <u.u>$.
Question: Is the metric space on $X$ an IPS ? i.e., is $\text{d}(x,y) = <x-y, x-y>^{\frac{1}{2}}$ an NLS ?
Hilbert Space
A Hilbert space is a vector space \(H\) (real or complex) with an inner product \( <f, g>\) such that the norm
\begin{align}
||f|| = \sqrt{<f,g>}
\end{align}
defined by an inner product, where \( f, g \in H\) turns \(H\) into a complete metric space. If the metric defined by the norm is not complete, then \(H\) is instead known as an inner product space.
In simple terms, a Hilbert space is a complete inner product (IPS) (complete in the metric).
Therefore,
- every normed linear space is a metric space,
- every IPS is a normed linear space (NLS), and
- Every Hilbert space (HS) is a Banach space.
Is every NLS an IPS ?
Answer: No.
Question 2: Consider $\text{C}[a,b]$, i.e the space of all continuous functions on $[a,b]$, such that
\begin{align}
||f|| = \underset{x\in [a,b]}{\text{sup}}|f(x)|.
\end{align}
We claim that $\text{C}[a,b]$ is a NLS. Is $\text{C}[a,b]$ an IPS?
Answer: No - Provide a counter example.
Exercise: Prove that $\mathbb{R}^n$ with $x,y \in \mathbb{R}^n$ such that $<x,y> = \sum_{i=1}^{n}x_iy_i$ is an IPS. Show further that it is a Hilbert space, i.e., an IPS which is complete.