Lecture 1. Notion of Derivative

by Nirakar Neo

We shall always assume that {X,Y} are Banach spaces over the complex field {\mathbb{C}}. The real number field is denoted by {\mathbb{R}}. We shall now explore differential calculus in Banach spaces. It will be instructive to try to understand the theorems in well known Banach spaces like {\mathbb{R}^{n}} (over {\mathbb{R}}) or {\ell^{2}} (over {\mathbb{C}}) or {C[a,b]} (over {\mathbb{R}}). In general there are two notions of derivatives – the Gateaux derivative and the Frechet derivative, where the latter is stronger in the sense that Frechet differentiability implies Gateaux differentiability.

Definition: Let {f:X \rightarrow Y}, where {X,Y} are Banach space. For {a,h\in X}, define

\displaystyle \delta f(a,h)=\lim_{t\rightarrow 0}\frac{f(a+th)-f(a)}{t},\quad t\in \mathbb{R}

provided the limit exists. Then {\delta f(a,h)} is called the directional derivative of {f} at {a} in the direction of {h}. If this limit exists for every {h\in X} and if the function {h\mapsto \delta f(a,h)} is linear and continuous in {h}, then we say that the function is Gateaux differentiable and the function defined by

\displaystyle Df(a)(h)=\delta f(a,h)

is called the Gateaux derivative at {a}. Thus the linearity and continuity condition means {Df(a)\in L(X,Y).}

[An example to show that the map {h\mapsto \delta f(a,h)} is not always linear, and even if {Df(a)} exists, it may not be continuous at {a}.]

Definition: Let {f:X \rightarrow Y}, where {X,Y} are Banach space. Let {U\subset X} be open subset. We say that {f} is Frechet differentiable at {a\in U}, if there exists a bounded linear map {A\in L(X,Y)} such that

\displaystyle \lim_{h\rightarrow 0}\frac{\left\|f(a+h)-f(a)-A(h)\right\|}{\left\|h\right\|}=0

Some remarks should be in order, which we leave as simple exercises. The Frechet derivative is unique, and Frechet differentiability implies continuity. If {f'(a)} exists, then also {Df(a)} exists, and we have

\displaystyle Df(a,h)=f'(a)h.

Usually when we say {f} is differentiable, we shall mean Frechet differentiable. [An example for which the Gateaux derivative exists but Frechet derivative fails to exist.]

Our next step is to prove the mean value theorem.

Theorem: Let {f:X\rightarrow Y}, and suppose {\delta f(a+t(b-a),b-a)} exists for all {t\in[0,1]}, and is a continuous map of {t}, then

\displaystyle f(b)-f(a)=\int_{0}^{1}\delta f(a+t(b-a),b-a) dt.

Proof: Let {\phi\in Y^{*}}. Define {g(t)=\phi(f(a+t(b-a),b-a)):[0,1]\rightarrow\mathbb{R}} is differentiable, since it is the composition of two differentiable function. We have {g'(t)=\phi(\delta f(a+t(b-a),b-a))}. Using the fundamental theorem of calculus for real valued functions, we get

g(1)-g(0) = \int_{0}^{1}g'(t) dt =\int_{0}^{1}\phi(\delta f(a+t(b-a),b-a)) dt

= \phi\left(\int_{0}^{1}\delta f(a+t(b-a),b-a) dt\right)

Also, {g(1)-g(0)=\phi(f(b)-f(a))}, so altogether we get,

\displaystyle \phi\left(f(b)-f(a)-\int_{0}^{1}\delta f(a+t(b-a),b-a) dt\right)=0.

Since this holds for all {\phi\in Y^{*}}, we get the desired result.

Notice further that

\displaystyle \|f(b)-f(a)\|\leq \int_{0}^{1}\|\delta f(a+t(b-a),b-a)\| dt\leq \sup_{t\in[0,1]}\|\delta f(a+t(b-a),b-a)\|.

This is referred to as the mean value theorem in higher dimensions. The usual mean value theorem does not hold in higher dimensions as is seen by this example. Define {f(t)=(\cos(t),\sin(t))} for {t\in[0,2\pi]}. Then {f} is differentiable, and {f(2\pi)-f(0)=0}, but there is no such {t\in(0,1)} for which {f'(t)=(-\sin(t),\cos(t))} is zero.

Our next step is to generalize the chain rule. However we shall not prove the rule using strong hypotheses that both functions in a composition are Frechet differentiable. Instead we have –

Theorem: Let {a\in X, g(a)=b\in Y}. Suppose {f} is Frechet differentiable at {b} and {g} is having directional derivative in the direction {h} at {a}, then {f\circ g} has a directional derivative at {a}, and

\displaystyle \delta(f\circ g)(a,h)=f'(b)\delta g(a,h).

Before plunging into a proof, let us have an informal discussion. On the right hand side of the above equation, we have a linear transformation {f'(b):Y\rightarrow Z} acting on an element of Y – {\delta g(a,b)}. Let us see what we have got. {g} has directional derivative at {a} in the direction of {h}. This translates into

\displaystyle \lim_{t\rightarrow 0}\frac{g(a+th)-g(a)}{t}=\delta g(a,h),

that is given {\epsilon>0} there exists {\delta>0} such that if {\left|t\right|<\eta} we have

\displaystyle \|g(a+th)-g(a)-t\delta g(a,h)\|<\epsilon\left| t\right|.

{f} is Frechet differentiable at {b=g(a)}, so

\displaystyle \|f(b+k)-f(b)-Ak\|\leq\theta\|k\|,

where the right hand side goes to zero as {\|k\|\rightarrow 0}. We may take

\displaystyle k=g(a+th)-g(a)=t\delta g(a,h)+\tilde{\theta}\left|t\right|,

which certainly goes to zero at {t\rightarrow 0}. If we plug this in the equation of {f}, we’ll get

\displaystyle f(b+k)-f(b)=f'(b)k+\theta \|k\|=t f'(h)\delta g(a,h)+\tilde{\theta}\|t\|f'(b)+\theta \|k\|.

To complete the proof we merely have to show that the last two terms go to zero. This we shall prove formally in the next post.