### Lecture 1. Notion of Derivative

We shall always assume that ${X,Y}$ are Banach spaces over the complex field ${\mathbb{C}}$. The real number field is denoted by ${\mathbb{R}}$. We shall now explore differential calculus in Banach spaces. It will be instructive to try to understand the theorems in well known Banach spaces like ${\mathbb{R}^{n}}$ (over ${\mathbb{R}}$) or ${\ell^{2}}$ (over ${\mathbb{C}}$) or ${C[a,b]}$ (over ${\mathbb{R}}$). In general there are two notions of derivatives – the Gateaux derivative and the Frechet derivative, where the latter is stronger in the sense that Frechet differentiability implies Gateaux differentiability.

Definition: Let ${f:X \rightarrow Y}$, where ${X,Y}$ are Banach space. For ${a,h\in X}$, define

$\displaystyle \delta f(a,h)=\lim_{t\rightarrow 0}\frac{f(a+th)-f(a)}{t},\quad t\in \mathbb{R}$

provided the limit exists. Then ${\delta f(a,h)}$ is called the directional derivative of ${f}$ at ${a}$ in the direction of ${h}$. If this limit exists for every ${h\in X}$ and if the function ${h\mapsto \delta f(a,h)}$ is linear and continuous in ${h}$, then we say that the function is Gateaux differentiable and the function defined by

$\displaystyle Df(a)(h)=\delta f(a,h)$

is called the Gateaux derivative at ${a}$. Thus the linearity and continuity condition means ${Df(a)\in L(X,Y).}$

[An example to show that the map ${h\mapsto \delta f(a,h)}$ is not always linear, and even if ${Df(a)}$ exists, it may not be continuous at ${a}$.]

Definition: Let ${f:X \rightarrow Y}$, where ${X,Y}$ are Banach space. Let ${U\subset X}$ be open subset. We say that ${f}$ is Frechet differentiable at ${a\in U}$, if there exists a bounded linear map ${A\in L(X,Y)}$ such that

$\displaystyle \lim_{h\rightarrow 0}\frac{\left\|f(a+h)-f(a)-A(h)\right\|}{\left\|h\right\|}=0$

Some remarks should be in order, which we leave as simple exercises. The Frechet derivative is unique, and Frechet differentiability implies continuity. If ${f'(a)}$ exists, then also ${Df(a)}$ exists, and we have

$\displaystyle Df(a,h)=f'(a)h.$

Usually when we say ${f}$ is differentiable, we shall mean Frechet differentiable. [An example for which the Gateaux derivative exists but Frechet derivative fails to exist.]

Our next step is to prove the mean value theorem.

Theorem: Let ${f:X\rightarrow Y}$, and suppose ${\delta f(a+t(b-a),b-a)}$ exists for all ${t\in[0,1]}$, and is a continuous map of ${t}$, then

$\displaystyle f(b)-f(a)=\int_{0}^{1}\delta f(a+t(b-a),b-a) dt.$

Proof: Let ${\phi\in Y^{*}}$. Define ${g(t)=\phi(f(a+t(b-a),b-a)):[0,1]\rightarrow\mathbb{R}}$ is differentiable, since it is the composition of two differentiable function. We have ${g'(t)=\phi(\delta f(a+t(b-a),b-a))}$. Using the fundamental theorem of calculus for real valued functions, we get

$g(1)-g(0) = \int_{0}^{1}g'(t) dt =\int_{0}^{1}\phi(\delta f(a+t(b-a),b-a)) dt$

$= \phi\left(\int_{0}^{1}\delta f(a+t(b-a),b-a) dt\right)$

Also, ${g(1)-g(0)=\phi(f(b)-f(a))}$, so altogether we get,

$\displaystyle \phi\left(f(b)-f(a)-\int_{0}^{1}\delta f(a+t(b-a),b-a) dt\right)=0.$

Since this holds for all ${\phi\in Y^{*}}$, we get the desired result.

Notice further that

$\displaystyle \|f(b)-f(a)\|\leq \int_{0}^{1}\|\delta f(a+t(b-a),b-a)\| dt\leq \sup_{t\in[0,1]}\|\delta f(a+t(b-a),b-a)\|.$

This is referred to as the mean value theorem in higher dimensions. The usual mean value theorem does not hold in higher dimensions as is seen by this example. Define ${f(t)=(\cos(t),\sin(t))}$ for ${t\in[0,2\pi]}$. Then ${f}$ is differentiable, and ${f(2\pi)-f(0)=0}$, but there is no such ${t\in(0,1)}$ for which ${f'(t)=(-\sin(t),\cos(t))}$ is zero.

Our next step is to generalize the chain rule. However we shall not prove the rule using strong hypotheses that both functions in a composition are Frechet differentiable. Instead we have –

Theorem: Let ${a\in X, g(a)=b\in Y}$. Suppose ${f}$ is Frechet differentiable at ${b}$ and ${g}$ is having directional derivative in the direction ${h}$ at ${a}$, then ${f\circ g}$ has a directional derivative at ${a}$, and

$\displaystyle \delta(f\circ g)(a,h)=f'(b)\delta g(a,h).$

Before plunging into a proof, let us have an informal discussion. On the right hand side of the above equation, we have a linear transformation ${f'(b):Y\rightarrow Z}$ acting on an element of Y – ${\delta g(a,b)}$. Let us see what we have got. ${g}$ has directional derivative at ${a}$ in the direction of ${h}$. This translates into

$\displaystyle \lim_{t\rightarrow 0}\frac{g(a+th)-g(a)}{t}=\delta g(a,h),$

that is given ${\epsilon>0}$ there exists ${\delta>0}$ such that if ${\left|t\right|<\eta}$ we have

$\displaystyle \|g(a+th)-g(a)-t\delta g(a,h)\|<\epsilon\left| t\right|.$

${f}$ is Frechet differentiable at ${b=g(a)}$, so

$\displaystyle \|f(b+k)-f(b)-Ak\|\leq\theta\|k\|,$

where the right hand side goes to zero as ${\|k\|\rightarrow 0}$. We may take

$\displaystyle k=g(a+th)-g(a)=t\delta g(a,h)+\tilde{\theta}\left|t\right|,$

which certainly goes to zero at ${t\rightarrow 0}$. If we plug this in the equation of ${f}$, we’ll get

$\displaystyle f(b+k)-f(b)=f'(b)k+\theta \|k\|=t f'(h)\delta g(a,h)+\tilde{\theta}\|t\|f'(b)+\theta \|k\|.$

To complete the proof we merely have to show that the last two terms go to zero. This we shall prove formally in the next post.