Basic calculus

[math] \newcommand{\mathds}{\mathbb}[/math]

5a. Real analysis

We discuss in what follows some applications of the theory that we developed above, for the most to questions in analysis. The idea will be that the functions of several variables [math]f:\mathbb R^N\to\mathbb R^M[/math] can be locally approximated by linear maps, in the same way as the functions [math]f:\mathbb R\to\mathbb R[/math] can be locally approximated by using derivatives:

[[math]] f(x+t)\simeq f(x)+f'(x)t\quad,\quad f'(x)\in M_{M\times N}(\mathbb R) [[/math]]

There are many things that can be said here, and we will be quite brief. As a plan for this chapter and the next one, we would like to quickly review the one-variable calculus, then develop the basics of multivariable calculus, and then get introduced to the Gaussian laws, and to probability theory in general. The instructions being as follows:

(1) In case all this is totally new to you, it is better to stop at this point with reading the present book, and quickly read some calculus. There are plenty of good books here, a standard choice being for instance the books of Lax-Terrell , .

(2) In case you know a bit about all this, stay with us. But have the books of Rudin , nearby, for things not explained in what follows. And have a probability book nearby too, such as Feller or Durrett , for some extra help with probability.

(3) Finally, in the case you know well analysis, have of course some fun in quickly reading the material below. But, in parallel to this, upgrade of course, say by learning some differential geometry, for instance from the books of do Carmo , .

Getting started now, let us first discuss the simplest case, [math]f:\mathbb R\to\mathbb R[/math]. Here we have the following result, which is the starting point for everything in analysis:

Theorem

Any function of one variable [math]f:\mathbb R\to\mathbb R[/math] is locally affine,

[[math]] f(x+t)\simeq f(x)+f'(x)t [[/math]]

with [math]f'(x)\in\mathbb R[/math] being the derivative of [math]f[/math] at the point [math]x[/math], given by

[[math]] f'(x)=\lim_{t\to0}\frac{f(x+t)-f(x)}{t} [[/math]]

provided that this latter limit converges indeed.

Show Proof

Assume indeed that the limit in the statement converges. By multiplying by [math]t[/math], we obtain that we have, once again in the [math]t\to0[/math] limit:

[[math]] f(x+t)-f(x)\simeq f'(x)t [[/math]]

Thus, we are led to the conclusion in the statement.

■

As a first example, the derivatives of the power functions are as follows:

Proposition

We have the differentiation formula

[[math]] (x^p)'=px^{p-1} [[/math]]

valid for any exponent [math]p\in\mathbb R[/math].

Show Proof

We can do this in three steps, as follows:

(1) In the case [math]p\in\mathbb N[/math] we can use the binomial formula, which gives, as desired:

[[math]] \begin{eqnarray*} (x+t)^p &=&\sum_{k=0}^n\binom{p}{k}x^{p-k}t^k\\ &=&x^p+px^{p-1}t+\ldots+t^p\\ &\simeq&x^p+px^{p-1}t \end{eqnarray*} [[/math]]

(2) In the general case [math]p\in\mathbb Q[/math], we can write [math]p=m/n[/math], with [math]m\in\mathbb N[/math] and [math]n\in\mathbb Z[/math], and again by using the binomial formula, we obtain, as desired:

[[math]] \begin{eqnarray*} (x+t)^{m/n}-x^{m/n} &=&\frac{(x+t)^m-x^m}{(x+t)^{m(n-1)/n}+\ldots+x^{m(n-1)/n}}\\ &\simeq&\frac{mx^{m-1}t}{nx^{m(n-1)/n}}\\ &=&\frac{m}{n}\cdot x^{m/n-1}\cdot t \end{eqnarray*} [[/math]]

(3) In the general case now, where [math]p\in\mathbb R[/math] is real, the same formula holds, namely [math](x^p)'=px^{p-1}[/math], by using what we found above, and a continuity argument.

■

There are many other computations that can be done, and we will be back to this later. Now back to the general level, let us record here the following key result:

Theorem

The derivatives are subject to the following rules:

• Leibnitz rule: [math](fg)'=f'g+fg'[/math].
• Chain rule: [math](f\circ g)'=f'(g)g'[/math].

Show Proof

Both formulae follow from the definition of the derivative, as follows:

(1) Regarding products, we have the following computation:

[[math]] \begin{eqnarray*} (fg)(x+t) &=&f(x+t)g(x+t)\\ &\simeq&(f(x)+f'(x)t)(g(x)+g'(x)t)\\ &\simeq&f(x)g(x)+(f'(x)g(x)+f(x)g'(x))t \end{eqnarray*} [[/math]]

(2) Regarding compositions, we have the following computation:

[[math]] \begin{eqnarray*} (f\circ g)(x+t) &=&f(g(x+t))\\ &\simeq&f(g(x)+g'(x)t)\\ &\simeq&f(g(x))+f'(g(x))g'(x)t \end{eqnarray*} [[/math]]

Thus, we are led to the conclusions in the statement.

■

There are many applications of the derivative, and we have for instance:

Proposition

The local minima and maxima of a differentiable function [math]f:\mathbb R\to\mathbb R[/math] appear at the points [math]x\in\mathbb R[/math] where:

[[math]] f'(x)=0 [[/math]]

However, the converse of this fact is not true in general.

Show Proof

The first assertion is clear from the formula in Theorem 5.1, namely:

[[math]] f(x+t)\simeq f(x)+f'(x)t [[/math]]

As for the converse, the simplest counterexample is [math]f(x)=x^3[/math], at [math]x=0[/math].

■

At a more advanced level now, we have the following result:

Theorem

Any function of one variable [math]f:\mathbb R\to\mathbb R[/math] is locally quadratic,

[[math]] f(x+t)\simeq f(x)+f'(x)t+\frac{f''(x)}{2}\,t^2 [[/math]]

where [math]f''(x)[/math] is the derivative of the function [math]f':\mathbb R\to\mathbb R[/math] at the point [math]x[/math].

Show Proof

This is something quite intuitive, when thinking geometrically. In practice, we can use L'H\^opital's rule, stating that the [math]0/0[/math] type limits can be computed as:

[[math]] \frac{f(x)}{g(x)}\simeq\frac{f'(x)}{g'(x)} [[/math]]

Observe that this formula holds indeed, as an application of Theorem 5.1. Now by using this, if we denote by [math]\varphi(t)\simeq P(t)[/math] the formula to be proved, we have:

[[math]] \begin{eqnarray*} \frac{\varphi(t)-P(t)}{t^2} &\simeq&\frac{\varphi'(t)-P'(t)}{2t}\\ &\simeq&\frac{\varphi''(t)-P''(t)}{2}\\ &=&\frac{f''(x)-f''(x)}{2}\\ &=&0 \end{eqnarray*} [[/math]]

Thus, we are led to the conclusion in the statement.

■

The above result substantially improves Theorem 5.1, and there are many applications of it. We can improve for instance Proposition 5.4, as follows:

Proposition

The local minima and maxima of a twice differentiable function [math]f:\mathbb R\to\mathbb R[/math] appear at the points [math]x\in\mathbb R[/math] where

[[math]] f'(x)=0 [[/math]]

with the local minima corresponding to the case [math]f'(x)\geq0[/math], and with the local maxima corresponding to the case [math]f''(x)\leq0[/math].

Show Proof

The first assertion is something that we already know. As for the second assertion, we can use the formula in Theorem 5.5, which in the case [math]f'(x)=0[/math] reads:

[[math]] f(x+t)\simeq f(x)+\frac{f''(x)}{2}\,t^2 [[/math]]

Indeed, assuming [math]f''(x)\neq 0[/math], it is clear that the condition [math]f''(x) \gt 0[/math] will produce a local minimum, and that the condition [math]f''(x) \lt 0[/math] will produce a local maximum.

■

We can further develop the above method, at order 3, at order 4, and so on, the ultimate result on the subject, called Taylor formula, being as follows:

Theorem

Any function [math]f:\mathbb R\to\mathbb R[/math] can be locally approximated as

[[math]] f(x+t)=\sum_{k=0}^\infty\frac{f^{(k)}(x)}{k!}\,t^k [[/math]]

where [math]f^{(k)}(x)[/math] are the higher derivatives of [math]f[/math] at the point [math]x[/math].

Show Proof

We use the same method as in the proof of Theorem 5.5. If we denote by [math]\varphi(t)\simeq P(t)[/math] the approximation to be proved, we have:

[[math]] \begin{eqnarray*} \frac{\varphi(t)-P(t)}{t^n} &\simeq&\frac{\varphi'(t)-P'(t)}{nt^{n-1}}\\ &\simeq&\frac{\varphi''(t)-P''(t)}{n(n-1)t^{n-2}}\\ &\vdots&\\ &\simeq&\frac{\varphi^{(n)}(t)-P^{(n)}(t)}{n!}\\ &=&0 \end{eqnarray*} [[/math]]

Thus, we are led to the conclusion in the statement.

■

As a basic application of the Taylor formula, we have:

Theorem

We have the following formulae,

[[math]] \sin x=\sum_{l=0}^\infty(-1)^l\frac{x^{2l+1}}{(2l+1)!}\quad,\quad \cos x=\sum_{l=0}^\infty(-1)^l\frac{x^{2l}}{(2l)!} [[/math]]

as well as the following formulae,

[[math]] e^x=\sum_{k=0}^\infty\frac{x^k}{k!}\quad,\quad \log(1+x)=\sum_{k=0}^\infty(-1)^{k+1}\frac{x^k}{k} [[/math]]

as Taylor series, and in general as well, with [math]|x| \lt 1[/math] needed for [math]\log[/math].

Show Proof

There are several statements here, the proofs being as follows:

(1) Regarding [math]\sin[/math] and [math]\cos[/math], we can use here the following well-known formulae:

[[math]] \sin(x+y)=\sin x\cos y+\cos x\sin y [[/math]]

[[math]] \cos(x+y)=\cos x\cos y-\sin x\sin y [[/math]]

With these formulae in hand we can appproximate both [math]\sin[/math] and [math]\cos[/math], and we get:

[[math]] (\sin x)'=\cos x\quad,\quad (\cos x)'=-\sin x [[/math]]

Thus, we can differentiate [math]\sin[/math] and [math]\cos[/math] as many times as we want to, and so we can compute the corresponding Taylor series, and we obtain the formulae in the statement.

(2) Regarding [math]\exp[/math] and [math]\log[/math], here the needed formulae, which lead to the formulae in the statement for the corresponding Taylor series, are as follows:

[[math]] (e^x)'=e^x\quad,\quad (\log x)'=x^{-1}\quad,\quad (x^p)'=px^{p-1} [[/math]]

(3) Finally, the fact that the formulae in the statement extend beyond the small [math]x[/math] setting, coming from Taylor series, is something standard too.

■

As another basic application of the Taylor formula, we have:

Theorem

We have the following generalized binomial formula, with [math]p\in\mathbb R[/math],

[[math]] (x+t)^p=\sum_{k=0}^\infty\binom{p}{k}x^{p-k}t^k [[/math]]

with the generalized binomial coefficients being given by the formula

[[math]] \binom{p}{k}=\frac{p(p-1)\ldots(p-k+1)}{k!} [[/math]]

valid for any [math]|t| \lt |x|[/math]. With [math]p\in\mathbb N[/math], we recover the usual binomial formula.

Show Proof

Consider indeed the following function:

[[math]] f(x)=x^p [[/math]]

The derivatives at [math]x=1[/math] are then given by the following formula:

[[math]] f^{(k)}(1)=p(p-1)\ldots(p-k+1) [[/math]]

Thus, the Taylor approximation at [math]x=1[/math] is as follows:

[[math]] f(1+t)=\sum_{k=0}^\infty\frac{p(p-1)\ldots(p-k+1)}{k!}\,t^k [[/math]]

But this is exactly our generalized binomial formula, so we are done with the case where [math]t[/math] is small. With a bit more care, we obtain that this holds for any [math]|t| \lt 1[/math].

■

We can see from the above the power of the Taylor formula. As an application now of our generalized binomial formula, we can extract square roots, as follows:

Proposition

We have the following formula,

[[math]] \sqrt{1+t}=1-2\sum_{k=1}^\infty C_{k-1}\left(\frac{-t}{4}\right)^k [[/math]]

with [math]C_k=\frac{1}{k+1}\binom{2k}{k}[/math] being the Catalan numbers. Also, we have

[[math]] \frac{1}{\sqrt{1+t}}=\sum_{k=0}^\infty D_k\left(\frac{-t}{4}\right)^k [[/math]]

with [math]D_k=\binom{2k}{k}[/math] being the central binomial coefficients.

Show Proof

The above formulae both follow from Theorem 5.9, as follows:

(1) At [math]p=1/2[/math], the generalized binomial coefficients are:

[[math]] \begin{eqnarray*} \binom{1/2}{k} &=&\frac{1/2(-1/2)\ldots(3/2-k)}{k!}\\ &=&(-1)^{k-1}\frac{1\cdot 3\cdot 5\ldots(2k-3)}{2^kk!}\\ &=&(-1)^{k-1}\frac{(2k-2)!}{2^{k-1}(k-1)!2^kk!}\\ &=&\frac{(-1)^{k-1}}{2^{2k-1}}\cdot\frac{1}{k}\binom{2k-2}{k-1}\\ &=&-2\left(\frac{-1}{4}\right)^kC_{k-1} \end{eqnarray*} [[/math]]

(2) At [math]p=-1/2[/math], the generalized binomial coefficients are:

[[math]] \begin{eqnarray*} \binom{-1/2}{k} &=&\frac{-1/2(-3/2)\ldots(1/2-k)}{k!}\\ &=&(-1)^k\frac{1\cdot 3\cdot 5\ldots(2k-1)}{2^kk!}\\ &=&(-1)^k\frac{(2k)!}{2^kk!2^kk!}\\ &=&\frac{(-1)^k}{4^k}\binom{2k}{k}\\ &=&\left(\frac{-1}{4}\right)^kD_k \end{eqnarray*} [[/math]]

Thus, we obtain the formulae in the statement.

■

Let us discuss as well the basics of integration theory. We will be very brief here, by insisting on the main concepts, and skipping technicalities. We first have:

Theorem

We have the Riemann integration formula,

[[math]] \int_a^bf(x)dx=(b-a)\times\lim_{N\to\infty}\frac{1}{N}\sum_{k=1}^Nf\left(a+\frac{b-a}{N}\cdot k\right) [[/math]]

which can serve as a formal definition for the integral.

Show Proof

Assume indeed that we are given a continuous function [math]f:[a,b]\to\mathbb R[/math], and let us try to compute the signed area below its graph, called integral and denoted [math]\int_a^bf(x)dx[/math]. Obviously, this signed area equals [math]b-a[/math] times the average of the function on [math][a,b][/math], and we are led to the following formula, with [math]x_1,\ldots,x_N\in[a,b][/math] being randomly chosen:

[[math]] \int_a^bf(x)dx=(b-a)\times\lim_{N\to\infty}\frac{1}{N}\sum_{k=1}^Nf(x_i) [[/math]]

This is the so-called Monte Carlo integration formula, which is extremely useful in practice, and is used by scientists, engineers and computers. However, for theoretical purposes, it is better assume that [math]x_1,\ldots,x_N\in[a,b][/math] are uniformly distributed. With this choice, which works of course too, the formula that we obtain is as follows:

[[math]] \int_a^bf(x)dx=(b-a)\times\lim_{N\to\infty}\frac{1}{N}\sum_{k=1}^Nf\left(a+\frac{b-a}{N}\cdot k\right) [[/math]]

Observe that this latter formula can be alternatively written as follows, which makes it clear that the formula holds indeed, as an approximation of an area by rectangles:

[[math]] \int_a^bf(x)dx=\lim_{N\to\infty}\sum_{k=1}^N\frac{b-a}{N}\times f\left(a+\frac{b-a}{N}\cdot k\right) [[/math]]

In any case, we have obtained the formula in the statement, and we are done.

■

The above was of course extremely brief, and for more on all this, including further functions that can be integrated, we refer to Rudin , or Lax-Terrell , . As a useful piece of advice, however, always keep in mind the Monte Carlo formula, briefly evoked above, because that is the real thing, in connection with anything integration.

The derivatives and integrals are related in several subtle ways, and we have:

Theorem

We have the following formulae, called fundamental theorem of calculus, integration by parts formula, and change of variable formula,

[[math]] \int_a^bF'(x)dx=\Big[F\Big]_a^b [[/math]]

[[math]] \int_a^b(f'g+fg')(x)dx=\Big[fg\Big]_a^b [[/math]]

[[math]] \int_a^bf(x)dx=\int_{\varphi^{-1}(a)}^{\varphi^{-1}(b)}f(\varphi(t))\varphi'(t)dt [[/math]]

with the convention [math][F]_a^b=F(b)-F(a)[/math], for the first two formulae.

Show Proof

Again, this is standard, the idea being that the first formula is clear from the area interpretation of the integral, and that the second and third formulae follow from it, by integrating respectively the Leibnitz rule and the chain rule from Theorem 5.3.

■

So long for one-variable calculus. For more on all this, we refer to any basic analysis book, good choices here being the books of Lax-Terrell , , or Rudin , .

5b. Several variables

Let us discuss now what happens in several variables. At order 1, the situation is quite similar to the one in 1 variable, but this time involving matrices, as follows:

Theorem

Any function [math]f:\mathbb R^N\to\mathbb R^M[/math] can be locally approximated as

[[math]] f(x+t)\simeq f(x)+f'(x)t [[/math]]

with [math]f'(x)[/math] being by definition the matrix of partial derivatives at [math]x[/math],

[[math]] f'(x)=\left(\frac{df_i}{dx_j}(x)\right)_{ij}\in M_{M\times N}(\mathbb R) [[/math]]

acting on the vectors [math]t\in\mathbb R^N[/math] by usual multiplication.

Show Proof

As a first observation, the formula in the statement makes sense indeed, as an equality, or rather approximation, of vectors in [math]\mathbb R^M[/math], as follows:

[[math]] f\begin{pmatrix}x_1+t_1\\ \vdots\\ x_N+t_N\end{pmatrix} \simeq f\begin{pmatrix}x_1\\ \vdots\\ x_N\end{pmatrix} +\begin{pmatrix} \frac{df_1}{dx_1}(x)&\ldots&\frac{df_1}{dx_N}(x)\\ \vdots&&\vdots\\ \frac{df_M}{dx_1}(x)&\ldots&\frac{df_M}{dx_N}(x) \end{pmatrix}\begin{pmatrix}t_1\\ \vdots\\ t_N\end{pmatrix} [[/math]]

In order to prove now this formula, which does make sense, the idea is as follows:

(1) First of all, at [math]N=M=1[/math] what we have is a usual 1-variable function [math]f:\mathbb R\to\mathbb R[/math], and the formula in the statement is something that we know well, namely:

[[math]] f(x+t)\simeq f(x)+f'(x)t [[/math]]

(2) Let us discuss now the case [math]N=2,M=1[/math]. Here what we have is a function [math]f:\mathbb R^2\to\mathbb R[/math], and by using twice the basic approximation result from (1), we obtain:

[[math]] \begin{eqnarray*} f\binom{x_1+t_1}{x_2+t_2} &\simeq&f\binom{x_1+t_1}{x_2}+\frac{df}{dx_2}(x)t_2\\ &\simeq&f\binom{x_1}{x_2}+\frac{df}{dx_1}(x)t_1+\frac{df}{dx_2}(x)t_2\\ &=&f\binom{x_1}{x_2}+\begin{pmatrix}\frac{df}{dx_1}(x)&\frac{df}{dx_2}(x)\end{pmatrix}\binom{t_1}{t_2} \end{eqnarray*} [[/math]]

(3) More generally, we can deal in this way with the general case [math]M=1[/math], with the formula here, obtained via a straightforward recurrence, being as follows:

[[math]] \begin{eqnarray*} f\begin{pmatrix}x_1+t_1\\ \vdots\\ x_N+t_N\end{pmatrix} &\simeq&f\begin{pmatrix}x_1\\ \vdots\\ x_N\end{pmatrix}+\frac{df}{dx_1}(x)t_1+\ldots+\frac{df}{dx_N}(x)t_N\\ &=&f\begin{pmatrix}x_1\\ \vdots\\ x_N\end{pmatrix}+ \begin{pmatrix}\frac{df}{dx_1}(x)&\ldots&\frac{df}{dx_N}(x)\end{pmatrix} \begin{pmatrix}t_1\\ \vdots\\ t_N\end{pmatrix} \end{eqnarray*} [[/math]]

(4) But this gives the result in the case where both [math]N,M\in\mathbb N[/math] are arbitrary too. Indeed, consider a function [math]f:\mathbb R^N\to\mathbb R^M[/math], and let us write it as follows:

[[math]] f=\begin{pmatrix}f_1\\ \vdots\\ f_M\end{pmatrix} [[/math]]

We can apply (3) to each of the components [math]f_i:\mathbb R^N\to\mathbb R[/math], and we get:

[[math]] f_i\begin{pmatrix}x_1+t_1\\ \vdots\\ x_N+t_N\end{pmatrix} \simeq f_i\begin{pmatrix}x_1\\ \vdots\\ x_N\end{pmatrix}+ \begin{pmatrix}\frac{df_i}{dx_1}(x)&\ldots&\frac{df_i}{dx_N}(x)\end{pmatrix} \begin{pmatrix}t_1\\ \vdots\\ t_N\end{pmatrix} [[/math]]

But this collection of [math]M[/math] formulae tells us precisely that the following happens, as an equality, or rather approximation, of vectors in [math]\mathbb R^M[/math]:

[[math]] f\begin{pmatrix}x_1+t_1\\ \vdots\\ x_N+t_N\end{pmatrix} \simeq f\begin{pmatrix}x_1\\ \vdots\\ x_N\end{pmatrix} +\begin{pmatrix} \frac{df_1}{dx_1}(x)&\ldots&\frac{df_1}{dx_N}(x)\\ \vdots&&\vdots\\ \frac{df_M}{dx_1}(x)&\ldots&\frac{df_M}{dx_N}(x) \end{pmatrix}\begin{pmatrix}t_1\\ \vdots\\ t_N\end{pmatrix} [[/math]]

Thus, we are led to the conclusion in the statement.

■

Generally speaking, Theorem 5.13 is what we need to know for upgrading from calculus to multivariable calculus. As a standard result here, we have:

Theorem

We have the chain derivative formula

[[math]] (f\circ g)'(x)=f'(g(x))\cdot g'(x) [[/math]]

as an equality of matrices.

Show Proof

Consider indeed a composition of functions, as follows:

[[math]] f:\mathbb R^N\to\mathbb R^M\quad,\quad g:\mathbb R^K\to\mathbb R^N\quad,\quad f\circ g:\mathbb R^K\to\mathbb R^M [[/math]]

According to Theorem 5.13, the derivatives of these functions are certain linear maps, corresponding to certain rectangular matrices, as follows:

[[math]] f'(g(x))\in M_{M\times N}(\mathbb R)\quad,\quad g'(x)\in M_{N\times K}(\mathbb R)\quad\quad (f\circ g)'(x)\in M_{M\times K}(\mathbb R) [[/math]]

Thus, our formula makes sense indeed. As for proof, this comes from:

[[math]] \begin{eqnarray*} (f\circ g)(x+t) &=&f(g(x+t))\\ &\simeq&f(g(x)+g'(x)t)\\ &\simeq&f(g(x))+f'(g(x))g'(x)t \end{eqnarray*} [[/math]]

Thus, we are led to the conclusion in the statement.

■

Regarding now the higher derivatives, the situation here is more complicated. Let us record, however, the following fundamental result, happening at order 2, and which does the job, the job in analysis being usually that of finding the minima or maxima:

Theorem

Given a function [math]f:\mathbb R^N\to\mathbb R[/math], construct its Hessian, as being:

[[math]] f''(x)=\left(\frac{d^2f}{dx_idx_j}(x)\right)_{ij} [[/math]]

We have then the following order [math]2[/math] approximation of [math]f[/math] around a given [math]x\in\mathbb R^N[/math],

[[math]] f(x+t)\simeq f(x)+f'(x)t+\frac{ \lt f''(x)t,t \gt }{2} [[/math]]

relating the positivity properties of [math]f''[/math] to the local minima and maxima of [math]f[/math].

Show Proof

This is something very standard, the idea being as follows:

(1) At [math]N=1[/math] the Hessian matrix is the [math]1\times1[/math] matrix having as entry the usual [math]f''(x)[/math], and the formula in the statement is something that we know well, namely:

[[math]] f(x+t)\simeq f(x)+f'(x)t+\frac{f''(x)t^2}{2} [[/math]]

(2) In general, our claim is that the formula in the statement follows from the one-variable formula above, applied to the restriction of [math]f[/math] to the following segment in [math]\mathbb R^N[/math]:

[[math]] I=[x,x+t] [[/math]]

To be more precise, let [math]y\in\mathbb R^N[/math], and consider the following function, with [math]r\in\mathbb R[/math]:

[[math]] g(r)=f(x+ry) [[/math]]

We know from (1) that the Taylor formula for [math]g[/math], at the point [math]r=0[/math], reads:

[[math]] g(r)\simeq g(0)+g'(0)r+\frac{g''(0)r^2}{2} [[/math]]

And our claim is that, with [math]t=ry[/math], this is precisely the formula in the statement.

(3) So, let us see if our claim is correct. By using the chain rule, we have the following formula, with on the right, as usual, a row vector multiplied by a column vector:

[[math]] g'(r)=f'(x+ry)\cdot y [[/math]]

By using again the chain rule, we can compute the second derivative as well:

[[math]] \begin{eqnarray*} g''(r) &=&(f'(x+ry)\cdot y)'\\ &=&\left(\sum_i\frac{df}{dx_i}(x+ry)\cdot y_i\right)'\\ &=&\sum_i\sum_j\frac{d^2f}{dx_idx_j}(x+ry)\cdot\frac{d(x+ry)_j}{dr}\cdot y_i\\ &=&\sum_i\sum_j\frac{d^2f}{dx_idx_j}(x+ry)\cdot y_iy_j\\ &=& \lt f''(x+ry)y,y \gt \end{eqnarray*} [[/math]]

(4) Time now to conclude. We know that we have [math]g(r)=f(x+ry)[/math], and according to our various computations above, we have the following formulae:

[[math]] g(0)=f(x)\quad,\quad g'(0)=f'(x)\quad,\quad g''(0)= \lt f''(x)y,y \gt [[/math]]

Buit with this data in hand, the usual Taylor formula for our one variable function [math]g[/math], at order 2, at the point [math]r=0[/math], takes the following form, with [math]t=ry[/math]:

[[math]] \begin{eqnarray*} f(x+ry) &\simeq&f(x)+f'(x)ry+\frac{ \lt f''(x)y,y \gt r^2}{2}\\ &=&f(x)+f'(x)t+\frac{ \lt f''(x)t,t \gt }{2} \end{eqnarray*} [[/math]]

Thus, we have obtained the formula in the statement.

■

Getting now to integration, as a key result here, we have:

Theorem

Given a transformation [math]\varphi=(\varphi_1,\ldots,\varphi_N)[/math], we have

[[math]] \int_Ef(x)dx=\int_{\varphi^{-1}(E)}f(\varphi(t))|J_\varphi(t)|dt [[/math]]

with the [math]J_\varphi[/math] quantity, called Jacobian, being given by

[[math]] J_\varphi(t)=\det\left[\left(\frac{d\varphi_i}{dx_j}(t)\right)_{ij}\right] [[/math]]

and with this generalizing the [math]1[/math]-variable formula that we know well.

Show Proof

This is something quite tricky, the idea being as follows:

(1) Observe first that this generalizes indeed the change of variable formula in 1 dimension, from Theorem 5.12, the point here being that the absolute value on the derivative appears as to compensate for the lack of explicit bounds for the integral.

(2) In general now, we can first argue that, the formula in the statement being linear in [math]f[/math], we can assume [math]f=1[/math]. Thus we want to prove [math]vol(E)=\int_{\varphi^{-1}(E)}|J_\varphi(t)|dt[/math], and with [math]D={\varphi^{-1}(E)}[/math], this amounts in proving [math]vol(\varphi(D))=\int_D|J_\varphi(t)|dt[/math].

(3) Now since this latter formula is additive with respect to [math]D[/math], it is enough to prove that [math]vol(\varphi(D))=\int_D J_\varphi(t)dt[/math], for small cubes [math]D[/math], and assuming [math]J_\varphi \gt 0[/math]. But this follows by using the definition of the determinant as a volume, as in chapter 2.

(4) The details and computations however are quite non-trivial, and can be found for instance in Rudin . So, please read Rudin. With this, reading the complete proof of the present theorem from Rudin, being part of the standard math experience.

■

5c. Volumes of spheres

We can discuss now some more advanced questions, related to the computation of volumes of the spheres, and to the integration over spheres. Before anything, do you know what [math]\pi[/math] is? I bet not, or at least my students usually don't. So, let me teach you:

Theorem

Assuming that the length of the unit circle is

[[math]] L=2\pi [[/math]]

it follows that the area of the unit disk is

[[math]] A=\pi [[/math]]

and so the two possible definitions of [math]\pi[/math] are indeed equivalent.

Show Proof

This follows by drawing polygons, and taking the [math]N\to\infty[/math] limit. To be more precise, let us cut the disk as a pizza, into [math]N[/math] slices, and leave aside the rounded parts:

[[math]] \xymatrix@R=23pt@C=8pt{ &\circ\ar@{-}[rr]\ar@{-}[dl]\ar@{-}[dr]&&\circ\ar@{-}[dl]\ar@{-}[dr]\\ \circ\ar@{-}[rr]&&\circ\ar@{-}[rr]&&\circ\\ &\circ\ar@{-}[rr]\ar@{-}[ul]\ar@{-}[ur]&&\circ\ar@{-}[ul]\ar@{-}[ur] } [[/math]]

The area to be eaten can be then computed as follows, where [math]H[/math] is the height of the slices, [math]S[/math] is the length of their sides, and [math]P=NS[/math] is the total length of the sides:

[[math]] A =N\times \frac{HS}{2} =\frac{HP}{2} \simeq\frac{1\times L}{2} =\pi [[/math]]

Thus, we are led to the conclusion in the statement.

■

In [math]N[/math] dimensions now, things are more complicated, and we will need spherical coordinates, in order to deal with such questions. Let us start with:

Theorem

We have polar coordinates in [math]2[/math] dimensions,

[[math]] \begin{cases} x\!\!\!&=\ r\cos t\\ y\!\!\!&=\ r\sin t \end{cases} [[/math]]

the corresponding Jacobian being [math]J=r[/math].

Show Proof

This is something elementary, the Jacobian being given by:

[[math]] \begin{eqnarray*} J &=&\begin{vmatrix} \cos t&-r\sin t\\ \sin t&r\cos t \end{vmatrix}\\ &=&r\cos^2t+r\sin^2t\\ &=&r \end{eqnarray*} [[/math]]

Thus, we have indeed the formula in the statement.

■

In 3 dimensions now the formula is similar, as follows:

Theorem

We have spherical coordinates in [math]3[/math] dimensions,

[[math]] \begin{cases} x\!\!\!&=\ r\cos s\\ y\!\!\!&=\ r\sin s\cos t\\ z\!\!\!&=\ r\sin s\sin t \end{cases} [[/math]]

the corresponding Jacobian being [math]J(r,s,t)=r^2\sin s[/math].

Show Proof

The fact that we have indeed spherical coordinates is clear. Regarding now the Jacobian, this is given by the following formula:

[[math]] \begin{eqnarray*} J(r,s,t) &=&\begin{vmatrix} \cos s&-r\sin s&0\\ \sin s\cos t&r\cos s\cos t&-r\sin s\sin t\\ \sin s\sin t&r\cos s\sin t&r\sin s\cos t \end{vmatrix}\\ &=&r^2\sin s\sin t \begin{vmatrix}\cos s&-r\sin s\\ \sin s\sin t&r\cos s\sin t\end{vmatrix} +r\sin s\cos t\begin{vmatrix}\cos s&-r\sin s\\ \sin s\cos t&r\cos s\cos t\end{vmatrix}\\ &=&r\sin s\sin^2 t \begin{vmatrix}\cos s&-r\sin s\\ \sin s&r\cos s\end{vmatrix} +r\sin s\cos^2 t\begin{vmatrix}\cos s&-r\sin s\\ \sin s&r\cos s\end{vmatrix}\\ &=&r\sin s(\sin^2t+\cos^2t)\begin{vmatrix}\cos s&-r\sin s\\ \sin s&r\cos s\end{vmatrix}\\ &=&r\sin s\times 1\times r\\ &=&r^2\sin s \end{eqnarray*} [[/math]]

Thus, we have indeed the formula in the statement.

■

Let us work out now the spherical coordinate formula in [math]N[/math] dimensions. The result here, which generalizes those at [math]N=2,3[/math], is as follows:

Theorem

We have spherical coordinates in [math]N[/math] dimensions,

[[math]] \begin{cases} x_1\!\!\!&=\ r\cos t_1\\ x_2\!\!\!&=\ r\sin t_1\cos t_2\\ \vdots\\ x_{N-1}\!\!\!&=\ r\sin t_1\sin t_2\ldots\sin t_{N-2}\cos t_{N-1}\\ x_N\!\!\!&=\ r\sin t_1\sin t_2\ldots\sin t_{N-2}\sin t_{N-1} \end{cases} [[/math]]

the corresponding Jacobian being given by the following formula:

[[math]] J(r,t)=r^{N-1}\sin^{N-2}t_1\sin^{N-3}t_2\,\ldots\,\sin^2t_{N-3}\sin t_{N-2} [[/math]]

Show Proof

As before, the fact that we have spherical coordinates is clear. Regarding now the Jacobian, also as before, by developing over the last column, we have:

[[math]] \begin{eqnarray*} J_N &=&r\sin t_1\ldots\sin t_{N-2}\sin t_{N-1}\times \sin t_{N-1}J_{N-1}\\ &+&r\sin t_1\ldots \sin t_{N-2}\cos t_{N-1}\times\cos t_{N-1}J_{N-1}\\ &=&r\sin t_1\ldots\sin t_{N-2}(\sin^2 t_{N-1}+\cos^2 t_{N-1})J_{N-1}\\ &=&r\sin t_1\ldots\sin t_{N-2}J_{N-1} \end{eqnarray*} [[/math]]

Thus, we obtain the formula in the statement, by recurrence.

■

As a comment here, the above convention for spherical coordinates, which is particularly beautiful, is one among many. Physicists for instance like to write things a bit upside down, and the same is actually true for physicists' notation for scalar products, which is [math] \lt x,y \gt =\sum_i\bar{x}_iy_i[/math], again upside down, and for many other things. If this bothers you, I can only recommend my physics book , written using mathematicians' notations.

By the way, talking physics, again and I insist, you should learn some, normally from Feynman , , , or Griffiths , , , or Weinberg , , . The point indeed is that using spherical coordinates, while being something usually labelled as “unconceptual”, and avoided by mathematicians, is the ABC of physicists, who use it all the time. Want to do some basic electrodynamics computations? Spherical coordinates. Want to solve the hydrogen atom? Spherical coordinates, again. And so on. So, nothing better than learning some physics, in order to get to know, and love, these spherical coordinates. That we will do love in this book, from the bottom of our hearts.

Back to work now, let us compute the volumes of spheres. For this purpose, we must understand how the products of coordinates integrate over spheres. Let us start with the case [math]N=2[/math]. Here the sphere is the unit circle [math]\mathbb T[/math], and with [math]z=e^{it}[/math] the coordinates are [math]\cos t,\sin t[/math]. We can first integrate arbitrary powers of these coordinates, as follows:

Proposition

We have the following formulae,

[[math]] \int_0^{\pi/2}\cos^pt\,dt=\int_0^{\pi/2}\sin^pt\,dt=\left(\frac{\pi}{2}\right)^{\varepsilon(p)}\frac{p!!}{(p+1)!!} [[/math]]

where [math]\varepsilon(p)=1[/math] if [math]p[/math] is even, and [math]\varepsilon(p)=0[/math] if [math]p[/math] is odd, and where

[[math]] m!!=(m-1)(m-3)(m-5)\ldots [[/math]]

with the product ending at [math]2[/math] if [math]m[/math] is odd, and ending at [math]1[/math] if [math]m[/math] is even.

Show Proof

Let us first compute the integral on the left [math]I_p[/math]. We have:

[[math]] \begin{eqnarray*} (\cos^pt\sin t)' &=&p\cos^{p-1}t(-\sin t)\sin t+\cos^pt\cos t\\ &=&p\cos^{p+1}t-p\cos^{p-1}t+\cos^{p+1}t\\ &=&(p+1)\cos^{p+1}t-p\cos^{p-1}t \end{eqnarray*} [[/math]]

By integrating between [math]0[/math] and [math]\pi/2[/math], we obtain the following formula:

[[math]] (p+1)I_{p+1}=pI_{p-1} [[/math]]

Thus we can compute [math]I_p[/math] by recurrence, and we obtain:

[[math]] \begin{eqnarray*} I_p &=&\frac{p-1}{p}\,I_{p-2}\\ &=&\frac{p-1}{p}\cdot\frac{p-3}{p-2}\,I_{p-4}\\ &=&\frac{p-1}{p}\cdot\frac{p-3}{p-2}\cdot\frac{p-5}{p-4}\,I_{p-6}\\ &&\vdots\\ &=&\frac{p!!}{(p+1)!!}\,I_{1-\varepsilon(p)} \end{eqnarray*} [[/math]]

On the other hand, at [math]p=0[/math] we have the following formula:

[[math]] I_0 =\int_0^{\pi/2}1\,dt =\frac{\pi}{2} [[/math]]

Also, at [math]p=1[/math] we have the following formula:

[[math]] I_1 =\int_0^{\pi/2}\cos t\,dt =1 [[/math]]

Thus, we obtain the result, by recurrence. As for the second formula, regarding [math]\sin t[/math], this follows from the first formula, with the change of variables [math]t=\frac{\pi}{2}-s[/math].

■

We can now compute the volume of the sphere, as follows:

Theorem

The volume of the unit sphere in [math]\mathbb R^N[/math] is given by

[[math]] V=\left(\frac{\pi}{2}\right)^{[N/2]}\frac{2^N}{(N+1)!!} [[/math]]

with the convention

[[math]] N!!=(N-1)(N-3)(N-5)\ldots [[/math]]

with the product ending at [math]2[/math] if [math]N[/math] is odd, and ending at [math]1[/math] if [math]N[/math] is even.

Show Proof

If we denote by [math]B^+[/math] the positive part of the unit sphere, we have:

[[math]] \begin{eqnarray*} V^+ &=&\int_{B^+}1\\ &=&\int_0^1\int_0^{\pi/2}\ldots\int_0^{\pi/2}r^{N-1}\sin^{N-2}t_1\ldots\sin t_{N-2}\,drdt_1\ldots dt_{N-1}\\ &=&\int_0^1r^{N-1}\,dr\int_0^{\pi/2}\sin^{N-2}t_1\,dt_1\ldots\int_0^{\pi/2}\sin t_{N-2}dt_{N-2}\int_0^{\pi/2}1dt_{N-1}\\ &=&\frac{1}{N}\times\left(\frac{\pi}{2}\right)^{[N/2]}\times\frac{(N-2)!!}{(N-1)!!}\cdot\frac{(N-3)!!}{(N-2)!!}\ldots\frac{2!!}{3!!}\cdot\frac{1!!}{2!!}\cdot1\\ &=&\frac{1}{N}\times\left(\frac{\pi}{2}\right)^{[N/2]}\times\frac{1}{(N-1)!!}\\ &=&\left(\frac{\pi}{2}\right)^{[N/2]}\frac{1}{(N+1)!!} \end{eqnarray*} [[/math]]

Thus, we are led to the formula in the statement.

■

As main particular cases of the above formula, we have:

Proposition

The volumes of the low-dimensional spheres are as follows:

• At [math]N=1[/math], the length of the unit interval is [math]V=2[/math].
• At [math]N=2[/math], the area of the unit disk is [math]V=\pi[/math].
• At [math]N=3[/math], the volume of the unit sphere is [math]V=\frac{4\pi}{3}[/math]
• At [math]N=4[/math], the volume of the corresponding unit sphere is [math]V=\frac{\pi^2}{2}[/math].

Show Proof

These are all particular cases of the formula in Theorem 5.22.

■

5d. Basic estimates

In order to obtain estimates for the volumes, in the large [math]N[/math] limit, we can use:

Theorem

We have the Stirling formula

[[math]] N!\simeq\left(\frac{N}{e}\right)^N\sqrt{2\pi N} [[/math]]

valid in the [math]N\to\infty[/math] limit.

Show Proof

This is something quite tricky, the idea being as follows:

(1) We have the following basic approximation, by using a Riemann sum:

[[math]] \begin{eqnarray*} \log(N!) &=&\sum_{k=1}^N\log k\\ &\approx&\int_1^N\log x\,dx\\ &=&\Big[x\log x-x\Big]_0^N\\ &=&N\log N-N+1 \end{eqnarray*} [[/math]]

(2) By exponentiating we get [math]N!\approx(N/e)^Ne[/math], which is not bad, but not enough. So, we have to fine-tune our method. By using trapezoids instead of rectangles, we get:

[[math]] \int_1^N\log x\,dx\approx \log(N!)-\frac{\log 1+\log N}{2}=\log(N!)-\frac{\log N}{2} [[/math]]

Thus, we have [math]\log(N!)\approx N\log N-N+\frac{\log N}{2}+1[/math], which by exponentiating gives:

[[math]] N!\approx\left(\frac{N}{e}\right)^N\sqrt{N}\cdot e [[/math]]

(3) This is better than before, but still not enough. So, we have to further fine-tune our method, and by using this time some heavier analysis methods, the idea is that we can estimate the error, with [math]\approx[/math] becoming [math]\simeq[/math], and with the [math]e[/math] factor becoming [math]\sqrt{2\pi}[/math].

■

With the above formula in hand, we have many useful applications, such as:

Proposition

We have the following estimate for binomial coefficients,

[[math]] \binom{N}{K}\simeq\left(\frac{1}{t^t(1-t)^{1-t}}\right)^N\frac{1}{\sqrt{2\pi t(1-t)N}} [[/math]]

in the [math]K\simeq tN\to\infty[/math] limit, with [math]t\in(0,1][/math]. In particular we have

[[math]] \binom{2N}{N}\simeq\frac{4^N}{\sqrt{\pi N}} [[/math]]

in the [math]N\to\infty[/math] limit, for the central binomial coefficients.

Show Proof

All this is very standard, by using the Stirling formula etablished above, for the various factorials which appear, the idea being as follows:

(1) This follows from the definition of the binomial coefficients, namely:

[[math]] \begin{eqnarray*} \binom{N}{K} &=&\frac{N!}{K!(N-K)!}\\ &\simeq&\left(\frac{N}{e}\right)^N\sqrt{2\pi N}\left(\frac{e}{K}\right)^K\frac{1}{\sqrt{2\pi K}}\left(\frac{e}{N-K}\right)^{N-K}\frac{1}{\sqrt{2\pi(N-K)}}\\ &=&\frac{N^N}{K^K(N-K)^{N-K}}\sqrt{\frac{N}{2\pi K(N-K)}}\\ &\simeq&\frac{N^N}{(tN)^{tN}((1-t)N)^{(1-t)N}}\sqrt{\frac{N}{2\pi tN(1-t)N}}\\ &=&\left(\frac{1}{t^t(1-t)^{1-t}}\right)^N\frac{1}{\sqrt{2\pi t(1-t)N}} \end{eqnarray*} [[/math]]

Thus, we are led to the conclusion in the statement.

(2) This estimate follows from a similar computation, as follows:

[[math]] \binom{2N}{N} \simeq\left(\frac{2N}{e}\right)^{2N}\sqrt{4\pi N}\left(\frac{e}{N}\right)^{2N}\frac{1}{2\pi N} =\frac{4^N}{\sqrt{\pi N}} [[/math]]

Alternatively, we can take [math]t=1/2[/math] in (1), then rescale. Indeed, we have:

[[math]] \binom{N}{[N/2]} \simeq\left(\frac{1}{(\frac{1}{2})^{1/2}(\frac{1}{2})^{1/2}}\right)^N\frac{1}{\sqrt{2\pi\cdot\frac{1}{2}\cdot\frac{1}{2}\cdot N}} =2^N\sqrt{\frac{2}{\pi N}} [[/math]]

Thus with the change [math]N\to 2N[/math] we obtain the formula in the statement.

■

Summarizing, we have so far complete estimate for the factorials. Regarding now the double factorials, that we will need as well, the result here is as follows:

Proposition

We have the following estimate for the double factorials,

[[math]] N!!\simeq\left(\frac{N}{e}\right)^{N/2}C [[/math]]

with [math]C=\sqrt{2}[/math] for [math]N[/math] even, and [math]C=\sqrt{\pi}[/math] for [math]N[/math] odd. Alternatively, we have

[[math]] (N+1)!!\simeq\left(\frac{N}{e}\right)^{N/2}D [[/math]]

with [math]D=\sqrt{\pi N}[/math] for [math]N[/math] even, and [math]D=\sqrt{2N}[/math] for [math]N[/math] odd.

Show Proof

Once again this is standard, the idea being as follows:

(1) When [math]N=2K[/math] is even, we have the following computation:

[[math]] N!! =\frac{(2K)!}{2^KK!} \simeq\left(\frac{N}{e}\right)^{N/2}\sqrt{2} [[/math]]

(2) When [math]N=2K+1[/math] is odd, we have the following computation:

[[math]] N!! =2^KK! \simeq\left(\frac{N}{e}\right)^{N/2}\sqrt{\pi} [[/math]]

(3) Back to the case where [math]N=2K[/math] is even, by using (2) we obtain:

[[math]] \begin{eqnarray*} (N+1)!! &\simeq&\left(\frac{N+1}{e}\right)^{(N+1)/2}\sqrt{\pi}\\ &=&\left(\frac{N}{e}\right)^{N/2}\left(\frac{N+1}{N}\right)^{N/2}\sqrt{\frac{N+1}{e}}\cdot\sqrt{\pi}\\ &\simeq&\left(\frac{N}{e}\right)^{N/2}\sqrt{\pi N} \end{eqnarray*} [[/math]]

(4) Finally, back to the case where [math]N=2K+1[/math] is odd, by using (1) we obtain:

[[math]] \begin{eqnarray*} (N+1)!! &\simeq&\left(\frac{N+1}{e}\right)^{(N+1)/2}\sqrt{2}\\ &=&\left(\frac{N}{e}\right)^{N/2}\left(\frac{N+1}{N}\right)^{N/2}\sqrt{\frac{N+1}{e}}\cdot\sqrt{2}\\ &\simeq&\left(\frac{N}{e}\right)^{N/2}\sqrt{2N} \end{eqnarray*} [[/math]]

Thus, we have proved the estimates in the statement.

■

We can now estimate the volumes of the spheres, as follows:

Theorem

The volume of the unit sphere in [math]\mathbb R^N[/math] is given by

[[math]] V\simeq\left(\frac{2\pi e}{N}\right)^{N/2}\frac{1}{\sqrt{\pi N}} [[/math]]

in the [math]N\to\infty[/math] limit.

Show Proof

We use Theorem 5.22. When [math]N[/math] is even, the estimate goes as follows:

[[math]] \begin{eqnarray*} V &\simeq&\left(\frac{\pi}{2}\right)^{N/2}2^N\left(\frac{e}{N}\right)^{N/2}\frac{1}{\sqrt{\pi N}}\\ &=&\left(\frac{2\pi e}{N}\right)^{N/2}\frac{1}{\sqrt{\pi N}} \end{eqnarray*} [[/math]]

In the case where [math]N[/math] is odd, the estimate goes as follows:

[[math]] \begin{eqnarray*} V &\simeq&\left(\frac{\pi}{2}\right)^{(N-1)/2}2^N\left(\frac{e}{N}\right)^{N/2}\frac{1}{\sqrt{2N}}\\ &=&\sqrt{\frac{2}{\pi}}\left(\frac{2\pi e}{N}\right)^{N/2}\frac{1}{\sqrt{2N}}\\ &=&\left(\frac{2\pi e}{N}\right)^{N/2}\frac{1}{\sqrt{\pi N}} \end{eqnarray*} [[/math]]

Thus, we are led to the uniform formula in the statement.

■

Getting back now to our main result so far, Theorem 5.22, we can compute in the same way the area of the sphere, the result being as follows:

Theorem

The area of the unit sphere in [math]\mathbb R^N[/math] is given by

[[math]] A=\left(\frac{\pi}{2}\right)^{[N/2]}\frac{2^N}{(N-1)!!} [[/math]]

with the our usual convention for double factorials, namely:

[[math]] N!!=(N-1)(N-3)(N-5)\ldots [[/math]]

In particular, at [math]N=2,3,4[/math] we obtain respectively [math]A=2\pi,4\pi,2\pi^2[/math].

Show Proof

Regarding the first assertion, we can use the slicing argument from the proof of Theorem 5.17, which shows that the area and volume of the sphere in [math]\mathbb R^N[/math] are related by the following formula, which together with Theorem 5.22 gives the result:

[[math]] A=N\cdot V [[/math]]

As for the last assertion, this can be either worked out directly, or deduced from the results for volumes that we have so far, by multiplying by [math]N[/math].

■

General references

Banica, Teo (2024). "Linear algebra and group theory". arXiv:2206.09283 [math.CO].