# Finite Differences

This chapter introduces the notion of finite differences. This is a useful construct for solving boundary value problems (BVPs) and partial differential equations (PDEs.) Apart from some elementary calculus, there isn't any assumed knowledge for this chapter.

## Estimating Derivatives

Harking back to the Euler Method derivation, recall the Taylor expansion of $f(x)$ with the higher-order terms expressed as a lower bound:

\eqalign{ f(x + \Delta x) &= f(x) + \frac{df(x)}{dx} \Delta x + \frac{d^2 f(x)}{dx^2}\frac{\Delta x^2}{2} \cdots \cr &= f(x) + \frac{df(x)}{dx} \Delta x + O(\Delta x^2) } \label{ffd-taylor}

Rewrite equation \eqref{ffd-taylor}

$$\frac{df(x)}{dx} = \frac{f(x + \Delta x) - f(x)}{\Delta x} + O(\Delta x) \label{ffd}$$

such that it approximates the first derivative of $f$ with regards to $x$. This is the forward finite difference equation.

Evaluate $f(x - \Delta x)$ in a similar fashion:

\eqalign{ f(x - \Delta x) &= f(x) - \frac{df(x)}{dx} \Delta x + \frac{d^2 f(x)}{dx^2}\frac{\Delta x^2}{2} \cdots \cr &= f(x) - \frac{df(x)}{dx} \Delta x + O(\Delta x^2) } \label{bfd-taylor}

Again, rewrite \eqref{bfd-taylor} to approximate $\frac{df(x)}{dx}$ with the backward finite difference equation:

$$\frac{df(x)}{dx} = \frac{f(x) - f(x - \Delta x)}{\Delta x} + O(\Delta x) \label{bfd}$$

To obtain the central difference equation, subtract \eqref{bfd-taylor} from \eqref{ffd-taylor} and rewrite in terms of $\frac{df(x)}{dx}$:

$$\frac{df(x)}{dx} = \frac{f(x + \Delta x) - f(x - \Delta x)}{2\Delta x} + O(\Delta x^2) \label{cfd}$$

Add equations \eqref{ffd-taylor} and \eqref{bfd-taylor} to obtain an estimate for $\frac{d^2f(x)}{dx^2}$:

$$\frac{d^2f(x)}{dx^2} = \frac{f(x + \Delta x) + f(x - \Delta x) - 2f(x)}{\Delta x^2} + O(\Delta x^2) \label{central-2nd-order}$$

Continue reading The Relaxation Method to discover how to use the first- and second-order central differences to solve BVPs. The concept of forward finite differences and central finite differences also come into play when solving PDEs numerically.