Today we introduced Newton’s method for finding the roots of a function. If a function has nice derivative, then it is easy to program Newton’s method into a computer (or even a calculator!). The iteration formula is \[x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}.\] We used Newton’s method to find the roots of these functions in class:
We also talked about rates of convergence for iterative methods like the bisection method and Newton’s method. We talked about the order of convergence and linear versus quadratic convergence. We proved the following theorem which shows that Newton’s method has quadratic convergence in many situations.
Let \(f \in C^2[a,b]\) and suppose that \(f\) has a root \(r \in (a,b)\). Suppose that there are constants \(L,M >0\) such that \(f'(x) \ge L\) and \(f''(x) \le M\) for all \(x \in [a,b]\). Then \[|x_{n+1} - r| \le \frac{M}{2L} |x_n-r|^2\] when \(x_n \in [a,b]\).
Proof. Start with the first degree Taylor polynomial (centered at \(x_n\)) for \(f(r)\) including the remainder term and the Newton’s method iteration formula:
\[f(r) = f(x_n) + f'(x_n)(r-x_n) + \frac{1}{2} f''(z)(r-x_n)^2 = 0,\] and \[x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \Rightarrow f'(x_n)(x_{n+1} - x_n) + f(x_n)=0.\]
Subtract these two formulas to get a formula that relates \((r-x_{n+1})\) with \((r-x_n)\).
\[f'(x_n)(r-x_{n+1}) + \frac{1}{2} f''(z)(r-x_n)^2 = 0.\]
Use this to get an upper bound on \(|r-x_{n+1}|\). \(\square\)
The error in the nth-iterate of Newton’s method satisfies \[|x_n-r| \le \left(\frac{M}{2L}\right)^{2^n-1} |x_0 - r|^{2^n}.\]
Today we did this lab in-class:
One of the goals in the lab is to write a Python function to implement Newton’s method for any function f as long as you can find a nice formula for f and its derivative Df. Be careful, if you try to define a function f by entering
f = x**3 - 5*x + 3 # f is a number or undefined (depending on the value of x).then f will either be the number you get when you calculate \(x^3 - 5x + 2\), or it will be undefined (depending on whether x has a numerical value before you enter that command.
If you want f to be a function and not a number, make sure use a lambda function:
f = lambda x: x**3 - 5*x + 3 # Now f is a function.