Video: Numerical Error

Here's a video on Numerical Error from a Numerical Methods And Programming class by Prof.P.B.Sunil Kumar, Dept. of Physics, IIT Madras (India)

Notes:

• The first part of the lecture talks on the Representation of Numbers(a continuation from past lecture) in our digital computer such as the integers and floating-point numbers.
• Difference of Chopping and rounding:
  Chopping is to merely chop off or cut off higher digits while rounding means that any quantity falling within an interval of length delta x will be represented as the nearest allowable number.
  Rounding yields lower error than chopping.
• The concept of zero
  The precise meaning of the number zero is lost while computing numerically
• Relationship between the exact(true) result and the approximation can be formulated as
  True value = approximation + error
  This relationship gives the value of the error but it does not take into account the order of the magnitude of the value under examination
• To account the magnitude of the quantities being evaluated, we normalize the error to the true value as in
  Fractional relative error = Error/true Value
  (Significance of this thought: An error of a centimeter is much more significant if we are measuring a rivet than a bridge. See the magnitude of the quantity of the item being evaluated)
• Fractional relative error multiplied by 100 would give us the Percent relative error
• In real applications, we don't know the true value. So an alternate solution is to normalize the error using the approximation itself as in,
     Ea = (approximate error/approximation)100%
  the subscript 'a' signifies that the error is normalized to an approximate value. "Ea" is read as "Epsilon sub a" 
• For iterative procedures, the error is estimated as the difference between the present and previous approximations. Thus the percent relative error is determined according to:
   Ea = [(present approximation - previous approximation)/present approximation] 100%
  Generally, one is not concerned with the sign of the error but is interested whether the absolute value is lower than a pre-specified tolerance Es.
  Computation in the iterative method is repeated until
    |Ea| < Es
  

In summary

• Two errors encountered are round-off errors(due to finite representation of numbers) and truncation errors(due to the use of an approximation instead of a finite mathematical operation).  Rounding errors appears due to arithmetic operations using finite floating point numbers and is a major source of error.
• The only way to minimize round-off errors is to increase the number of significant figures of the computer
• A decrease in step-size (for iterative) can lead to subtractive cancellation or an increase in computations, thus the truncation errors are decreased as the round-off errors are increased.