CS 136 - Lecture 12

  1. Recursion and Induction
    1. Recursion
    2. Mathematical Induction
      1. Proving Correctness - an example
      2. Proving Time Complexity - an example
      3. Another example

Recursion and Induction

Recursion

Many algorithms may be recursive. Once you are used to them, they can be easier to understand (and prove correct) than iterative algorithms: 

protected void recSelSort(int lastIndex, Comparable[] elts)
{
    if (lastIndex > 0)   // more than 1 element to sort
    {
        int extreme = 0;    // index of element w/ largest value 
                    
        // Find "extreme", index of elts w/ largest value. 
        for (int searchIndex = 1; searchIndex <= lastIndex; searchIndex++) 
        {
            if (elts[extreme].lessThan(elts[searchIndex])) 
                extreme = searchIndex;
        }
        
        // swap largest elt (at extreme) w/ one at lastIndex
        Comparable tempElt = elts[extreme]; 
        elts[extreme] = elts[lastIndex];
        elts[lastIndex] = tempElt;
        // elts[lastIndex] now largest in   
        //  elts[0..lastIndex]
            
        recSelSort(lastIndex-1,elts);
       // elts[0..lastIndex] are sorted.
    }

Mathematical Induction

Principle of Mathematical Induction

Let A be a set of natural numbers such that
  1. 0 is an element of A, and
  2. for each n, if {0, 1, ..., n} in A, then (n+1) is also in A. Then A is the set of natural numbers.
We use this to prove statements of the form: For all natural numbers n, P is true.

Example. Show by induction that 


03 + 13 + 23 + ... + n3 = (0+1+2+ ... +n)2



Base. Show for n=0.  03 = 0 = (0)2 [Done.]


Inductive Hypothesis.  Assume that for some n >= 0,


03 + 13 + 23 + ... + m3 = (0+1+2+ ... +m)2 , m <= n.


Induction Step. Show that 03 + 13 + 23 + ... + (n+1)3 = (0+1+2+ ... + n + n+1)2.


03 + 13 + 23 + ... + (n+1)3


= (0+1+2+ ... +n)2 + (n+1)3


= ((n)(n+1)/2)2 + (n+1)3


= ((n)(n+1)/2)2 + (n+1)2(n+1)


= ((n2 + n)/2)2 + n(n+1)2 + (n+1)2


= (n4 + 2n3 + n2)/4 + 4/4(n)(n+1)2 + (4/4)(n+1)2


= (n4 + 6n3 + 13n2 + 12n + 4)/4


= (n2 + 3n + 2)2/4


= ((n+1)(n+2)/2)2


= (0 + 1 + 2 + ... + n + n+1)2

What does this have to do with recursion?

Notice form of the Principle of Mathematical Induction and the form of the proof just given. This relationship will allow us to use induction to prove various things about recursive algorithms -- specifically correctness and time complexity.

Proving correctness - an example.

A. Prove base case(s). (Usually this is trivial)

B. Show that if algorithm works correctly for all simpler (i.e., smaller) input, then will work for current input.

To show that the recSelSort algorithm given earlier is correct, we will reason by induction on the size of the array (i.e. on lastIndex)

Base: If lastIndex <= 0 then at most one element in elts, and hence sorted - correct.

Induction Hypothesis: Suppose works if lastIndex < n.

Induction: Show it works if last = n (> 0).

Loop finds largest element and then swaps with elts[lastIndex].

Thus elts[lastIndex] holds largest elt of list, while others held in elts[0..lastIndex-1].

Since lastIndex- 1 < lastIndex, know (by induction hypothesis) that
recSelSort(lastIndex-1,elts) sorts elts[0..lastIndex-1].

Because elts[0..lastIndex-1] in order and elts[lastIndex] is >= all of them, elts[0..lastIndex] is sorted.

Proving Time Complexity - an example.

Claim: recSelSort(n-1,elts) (i.e, on n elements) involves n*(n-1)/2 comparisons of elements of array.
Base: n = 0 or 1, 0 comparisons and n*(n-1)/2 = 0.

Induction hypothesis: Suppose recSelSort(k-1,elts) takes k*(k-1)/2 comparisons for all k < n.

Induction step: Show true for n as well.

Look at algorithm: Run algorithm on recSelSort(n-1,elts). Therefore, last = n-1.

Go through for loop "last" = n-1 times, w/ one comparison each time through.

Thus n-1 comparisons. Only other comparisons are in recursive call:
recSelSort(last-1,elts) where last = n-1.

But by induction (since last < n), this takes last*(last-1)/2 = (n-1)*(n-2)/2 comparisons.

Therefore altogether have (n-1) + (n-1)*(n-2)/2 = (n-1)*2/2 + (n-1)*(n-2)/2
= (n-1)*(2 + n-2)/2 = (n-1)*n/2 = n(n-1)/2 comparisons.

Finished proof.

Therefore RecSelSort takes O(n2) comparisons.

Note: Iterative version of SelectionSort is similar, but needs an extra for loop.

Another example: FastPower


    // pre:  exponent >= 0
    // post - Returns baseexponent
public int fastPower(int base, int exponent) {
    if (exponent == 0) 
        return 1
    else if (exponent%2 == 1)       // exponent is odd
        return base * fastPower(base, exponent-1)
    else                            // exponent is even 
        return fastPower(base * base, exponent / 2)
}

Show FastPower(base,exp) = baseexp.

By induction on size of exp:
    Base case: exp = 0, FastPower(base,0) = 1 = base0 -- correct

    Suppose FastPower(base,exp) = baseexp for all exp < n.

              Show FastPower(base,n) = basen

 Two cases: 

    n is odd:  FastPower(base,n)   = base * FastPower(base, n-1)
                                   = base * basen-1 (by induction)

                                                                                                = basen
 

         n is even: FastPower(base,n) = FastPower(base*base,n / 2)

                                 = (base2)(n / 2) (by induction)

                                                                                           =  basen    (since n is even)

Done with proof of correctness!