CS 136 - Lecture 13

  1. Induction
    1. Proving Time Complexity - an example
    2. Another example
  2. PRO & CON of Recursion
  3. More examples
    1. Recursive insertion sort
    2. Correctness
    3. Complexity


We will use induction to prove various properties of recursive algorithms -- specifically correctness and time complexity.

Proving Time Complexity - an example

Claim: recSelSort(n-1,elts) (i.e, on n elements) involves n*(n-1)/2 comparisons of elements of array. Note that we are including only array element to array element comparisons here, rather than all comparisons, since those dominate.
Base: n = 1, 0 comparisons and n*(n-1)/2 = 0.

Induction hypothesis: Suppose recSelSort(k-1,elts) takes k*(k-1)/2 comparisons for all k < n, n > 1.

Induction step: Show true for n as well.

Look at algorithm: Run algorithm on recSelSort(n-1,elts). Therefore, last = n-1.

Go through for loop "last" = n-1 times, w/ one comparison each time through.

Thus n-1 comparisons. Only other comparisons are in recursive call:
recSelSort(last-1,elts) where last = n-1.

But by induction (since last < n), this takes last*(last-1)/2 = (n-1)*(n-2)/2 comparisons.

Therefore altogether have (n-1) + (n-1)*(n-2)/2 = (n-1)*2/2 + (n-1)*(n-2)/2
= (n-1)*(2 + n-2)/2 = (n-1)*n/2 = n(n-1)/2 comparisons.

Finished proof.

Therefore RecSelSort takes O(n2) comparisons.

Note: Iterative version of SelectionSort is similar, but needs an extra for loop.

Another example: FastPower

    // pre:  exponent >= 0, base > 0.
    // post - Returns baseexponent
public int fastPower(int base, int exponent) {
    if (exponent == 0) 
        return 1
    else if (exponent%2 == 1)       // exponent is odd
        return base * fastPower(base, exponent-1)
    else                            // exponent is even 
        return fastPower(base * base, exponent / 2)

Show FastPower(base,exp) = baseexp.

By induction on size of exp:
    Base case: exp = 0, FastPower(base,0) = 1 = base0 -- correct

    Suppose FastPower(base,exp) = baseexp for all exp < n.

              Show FastPower(base,n) = basen

 Two cases:

    n is odd:  FastPower(base,n)   = base * FastPower(base, n-1)
                                   = base * basen-1 (by induction)

                                                                                                = basen

         n is even: FastPower(base,n) = FastPower(base*base,n / 2)

                                 = (base2)(n / 2) (by induction)

                                                                                           =  base   (since n is even)

Done with proof of correctness!

PRO & CON of Recursion



More examples

Recursive insertion sort

protected void recInsSort(int last, Comparable[] elts){
    if (last > 0){
        recInsSort(last-1, elts); // Sort elts[0..last-1]
        int posn= last-1;   // posn is index where last will be inserted

        // Search for first elt (from rear) <=  elts[last]
        while (posn >= 0 && elts[last].lessThan(elts[posn]))

        posn++; // insert elts[index] at posn

        // move elts[posn .. last-1] to put in elts[last] 
        Comparable tempElt = elts[last];
        for (int moveIndex = last-1; moveIndex >= posn; moveIndex--)
            elts[moveIndex+1] = elts[moveIndex];

      // insert element into proper position
        elts[posn] = tempElt;
      // now elts[0..last] are in order


Proof by induction on size of last parameter:

Base recInsSort(0, elts) clearly works correctly. If last is 0, the array has a single element and no sorting needs to be done.

Inductive Hypothesis Suppose recInsSort(k, elts) correctly sorts elts[0..k] for k < n.

Induction Step Show recInsSort(n, elts) works correctly.

Algorithm first calls recInsSort(n-1, elts).

By the Inductive Hypothesis, we know that recInsSort(n-1, elts) correctly sorts elts[0..n-1].

Next the algorithm finds first posn (from rear) where elts[posn].lessThan(elts[n]).

Update posn = posn+1, so elts[posn] >= elts[n].

Now the algorithm moves each of elts[posn..last-1] one step to right, and inserts value of elts[last] in elts[posn].

Now elts[posn+1..last] >= elts[posn] > elts[posn-1]. and elts[0..posn-1] are sorted.

Therefore elts[0..last] is sorted.


Complexity is O(n2) again because can show recInsSort(n-1,elts) takes <= n*(n-1)/2 comparisons by induction. (I.e., show for n = 1. Then assume induction hypothesis for k < n, and show for n.)

Because while loop may quit early, insertion sort only uses half as many comparisons (on average) than selection sort. Thus usually twice as fast (but same "O" complexity).