CS 136 - Lecture 13

1. Induction
1. Proving Time Complexity - an example
2. Another example
2. PRO & CON of Recursion
3. More examples
1. Recursive insertion sort
2. Correctness
3. Complexity

Induction

Proving Time Complexity - an example

Base: n = 1, 0 comparisons and n*(n-1)/2 = 0.

Induction hypothesis: Suppose recSelSort(k-1,elts) takes k*(k-1)/2 comparisons for all k < n, n > 1.

Induction step: Show true for n as well.

Look at algorithm: Run algorithm on recSelSort(n-1,elts). Therefore, last = n-1.

Go through for loop "last" = n-1 times, w/ one comparison each time through.

Thus n-1 comparisons. Only other comparisons are in recursive call:
recSelSort(last-1,elts) where last = n-1.

But by induction (since last < n), this takes last*(last-1)/2 = (n-1)*(n-2)/2 comparisons.

Therefore altogether have (n-1) + (n-1)*(n-2)/2 = (n-1)*2/2 + (n-1)*(n-2)/2
= (n-1)*(2 + n-2)/2 = (n-1)*n/2 = n(n-1)/2 comparisons.

Therefore RecSelSort takes O(n²) comparisons.

Note: Iterative version of SelectionSort is similar, but needs an extra for loop.

Another example: FastPower

    // pre:  exponent >= 0, base > 0.
    // post - Returns baseexponent
public int fastPower(int base, int exponent) {
    if (exponent == 0) 
        return 1
    else if (exponent%2 == 1)       // exponent is odd
        return base * fastPower(base, exponent-1)
    else                            // exponent is even 
        return fastPower(base * base, exponent / 2)
}

Show FastPower(base,exp) = base^exp.

    Base case: exp = 0, FastPower(base,0) = 1 = base⁰ -- correct

    Suppose FastPower(base,exp) = base^exp for all exp < n.

 Two cases:

    n is odd:  FastPower(base,n)   = base * FastPower(base, n-1)

                                                                                                = baseⁿ
 

         n is even: FastPower(base,n) = FastPower(base*base,n / 2)

                                 = (base²)^{(n / 2)} (by induction)

                                                                                           =  baseⁿ    (since n is even)

Done with proof of correctness!

PRO & CON of Recursion

CON

• Any recursive program requires additional storage on (implicit) run-time stack.
• Also slight run-time overhead in procedure call since must push activation records on stack.
• Often slightly faster and less space if can do iteratively.

PRO

• Often easier to construct recursive solution.
• Resulting code may be significantly more clear.
• Smart compilers (particularly for functional languages) will remove "tail" recursion.

More examples

Recursive insertion sort

protected void recInsSort(int last, Comparable[] elts){
    if (last > 0){
        recInsSort(last-1, elts); // Sort elts[0..last-1]
            
        int posn= last-1;   // posn is index where last will be inserted

        // Search for first elt (from rear) <=  elts[last]
        while (posn >= 0 && elts[last].lessThan(elts[posn]))
            posn--;

        posn++; // insert elts[index] at posn

        // move elts[posn .. last-1] to put in elts[last] 
        Comparable tempElt = elts[last];
        for (int moveIndex = last-1; moveIndex >= posn; moveIndex--)
            elts[moveIndex+1] = elts[moveIndex];

      // insert element into proper position
        elts[posn] = tempElt;
      // now elts[0..last] are in order
    }
}

Correctness

Proof by induction on size of last parameter:

Base recInsSort(0, elts) clearly works correctly. If last is 0, the array has a single element and no sorting needs to be done.

Inductive Hypothesis Suppose recInsSort(k, elts) correctly sorts elts[0..k] for k < n.

Induction Step Show recInsSort(n, elts) works correctly.

Algorithm first calls recInsSort(n-1, elts).

By the Inductive Hypothesis, we know that recInsSort(n-1, elts) correctly sorts elts[0..n-1].

Next the algorithm finds first posn (from rear) where elts[posn].lessThan(elts[n]).

Update posn = posn+1, so elts[posn] >= elts[n].

Now the algorithm moves each of elts[posn..last-1] one step to right, and inserts value of elts[last] in elts[posn].

Now elts[posn+1..last] >= elts[posn] > elts[posn-1]. and elts[0..posn-1] are sorted.

Therefore elts[0..last] is sorted.

Complexity

Because while loop may quit early, insertion sort only uses half as many comparisons (on average) than selection sort. Thus usually twice as fast (but same "O" complexity).