Scratch pad

You can use additional cells in Jupyter Notebook to work through small pieces of code that you don't understand.

import numpy as np 

def logistic_function(z: np.ndarray) -> np.ndarray:
    """
    Impelmentation of the logistic function 

    Args:
        z: A numpy array.
    Returns:
        The numpy array z with the logistic function applied element-wise

    Hints
     - You should be using numpy and array broadcasting here 
     - np.exp may be helpful 
    """
    # TODO: implement your solution here 
    # CODE START
    raise NotImplementedError("Solution not yet implemented!") #delete this line and add your solution
    # CODE END

a = np.array([1, 2, 3])
np.exp(a)

array([ 2.71828183,  7.3890561 , 20.08553692])

1/a

array([1.        , 0.5       , 0.33333333])

Vectorizing code

We can use numpy to "vectorize" code involving arrays. This means we replace for-loops with numpy's built-in functions like np.dot.

import numpy as np 

N = 1000 
array1 = np.random.random((2*N, N))
array2 = np.random.random((N))

%%time
m = 0 
a = -1 
for i in range(array1.shape[0]):
    value = 0
    for j in range(array1.shape[1]):
        value += array1[i][j] * array2[j]
    if value > m:
        m = value
        a = i
print(f"a={a}, m={m}")

a=1704, m=263.44477397540743
CPU times: user 620 ms, sys: 339 ms, total: 959 ms
Wall time: 450 ms

# Before we apply a dot product, check our assumptions about the dimensions
assert array1.shape[1] == array2.shape[0]

%%time

#Compute the max dot product
#Using numpy methods  

dot_product = np.dot(array1, array2)
print(f"argmax={np.argmax(dot_product)}, max={np.max(dot_product)}")

argmax=1704, max=263.4447739754075
CPU times: user 8.98 ms, sys: 4.96 ms, total: 13.9 ms
Wall time: 1.57 ms

# What happens if we turn up to 100k examples? 

More numerical stability

Add a small number to np.log() so you don't get zeros.

np.log(0)

/var/folders/hw/ghdf3kcs6wl0q0hlk8nmlh3c0000gn/T/ipykernel_14113/2933082444.py:1: RuntimeWarning: divide by zero encountered in log
  np.log(0)

-inf

np.log(0 + 1e-8)

-18.420680743952367

np.log(0.99)

-0.01005033585350145

np.log(0.99 + 1e-8)

-0.01005032575249135