Start learning free

LESSON 89 OF 165 · MACHINE LEARNING: ML INTERVIEW QUESTIONS

PCA from Scratch in Python

Practice

An interviewer asks: "Implement PCA with numpy only." PCA (principal component analysis) finds the directions in which the rows spread the most, then describes each row by its position along the top few of them. You get fewer columns and keep most of the information.

First subtract each column's mean: X.mean(axis=0) gives back one mean per column. Then build the covariance matrix: the centered data's transpose (.T swaps rows and columns) times the centered data, with @, divided by the number of rows minus 1. It has one row and one column per feature, with each column's variance on the diagonal. np.linalg.eigh gives back its eigenvalues, how much the data spreads along each direction, smallest first, and a matrix whose columns are the matching eigenvectors: the directions.

To keep the biggest ones, np.argsort gives back the positions that would sort an array, from the smallest value to the biggest. On another array:

values = np.array([0.5, 8.0, 1.5])
print(np.argsort(values))            # [0 2 1]
print(np.argsort(values)[::-1][:2])  # [1 2]: the 2 biggest, biggest first

A row's position along a direction is the row times the direction, with @. Each kept eigenvalue divided by the sum of all of them is the explained share: how much of the spread its direction keeps. On the 6 rows of 3 columns in the starting code, the program must print:

Explained share: [0.929 0.062]
The rows on the top 2 directions:
[[ 4.386  0.325]
 [ 1.39   0.21 ]
 [-1.915 -1.118]
 [-4.602 -0.019]
 [ 2.417 -0.717]
 [-1.676  1.32 ]]

Your task: inside pca(X, n_components), center X, build the covariance matrix, take its eigenvalues and eigenvectors, keep the positions of the n_components biggest, project the centered rows on those directions and compute the explained share. The function already returns projected, share; the hidden check compares them with scikit-learn's PCA.


Assignment
Follow the Coding Tutorial to write pca with numpy only, one line per step.


Hint
eigh gives the eigenvalues smallest first, so reverse the argsort before keeping the first n_components. vectors[:, top] keeps the chosen directions as columns. Your signs may differ from scikit-learn's: a direction and its opposite lie on the same line, so the check compares the rows up to sign.


STEP 1 OF 6
    Ctrl + Enter to check

    Mlq Pca in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free