Start learning free

LESSON 84 OF 165 · MACHINE LEARNING: ML INTERVIEW QUESTIONS

Gini Impurity in Python

Practice

An interviewer asks: "What is Gini impurity? Write it for a set of labels, then use it to score a split." The idea in one sentence: the Gini impurity of a group of labels is 1 minus the sum of the squared shares of its classes, so a pure group scores 0 and a mixed group scores more.

Counting the classes: np.unique(labels, return_counts=True) gives back two arrays: the different labels, sorted, and how many times each one appears. Dividing the counts by len(labels) gives each class's share. For 4 dogs and 1 cat:

pets = np.array(["dog", "cat", "dog", "dog", "dog"])
names, counts = np.unique(pets, return_counts=True)
print(names)               # ['cat' 'dog']
print(counts / len(pets))  # [0.2 0.8]

The Gini of these pets is 1 − (0.2² + 0.8²) = 1 − (0.04 + 0.64) = 0.32. In numpy, shares ** 2 squares every item of an array and .sum() adds them all up.

Scoring a split: a decision tree cuts the labels into a left and a right group. The weighted Gini of the split adds each group's Gini times its share of all the labels, so a big group counts more than a small one. Lower is better.

The starting code has 10 labels (7 ones, 3 zeros) and a split of them into left and right, 5 labels each. Your code must print:

Gini of all 10 labels: 0.42
Gini of the left group: 0.0
Gini of the right group: 0.48
Weighted Gini of the split: 0.24

Your task: write gini(labels), which returns the Gini impurity of an array of labels (any number of classes, numbers or words), and split_gini(left, right), which returns the weighted Gini of a split. Use numpy only: the check compares your numbers with scikit-learn's decision tree.


Assignment
Follow the Coding Tutorial to write gini and split_gini, one line at a time, and print the four scores.


Hint
A pure group has a single share of 1, so its Gini is 1 − 1² = 0. In split_gini, n is len(left) + len(right), and the left group's weight is len(left) / n.


STEP 1 OF 7
    Ctrl + Enter to check

    Mlq Gini in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free