Common Words Count Less: TF-IDF in Python

Plain counts favour filler. In the review "The teapot pours well and the lid fits", the top count belongs to the, a word every review uses, which tells you nothing about this one. TF-IDF (term frequency times inverse document frequency) fixes that: it weighs each word by how rare it is across all the texts.

Each word gets an idf, one number for the whole collection: 1.0 for a word that is in every text, and higher the fewer texts have it. A word's TF-IDF weight in one text is its count there times its idf. Then each text's row is scaled down so that long and short texts compare fairly; the order of the words stays the same.

TfidfVectorizer is used just like CountVectorizer: fit_transform(texts) learns the same vocabulary, in the same order, and gives back one row of weights per text, as a sparse matrix (.toarray() gives the full table). After fitting, its idf_ attribute is an array with the idf of each vocabulary word, in column order.

vec = TfidfVectorizer()
w = vec.fit_transform(["fast delivery", "slow delivery"])
print(vec.idf_.round(3))   # delivery 1.0 (in both texts), fast 1.405, slow 1.405

The starting code already prints the first review's counts, top count first. Below them, your script must print the same words with their idf and TF-IDF weight, top weight first:

        count    idf  tf-idf
and         1  2.504   0.397
fits        1  2.504   0.397
lid         1  2.504   0.397
pours       1  2.504   0.397
teapot      1  2.504   0.397
well        1  2.099   0.333
the         2  1.000   0.317

Your task: make a TfidfVectorizer and fit it on reviews, add an "idf" column and a "tf-idf" column (the first review's weights) to table, both rounded to 3 decimals, and print the first review's words sorted by TF-IDF.

Assignment
Follow the Coding Tutorial to weigh the first review's words with TF-IDF and compare them with the plain counts.

Hint
Both vectorizers learn the same words in the same order, so vectorizer.idf_ and weights.toarray()[0] line up with the rows of table. The last print is the one in the starting code, sorted by "tf-idf" instead of "count".