Start learning free

LESSON 121 OF 129 · AI ENGINEERING: INTERVIEW QUESTIONS

Split a Document into Chunks in Python

Practice

The interview question: "Write chunk_words(text, size, overlap). Split a document into chunks of size words, where each chunk repeats the last overlap words of the one before. Never return an empty chunk, or a last chunk that only repeats the one before it."

The idea: each chunk starts size - overlap words after the one before, and the loop stops as soon as a chunk reaches the last word. Why overlap at all? A sentence cut at one chunk's edge is still found whole in its neighbor.

letters = ["a", "b", "c", "d", "e"]
print(list(range(0, 5, 3)))   # [0, 3]: a third number is the jump
print(letters[3:6])           # ['d', 'e']: a slice stops at the end

The starting code already splits the policy into words (26 of them) and returns chunks. With size 8 and overlap 2 your program must print four chunks. The check then compares overlap 0 with overlap 2 and tries two special texts:

[Returns are free within 30 days of delivery.]
[of delivery. Pack the item in its original]
[its original box. Refunds take 5 days. Write]
[days. Write to us with your order number.]
Overlap 0 -> 4 chunks; the whole sentence 'Refunds take 5 days.' is in chunks []
Overlap 2 -> 4 chunks; the whole sentence 'Refunds take 5 days.' is in chunks [2]
Exactly 8 words: ['one two three four five six seven eight']
Empty text: []

Your task: fill in chunk_words: compute the step, loop over the start positions, slice and join each piece, and break once a chunk reaches the last word.


Assignment
Follow the Coding Tutorial to fill in the loop of chunk_words.


Hint
Check for the end after the append, so the chunk that holds the last word is kept: start + size >= len(words) means this chunk already reaches it.


STEP 1 OF 6
    Ctrl + Enter to check

    Aiq Chunking in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free