create a function that splits a generator into chunks without calling len on the generator in python

To split a generator into chunks of a given size without calling len on the generator, you can use the itertools module and the iter() function. The idea is to create an iterator from the generator, and then use itertools.islice() to slice the iterator into chunks of the desired size. Here's an example implementation:

main.py
from itertools import islice

def chunk_generator(generator, chunk_size):
    while True:
        chunk = list(islice(generator, chunk_size))
        if not chunk:
            break
        yield chunk
202 chars
9 lines

This function takes a generator and a chunk size as input parameters. It then creates an iterator from the generator using the iter() function. The while loop runs indefinitely until there is no more data in the generator. Each iteration, it slices the iterator using itertools.islice(), which returns a new iterator that contains the next chunk_size elements from the original iterator. This iterator is then converted to a list and yielded as a chunk. If there are no elements left in the original iterator, the loop breaks and the function returns.

Note that this approach uses lazy evaluation, meaning that it only reads data from the generator as necessary to create each chunk. This is more memory-efficient than reading the entire generator into memory at once, which would be required to call len() on the generator.