Functional tools for iterables

g() is very convenient, but it’s only a thin wrapper on top of the tools from this module.

So if you want to apply some of the goodies from it without having to turn your iterables into IterableWrapper objects, you can use the functions from this module directly.

Example

>>> from ww.tools.iterables import chunks  # same as g().chunks()
>>> list(chunks(range(10), 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9,)]

You’ll find bellow the detailed documentation for each functions. Remember they all take an iterable as input, and most often ouput a generator.

Go have a look, there is some great stuff here!

ww.tools.iterables.at_index(iterable, index)

” Return the item at the index of this iterable or raises IndexError.

WARNING: this will consume generators.

Negative indices are allowed but be aware they will cause n items to be held in memory, where n = abs(index)

ww.tools.iterables.chunks(iterable, chunksize, cast=<class 'tuple'>)

Yields items from an iterator in iterable chunks.

ww.tools.iterables.first_true(iterable, func)

” Return the first item of the iterable for which func(item) == True.

Or raises IndexError.

WARNING: this will consume generators.

ww.tools.iterables.firsts(iterable, items=1, default=None)

Lazily return the first x items from this iterable or default.

ww.tools.iterables.iterslice(iterable, start=0, stop=None, step=1)

Like itertools.islice, but accept int and callables.

If start is a callable, start the slice after the first time start(item) == True.

If stop is a callable, stop the slice after the first time stop(item) == True.

ww.tools.iterables.lasts(iterable, items=1, default=None)

Lazily return the last x items from this iterable or default.

ww.tools.iterables.skip_duplicates(iterable, key=None, fingerprints=())

Returns a generator that will yield all objects from iterable, skipping duplicates.

Duplicates are identified using the key function to calculate a unique fingerprint. This does not use natural equality, but the result use a set() to remove duplicates, so defining __eq__ on your objects would have no effect.

By default the fingerprint is the object itself, which ensure the functions works as-is with an iterable of primitives such as int, str or tuple.

Example:

>>> list(skip_duplicates([1, 2, 3, 4, 4, 2, 1, 3 , 4]))
[1, 2, 3, 4]

The return value of key MUST be hashable, which means for non hashable objects such as dict, set or list, you need to specify a a function that returns a hashable fingerprint.

Example:

>>> list(skip_duplicates(([], [], (), [1, 2], (1, 2)),
...                      lambda x: tuple(x)))
[[], [1, 2]]
>>> list(skip_duplicates(([], [], (), [1, 2], (1, 2)),
...                      lambda x: (type(x), tuple(x))))
[[], (), [1, 2], (1, 2)]

For more complex types, such as custom classes, the default behavior is to remove nothing. You MUST provide a key function is you wish to filter those.

Example:

>>> class Test(object):
...    def __init__(self, foo='bar'):
...        self.foo = foo
...    def __repr__(self):
...        return "Test('%s')" % self.foo
...
>>> list(skip_duplicates([Test(), Test(), Test('other')]))
[Test('bar'), Test('bar'), Test('other')]
>>> list(skip_duplicates([Test(), Test(), Test('other')],                                     lambda x: x.foo))
[Test('bar'), Test('other')]

ww.tools.iterables.starts_when(iterable, condition)

Start yielding items when a condition arise.

Parameters:

• iterable – the iterable to filter.
• condition – if the callable returns True once, start yielding items. If it’s not a callable, it will be converted to one as lambda condition: condition == item.

Example

>>> list(starts_when(range(10), lambda x: x > 5))
[6, 7, 8, 9]
>>> list(starts_when(range(10), 7))
[7, 8, 9]

ww.tools.iterables.stops_when(iterable, condition)

Stop yielding items when a condition arise.

Parameters:

• iterable – the iterable to filter.
• condition – if the callable returns True once, stop yielding items. If it’s not a callable, it will be converted to one as lambda condition: condition == item.

Example

>>> list(stops_when(range(10), lambda x: x > 5))
[0, 1, 2, 3, 4, 5]
>>> list(stops_when(range(10), 7))
[0, 1, 2, 3, 4, 5, 6]

ww.tools.iterables.window(iterable, size=2, cast=<class 'tuple'>)

Yields iterms by bunch of a given size, but rolling only one item in and out at a time when iterating.

>>> list(window([1, 2, 3]))
[(1, 2), (2, 3)]

By default, this will cast the window to a tuple before yielding it; however, any function that will accept an iterable as its argument is a valid target.

If you pass None as a cast value, the deque will be returned as-is, which is more performant. However, since only one deque is used for the entire iteration, you’ll get the same reference everytime, only the deque will contains different items. The result might not be what you want :

>>> list(window([1, 2, 3], cast=None))
[deque([2, 3], maxlen=2), deque([2, 3], maxlen=2)]