An object is said to be iterable if it has the iter method defined. The __iter_₍₎ will reutrn the iterator object.

An object is said to be a iterator if it has following method defined:

1. iter: which return itself Can be tested the it.__iter_₍₎ == it is true
2. next: return the next value every time it is invoked.

So an iterator is an iterable, call iter(iterable) to get an iterator.

The iter(iterable) function: it will return 'iterable.__iter_₍₎'

So if obja is an iterable, then iter(obja) equal obja.__iter()

obja = [1, 2, 3]
ia = obja.__iter__()
ib = iter(obja)
ic = iter(ib)
print(ia)
print(ib)
print(ib is ic)
print(next(ia), next(ib))

if obja is iterator, then iter(obja) and obja is the same object.

A good ref: http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html

1. it first get the iterable's iterator object, by calling its __iter_₍₎ method
2. get the element by invoke the iterator's __next_₍₎ method, and bind the value to the variable.
3. stop when an 'StopIteration' exception happens.

it just return iterator.__next_₍₎

it just return iterable.__iter_₍₎

it just return obj.__len_₍₎

When create a class the with a underline container, just define an __iter_₍₎ method that forward the request to the underlineing container object.

a generator is a function that contains at lease one 'yeild' statement.

Unlike normal function, it's boyd will not be executed when it is be called, instead, it will return a generator object.

use the generator instead of the next method, which will be much simple.

使用yeild 创建一个Tree Node,比使用__next__{函数简单多了。}

yeild from syntax.

reversed only works if the obj

• the obj defined a __reversed_₍₎ method. or
• the obj's size can be determined.

It returns an iterator.

For example, a file handler returned by the 'open()' function can't be used with the reversed function. to use it, first convert it to a list, then pass it to the reversed() function.

with open("1.txt") as f:
    a = reversed(list(f))
    print(next(a), next(a))

class CountDown():
    def __init__(self, start):
	self._start = start

    def __iter__(self):
	return self

    def __next__(self):
	if self._start >=0:
	    n = self._start
	    self._start -= 1
	    return n
	else:
	    raise StopIteration

    def __reversed__(self):
	return ReversedCountDown(self)

class ReversedCountDown():
    def __init__(self, orig):
	self._orig = orig
	self._n = -1
    def  __iter__(self):
	return self
    def __next__(self):
	if self._n <= self._orig._start:
	    self._n += 1
	    return self._n
	else:
	    raise StopIteration

# if __name__ == '__main__':
cd = CountDown(2)
# for a in cd:
#     print(a)

print("reversed")
for a in reversed(cd):
    print(a)

Implemet the iterator protocal by next method is a little complex compared to by use the yield statement. The differenc is that then the object is …

class CountDown():
    def __init__(self, start):
	self._start = start

    def __iter__(self):
	n = self._start
	while n >=0:
	    yield n
	    n -=1

    def __reversed__(self):
	n = 0
	while n <=self._start:
	    yield n
	    n+=1

cd = CountDown(3)
for a in cd:
    print(a)

print ("reversed")
for a in reversed(cd):
    print(a)

import itertools
with open('1.txt') as f:
    for line in itertools.dropwhile(lambda x: x.startswith('#'), f):
	print(line, end='')

This is different from filtering

if the position is known, then we can use itertools.islice(iterable, start, None) to drop the first 'start' items.

An important aspect of itertools module: for complex iteration tasks, it is very likely there is an exist solution.

The zip function will create an iterator that return tuples: first element from iterable1, second element from iterable2, … Should the size of all iterables be the same? => No, it can be different. the returned size is the same as the shortest size of all iterables.

a =  [1,  2, 3]
b = ['a', 'b', 'c', 'd']
for v in zip(a, b):
    print(v)

If you want the returned iterator take the longest size, then use zip_longest. The element value will be None if that iterable is exzasted.

From the two functions: zip and zip_longest, there is a lesson: it better to create different function name, than add a more parameter.

from itertools import zip_longest
a =  [1,  2, 3]
b = ['a', 'b', 'c', 'd']
for v in zip_longest(a, b):
    print(v)

This section is about divide a task to many small pipelines(steps), by use of generator Generator is a producer, for loop is a comsumer.

Why this function is not included in itertools module?

from collections import Iterable
def  flatten(items, ignored_types=(str, bytes)):
    for v in items:
	if isinstance(v, Iterable) and not isinstance(v, ignored_types):
	    yield from flatten(v, ignored_types)
	else:
	    yield v

a = [1, 2, [3, 4, [5, 6], 7],  8, 'abc']
for v in a:
    print(v)

print("the flattened version")
for v in flatten(a):
    print(v)

the input iterables should in sorted order. then it will create an new iterable of sorted items from all input.

a = [1, 4, 8]
b = [2, 3,  7, 9]

import heapq
for v in heapq.merge(a, b):
    print(v)

The function will only get the needed items into memory. So it better to merge two sorted files.

Similar to sorted(itertools.chain(*iterables)), but will not read all content to memory.

invoke the callable UNTIL it returns the sentinel

Means: repeated invoke the callable, and return its return value, until the return value equal to the sentinel.

a = [1, 2, 3, 4, 5]
idx = -1
def foo():
    global idx
    idx+=1
    return a[idx]

for v in iter(foo, 3):
    print(v)