生成器与迭代器,merge可迭代对象合并函数
分类:编程应用

pyextend库-merge可迭代指标合併函数,pyextend-merge

pyextend - python extend lib

merge (iterable1, *args)

参数: 

iterable1: 实现 __iter__的可迭代对象, 如 str, tuple, dict, list

*args: 其他实现 __iter__的可迭代对象

返回值:

集结后的迭代对象

使用楷模:

    Example 1:
        source = ['a', 'b', 'c']
        result = merge(source, [1, 2, 3])
        self.assertEqual(result, ['a', 'b', 'c', 1, 2, 3])

        result = merge(source, [1, 2, 3], ['x', 'y', 'z'])
        self.assertEqual(result, ['a', 'b', 'c', 1, 2, 3, 'x', 'y', 'z'])

    Example 2:
        source = 'abc'
        result = merge(source, '123')
        self.assertEqual(result, 'abc123')

        result = merge(source, '123', 'xyz')
        self.assertEqual(result, 'abc123xyz')

    Example 3:
        source = ('a', 'b', 'c')
        result = merge(source, (1, 2, 3))
        self.assertEqual(result, ('a', 'b', 'c', 1, 2, 3))

        result = merge(source, (1, 2, 3), ('x', 'y', 'z'))
        self.assertEqual(result, ('a', 'b', 'c', 1, 2, 3, 'x', 'y', 'z'))

    Example 4:
        source = {'a': 1, 'b': 2, 'c': 3}
        result = merge(source, {'x': 'm', 'y': 'n'}, {'z': '1'})
        self.assertEqual(result, {'a': 1, 'b': 2, 'c': 3, 'x': 'm', 'y': 'n', 'z': '1'})

代码:

@accepts(iterable1='__iter__')
def merge(iterable1, *args):
    """
    Returns an type of iterable1 value, which merged after iterable1 used *args

    :exception TypeError: if any parameter type of args not equals type(iterable1)

    """

    result_list = list(iterable1) if not isinstance(iterable1, dict) else eval('list(iterable1.items())')

    for i, other in enumerate(args, start=1):
        if not isinstance(other, type(iterable1)):
            raise TypeError('the parameter type of index {} not equals type of index 0'.format(i))
        if not isinstance(other, dict):
            result_list[len(result_list):len(result_list)] = list(other)
        else:
            result_list[len(result_list):len(result_list)] = list(other.items())

    if isinstance(iterable1, str):
        return ''.join(result_list)
    elif isinstance(iterable1, tuple):
        return tuple(result_list)
    elif isinstance(iterable1, dict):
        return dict(result_list)
    else:
        return result_list

 

pyextend - python extend lib merge (iterable1, *args) 参数: iterable1: 实现 __iter__的可迭代对象, 如 str, t...

[toc]

4 迭代器与生成器

for xxx in xxx理解

__iter__几乎是for xx in xx 此设计的
  • for xxx in xxx在迭代器中动用
  1. 调用iter()方法,重临多个对象 itmes = iter(object)
  2. 调用此指标 items.next(), 捕捉到StopIteration后 停止
########for xxx in xxx理解
###########1
it = iter(lines) # it = lines.__iter__()
    while True:
        try:
            next(it)
        except StopIteration:
            print 'finish iteration'
            break
################2
class ABC(object):
    def __iter__(self):
        return ABC2()

class ABC2(object):
    num = 100
    def next(self):
        self.num -= 1
        if self.num < 0:
            raise StopIteration()
        return self.num

for i in ABC():
    print i
  • for xxx in xxx在生成器
    运作生成器内部代码,将yield的每个值抽取
####for xxx in xxx理解
def abc(x):
    print 123
    yield x

for i in abc('hello world')
    print i
#abc('hello world')内置__iter__,next,send,throw方法

##类的实现
class ABC(object):
    def __iter__(self):
        return self.abc('hello world')
    def abc(self, x):
        print 123
        yield x

for i in ABC():
    print i
#和下面的等同
items = iter(ABC())
while True:
    try:
        print items.next()
    except StopIteration:
        break

迭代器

  • 在python中落到实处了__iter__艺术的目的足以迭代的,即能够调用内置函数iter()方法,return对象.
  • 福寿年高了next()方法的对象实迭代器,获得iter()重临的目的,不断调用next()方法
class Fib(object):
    def __init__(self):
        self.a, self.b = 0, 1

    def __iter__(self):
        return self

    def next(self):
        self.a, self.b = self.b, self.a + self.b
        if self.a > 100:
            raise StopIteration()
        return self.a

fetch = iter(Fib()) #获取用于迭代的对象
while True:
    #不断调用next(),捕捉到StopIteration结束循环
    try:
        i = fetch.next()
        print i
    except StopIteration:
        break

#等同for xxx in XXX      
for i in Fib():
    print i

生成器

生成器是对象.保留栈帧的上下文

def abc(x):
    print 123
    yield x
abc("hello world")  # 此时并不会打印出123, 这一步执行产生一个生成器对象
abc("hello world").next() #此时才去执行生成器内的代码

4.4 完成深度优先的遍历树形节点的生成器

  • for 循环的明亮
  • yield 别的对象的 yield
class Node(object):
    def __init__(self, value):
        self._value = value
        self._children =[]

    def __repr__(self):
        return "Node{!r}".format(self._value)

    def __iter__(self):
        return iter(self._children)

    def add_child(self, node):
        return self._children.append(node)

    def depth_first(self):
    """
    yield 相于return, 再次调用的时候,从原先的地方执行
    将其他的对象的yield后 再次yield出来,分为两次
    """
        yield self
        for c in self:
            #下面的代码相当于yield from c.depth_first()
            for items in c.depth_first():
                yield items

if __name__ == '__main__':

    root = Node(0)
    child1 = Node(1)
    child2 = Node(2)

    child11 = Node(11)
    child21 = Node(21)

    root.add_child(child1)
    root.add_child(child2)

    child1.add_child(child11)
    child2.add_child(child21)

    child11.add_child(Node(100))
    child21.add_child(Node(200))
    child21.add_child(Node(201))

"""
对于for循环的理解:
ch 为形参,接受 root.depth_first() 返还的实参(相当于return)
root.depth_first()必须实现迭代协议,可以为生成器
"""
    for ch in root.depth_first():
        print ch

depth_first() 方法首先重临(yield)自身并迭代每两个节点的depth_first()方法,并赶回(yield)对应成分

  • 历史观情势的达成,短处繁琐
class Node2(object):
    def __init__(self, value):
        self._value = value
        self._children = []

    def __repr__(self):
        return "Node{!r}".format(self._value)

    def __iter__(self):
        return iter(self._children)

    def add_child(self, node):
        return self._children.append(node)

    def depth_first(self):
        return DepthFirstIterator(self)


class DepthFirstIterator(object):
    def __init__(self, start_node):
        self._node = start_node
        self._children_iter = None
        self._child_iter = None

    def __iter__(self):
        return self

    def __next__(self):
        if self._children_iter is None:
            self._children_iter = iter(self._node)
            return self._node

        elif self._child_iter:
            try:
                nextchild = next(self._child_iter)
                return nextchild
            except StopIteration:
                self._child_iter = None
                return next(self)

        else:
            self._child_iter = next(self._children_iter).depth_first()
            return next(self)

4.5 反向迭代

  • 采用内置的函数 reversed()
  • 必得兑现内置的reversed()方法
class Countdown(object):
    def __init__(self, start):
        self.start = start

    def __iter__(self):
        n = self.start
        while n > 0:
            yield n
            n -= 1

    def __reversed__(self):
        n = 1
        while n <= self.start:
            yield n
            n += 1


if __name__ == '__main__':
    for rr in reversed(Countdown(30)):
        print rr
    for rr in Countdown(30):
        print rr

4.6 带有外部参数生成器函数

from collections import deque

class LineHistory:
    def __init__(self, lines, hislen=3):
        self.lines = lines
        self.history = deque(maxlen=hislen)

    def __iter__(self):
        for lineno, line in enumerate(self.lines, 1):
            self.history.append((lineno, line))
            yield line

    def clear(self):
        self.history.clear()

4.7 迭代器切成片

赢得迭代器生成的切块对象

import itertools
def count(n):
    while True:
        yield n
        n += 1
c = count(0)
# c[10:20]  >>>TypeError: 'generator' object has no attribute '__getitem__'
for items in itertools.islice(c, 10, 21):
    print items

函数 islice() 重回三个方可变动钦点成分的迭代器,它经过遍
历并舍弃直到切成条起首索引地方的兼具因素。然后才开头一个个的归来成分,并直到切块截至索引地点。劣势不可能重复使用迭代器里面包车型客车数码

4.8 跳过没有须要的迭代部分

???跳过三个可迭代对象的发端有的,对前面包车型地铁不影响?
创办一个迭代器,
只要函数predicate(item)为True,就吐弃iterable中的项,
比方predicate重返False,就能够生成iterable中的项和具有继续项。

from itertools import dropwhile
with open('manage.py') as f:
    for line in dropwhile(lambda line: line.startwith("#"), f):
        print line

4.9 排列组合实现

譬喻排列A23 ,组合 C23等

from itertools import permutations,combinations, combinations_with_replacement
items = ['a', 'b', 'c']
for c in permutations(items) # 排列A33
for c in permutations(items, 2) # 排列A33
for c in combinations(items, 3) # 组合 C23
for c in combinations_with_replacement(items, 3) # 同一元素重复使用 3*3*3

4.10 种类上索引迭代

my_list = ['a', 'b', 'c']
for idx, val in enumerate(my_list, 1):
print(idx, val)

这种情况在您遍历文件时想在错误消息中利用行号定位时候极度有效:

def parse_data(filename):
    with open(filename, 'rt') as f:
        for lineno, line in enumerate(f, 1):
            fields = line.split()
            try:
                count = int(fields[1])
                ...
            except ValueError as e:
                print('Line {}: Parse error: {}'.format(lineno, e))

data = [ (1, 2), (3, 4), (5, 6), (7, 8) ]
for n, (x, y) in enumerate(data):

4.11 迭代多少个连串 zip()

zip() 会成立叁个迭代器来作为结果重回

  • 着力用法 压缩
a = [1, 2, 3]
b = ['w', 'x', 'y', 'z']
for i in zip(a,b):
    print(i)
>>>(1,'w')
>>>(2,'x')
>>>(3,'y')

from itertools import zip_longest
for i in zip_longest(a,b,fillvalue=None):
print(i)
>>>(1, 'w')
>>>(2, 'x')
>>>(3, 'y')
>>>(None, 'z')
  • 打包字典,变为列表
headers = ['name', 'shares', 'price']
values = ['ACME', 100, 490.1]
s = dict(zip(headers,values))

list(zip(headers, values))
  • zip() 能够承受多于三个的行列的参数 zip(a, b, c)

4.12 分化集合上元节素的迭代 chain()

from itertools import chain
a = [1, 2, 3, 4]
b = ['x', 'y', 'z']
for x in chain(a, b):
    print(x)

a,b可以为差异的门类 chain(set,list)乃至是chain(dict,list)

# Inefficent
for x in a + b:
# Better
for x in chain(a, b):

率先种方案中, a + b 操作会创设一个全新的体系并需求 a 和 b 的体系一致
chian() 不会有这一步,所以只要输入类别相当大的时候会很省外部存款和储蓄器。何况当可迭代
目的类型不一样的时候 chain() 同样可以很好的职业。

4.13 创设数量管道

os.walk 从文件夹某些地方上马遍历

# x为当前的目录 y为当前目录下包含的文件夹 z 为当前目录下的文件
for x, y, z, in os.walk(r"D:Workspacesell"):
    for zpieces in z :
        print '{}{}'.format(x,zpieces)

fnmatch.filter(filellist, filepat)
filelist为list则赶回符合filepart的公文
filelist为str 则赶回布尔值

# encoding:utf-8
import os
import fnmatch
import gzip
import bz2
import re


def gen_find(filepat, top):
    """
    根据filepat的文件类型,查找当前目录下的文件
    """
    for path, dirlist, filelist in os.walk(top):
            # 过滤符合格式的地址并返回
        for name in fnmatch.filter(filelist, filepat):
            yield os.path.join(path, name)  # 文件的绝对地址的生成器


def gen_opener(filenames):
    """
    打开文件,yield文件,并关闭
    """
    for filename in filenames:  # 从生成器中取出绝对地址 filename为地址  filenames为含地址的生成器
        if filename.endswith('.gz'):
            f = gzip.open(filename, 'rt')
        elif filename.endswith('.bz2'):
            f = bz2.open(filename,"rt")
            ##todo 可能有问题
        else:
            f = open(filename, "r") 
        yield f  # 文件对象的生成器 
        f.close()


def gen_concatenate(iterators):
    for it in iterators: #it为文件对象,iterators是文件对象生成器
        for items in it: # items 句子 it文件对象
            yield items # 抛出句子生成器 在外部用for xx in xx得到


def gen_grep(pattern, lines):
    """
    匹配文中的语句
    """
    pat = re.compile(pattern)
    for line in lines:
        if pat.search(line):
            yield line


lognames = gen_find("*.py", r"D:Workspacesell")
files = gen_opener(lognames)
lines = gen_concatenate(files)

pylines = gen_grep(r'^class ', lines)  # 打印类名
for line in pylines:
    print line
#todo 不太懂
#bytecolumn = (line.rsplit(None,1)[1] for line in pylines)
#bytes = (int(x) for x in bytecolumn if x != '-')
#print('Total', sum(bytes))

看不懂嵌套的生成器,请看上边包车型客车事例

def gen1 ():
    for i in [[1,2,3,4,5],[6,7,8,9,0]]:
        yield i
def gen2 (i):
    for j in i:
        for k in j:
            yield k
g1 = gen1()
g2 = gen2(g1)
for x in g2:
    print x

不太可靠的知道 ,for xx in xx 可以解开生成器,要想获得生成器里的剧情,for xx in xx 层数大于生成器嵌套的层数。

4.14 递归生成器展开嵌套的队列

原代码选择 yield from 完毕python2 不帮助可用 for i in xx : yield i 替代

# encoding:utf-8
from collections import Iterable
def flatten(items, ignore_types=(str, bytes)):
    for x in items:
    #isinstance(x, Iterable) 判断是否可以迭代 ,可以则继续递归
    #not isinstance(x, ignore_types),排除字符串,字节,这两者也可以迭代
        if isinstance(x, Iterable) and not isinstance(x, ignore_types):
            for i in flatten(x):
                yield i
        else:
            yield x
items1 = [1, 2, [3, 4, [5, 6], 7], 8]
items2 = ['Dave', 'Paula', ['Thomas', 'Lewis']]
l1 = [x for x in flatten(items1)]
l2 = [x for x in flatten(items2)]

4.15 有序对象合併再排序

heapq.merge()
heapq.merge 生成器迭代脾气意味着它不会立马读取全体类别。那就表示你能够在非
常长的队列中选择它,而不会有太大的付出

import heapq
a = [1, 4, 7, 10]
b = [2, 5, 6, 11]
l = [x for x in heapq.merge(a, b)] ##heapq.merge(a, b)是生成器
>>>[1, 2, 4, 5, 6, 7, 10, 11]

4.16 迭代器替代while循环

其实正是用遍历替代while.
路子:iter(functiong, status)能够迭代,具体参照他事他说加以考察本节末尾
常见的IO程序,伪代码

CHUNKSIZE = 8192
def reader(s):
    while True:
        data = s.recv(CHUNKSIZE)
        if data == b'':
            break
        process_data(data)

f = open("views.py", "r")
reader(f)

#用iter()循环代替
def reader2(s):
    for chunk in iter(lambda : s.recv(CHUNKSIZE),b""):
        pass
        #process_data(data)

实例代码

import sys
f = open("views.py","r")
for chunk in iter(lambda: f.read(10), ""):
    n = sys.stdout.write(chunk)

iter()内置函数:
单参数时Iter(func),fun对象帮忙迭代公约,不然报错
五个参数时Iter(func,arg),它接受叁个可选的 callable 对象和贰个标识(结
尾) 值作为输入参数,不断调用next(),func重返值和标志同样时,抛出StopIteration

x = 0
def func():
    global x
    x +=1
    print x
    return x

while True:
    i = iter(func,100)
    try:
        i.next() #renturn值为100的时候抛出StopIteration
    except StopIteration:
        print '停止迭代'
        break

本章计算

  • 迭代器
    迭代器左券差不离是为 for xx inxx设计的,
    什么样是迭代器呢?遵从__iter__,next()这三个斟酌的目的。即为__iter__针对迭代的某些对象,那些指标有next()方法。不断的调用next
    (),抛出Stopiteration迭代甘休。
  • 生成器
    关联太多不在本章张开

本文由正版必中一肖图发布于编程应用,转载请注明出处:生成器与迭代器,merge可迭代对象合并函数

上一篇:教学Python的Scrapy爬虫框架使用代理进行采撷的章 下一篇:常用python编制程序模板汇总
猜你喜欢
热门排行
精彩图文