代码之家  ›  专栏  ›  技术社区  ›  Karn Kumar

python根据主机名从复合日志文件中分离行

  •  0
  • Karn Kumar  · 技术社区  · 6 年前

    我正在寻找一种方法来读取日志文件,并根据第5列读取主机名,并将主机名与所有迭代保持在一起,直到新名称出现,或者说,在找到新名称时只标记一个双空格,但它需要打印整行。

    只是读取文件:

    $ cat test.py
    with open("file", "r") as f:
        for line in f:
            fr = f.read()
            print(fr)
    

    文件内容输出:

    $ ./test.py
    Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
    Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002   not responding, still trying
    Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001   not responding, still trying
    Jul 17 04:34:08 anujv kernel: nfs: server hpstor003  not responding, still trying
    Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003  not responding, still trying
    Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003  not responding, still trying
    Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003  not responding, still trying
    

    期望输出:

    Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
    
    
    Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
    Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
    
    
    Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
    

    或者再多点化妆品:

    hw067 
    Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
    Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
    
    hw068 
    Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
    Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
    
    hw069
    Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
    Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
    

    除了用普通的蟒蛇方式,我们能用蟒蛇熊猫来实现这一点吗? -谢谢

    1 回复  |  直到 6 年前
        1
  •  0
  •   user10029838    6 年前

    你可以通过itertools和operator函数,我是一个alos学习者,我从我的一个帖子的ans中借用了代码,但是我收集了一些关于这两个函数的细节,希望这两个功能都有用。

    itertools.groupby(iterable,key=none或some func) 获取iterables列表,并根据指定的键对其进行分组。 键指定要应用于每个iterable的操作,然后将其结果用作每个项目分组的标题; 最终具有相同“key”值的项将在同一组中结束。 返回值是一个类似于字典的iterable,其形式为key:value。

    operator.itemgetter(n) 构造一个假定可iterable对象(例如list、tuple、set)作为输入的可调用对象,并从中提取第n个元素。

    operator.itemgetter(n) 将提供:一个从类似列表的对象中获取项的函数。 像operator.itemgetter(3)从列表中获取第三个项目。

    更准确地说,这些是可调用的,而不是函数,但这是一个经常被忽略的区别。

    以下代码适用于您:

    import itertools
    import operator
    
    log_list = []
    
    file = open('file')
    for line in file:
        loggs = line.strip().split()
        log_list.append(loggs)
    
    key = operator.itemgetter(3)
    # this step is only required if the list is not sorted by the key
    log_list.sort(key=key)
    for index, logs in itertools.groupby(log_list, key):
        print(index)
        for log in logs:
            print("    " + " ".join(log))
        print("")
    

    以上代码的结果,我拿了你的文件进行测试。

    anujv
        Jul 17 04:34:08 anujv kernel: nfs: server hpstor003 not responding, still trying
    
    hw060
        Jul 18 15:46:02 hw060 kernel: nfs: server hpstor001 not responding, still trying
    
    hw067
        Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002 not responding, still trying
        Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001 not responding, still trying
        Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001 not responding, still trying
        Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001 not responding, still trying
    
    hw068
        Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002 not responding, still trying
        Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001 not responding, still trying
        Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001 not responding, still trying
    
    hw069
        Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001 not responding, still trying
        Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001 not responding, still trying
        Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001 not responding, still trying
        Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001 not responding, still trying
    
    hw070
        Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001 not responding, still trying
        Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002 not responding, still trying
    
    hw071
        Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002 not responding, still trying
        Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002 not responding, still trying
        Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002 not responding, still trying
    
    hw073
        Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002 not responding, still trying
    
    hw074
        Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001 not responding, still trying
        Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002 not responding, still trying
        Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002 not responding, still trying
    
    hw079
        Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001 not responding, still trying
        Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001 not responding, still trying
    
    sanujv
        Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003 not responding, still trying
        Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003 not responding, still trying
        Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003 not responding, still trying