代码之家  ›  专栏  ›  技术社区  ›  pyeR_biz

代码未按预期写入子目录,内存使用率很高

  •  -1
  • pyeR_biz  · 技术社区  · 6 年前

    import xmltodict
    import numpy as np
    import pandas as pd
    from collections import Counter
    import os
    import glob
    import re
    
    #from list of directories, iterate through each directory and apply my functions    
    def change_dir (directories):
        for i in directories:
            os.chdir(i)  
            return parse_xml (i)
    
    #Read xml files from each subdir,parse and append to a list            
    def parse_xml (subdirectory):
        doc = []
        for i in glob.glob(os.path.join(subdirectory, '*.xml')):
            with open(i, 'r') as fd:
                doc.append(xmltodict.parse(fd.read()))
        return make_pandas(doc)
    
    #Take list of each subdir xml files, create required df and write csv to subdir
    def make_pandas (list_parsedFiles):
        logData_list = []
        for xmlval in list_parsedFiles:
            channel_list = xmlval['logs']['log']['logData']['mnemonicList'].split(",")
            temp = [i.split(",") for i in xmlval ['logs']['log']['logData']['data']]
            temp.insert(0, xmlval ['logs']['log']['logData']['unitList'].split(","))
            logData_list.extend(temp)
            df = pd.DataFrame(np.array(logData_list).reshape(len(logData_list),len(channel_list)), columns = channel_list).drop_duplicates()
            df.to_csv('out.csv')
    
    %%capture
    tocsv = change_dir (list_smallLogs)
    
    1 回复  |  直到 6 年前
        1
  •  1
  •   John Zwinck    6 年前

    你的代码被破坏了因为你 chdir() glob(directory) ,这意味着您要在中查找文件 foo/foo , bar/bar chdir() -这是不好的做法,您应该只在每个函数调用中指定路径(正如您已经做过的那样)。