代码之家 › 专栏 › 技术社区 › Amen Aziz

熊猫中的数据被覆盖

web-scraping dataframe pandas python

Amen Aziz · 技术社区 · 2 年前

当我使csv文件数据在csv文件中被覆盖时,如果有任何解决方案,请为我提供页面链接 https://www.aeafa.es/asociados.php?provinput=&_pagi_pg=1 已经在这里搜索了答案,在谷歌上花了很长时间,但什么都没有。。。我已经尝试用“w”而不是“r”或“a”打开文件,但我仍然无法让代码

import requests
from bs4 import BeautifulSoup
import pandas as pd

headers ={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36'
}
for page in range(1,3): 
    r =requests.get('https://www.aeafa.es/asociados.php?provinput=&_pagi_pg={page}'.format(page=page),
    headers=headers) 
    soup=BeautifulSoup(r.content, 'lxml')
    tag=soup.find_all('div',class_='col-md-8 col-sm-8')
    temp=[]
    for pro in tag:
        data=[tup.text for tup in pro.find_all('p')]
      
        DirecciÃ³n=data[2]
        DirecciÃ³n=DirecciÃ³n[12:]
        
    
        PoblaciÃ³n=data[3]
        PoblaciÃ³n=PoblaciÃ³n[14:]

    

        Provincia=data[4]
        Provincia=Provincia[14:] 

    
        TelÃ©fono=data[5]
        TelÃ©fono="+" + TelÃ©fono[11:].replace('.', "")
        


        Email=data[6]
        Email=Email[10:]
        
        
        

        temp.append([DirecciÃ³n,Provincia,PoblaciÃ³n,TelÃ©fono, Email])
    df=pd.DataFrame(temp,columns=["DirecciÃ³n","Provincia","PoblaciÃ³n","TelÃ©fono","Email"])
    df.to_csv('samp.csv')

1 回复 | 直到 2 年前

Andrej Kesely 2 年前

尝试放置列表 temp 在for循环之外。然后,在所有循环完成后创建数据帧:

import requests
import pandas as pd
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"
}

temp = []

for page in range(1, 3):
    r = requests.get(
        "https://www.aeafa.es/asociados.php?provinput=&_pagi_pg={page}".format(
            page=page
        ),
        headers=headers,
    )
    soup = BeautifulSoup(r.content, "lxml")
    tag = soup.find_all("div", class_="col-md-8 col-sm-8")
    for pro in tag:
        data = [tup.text for tup in pro.find_all("p")]

        DirecciÃ³n = data[2]
        DirecciÃ³n = DirecciÃ³n[12:]

        PoblaciÃ³n = data[3]
        PoblaciÃ³n = PoblaciÃ³n[14:]

        Provincia = data[4]
        Provincia = Provincia[14:]

        TelÃ©fono = data[5]
        TelÃ©fono = "+" + TelÃ©fono[11:].replace(".", "")

        Email = data[6]
        Email = Email[10:]

        temp.append([DirecciÃ³n, Provincia, PoblaciÃ³n, TelÃ©fono, Email])

df = pd.DataFrame(
    temp, columns=["DirecciÃ³n", "Provincia", "PoblaciÃ³n", "TelÃ©fono", "Email"]
)
df.to_csv("samp.csv")

print(len(df))

打印:

LibreOffice截图: