代码之家 › 专栏 › 技术社区 › SIM

不能用“加载更多”按钮来显示所有的头条新闻

web-scraping selenium-webdriver selenium python-3.x python

3

SIM · 技术社区 · 7 年前

我试着一直点击 more 按钮位于网页底部(在其登录页中),用于显示所有标题。问题是,当我执行脚本时,它只单击一次然后停止。我怎么能一直点击那个按钮,直到没有更多的选择点击?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://www.newsnow.co.uk/h/Sport/Football/Championship/Transfer+News"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)

while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[class^='hl_more']")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except Exception: break
driver.quit()

1 回复 | 直到 7 年前

1

2

Andersson 7 年前

尝试以下代码来模拟所需的行为:

header = driver.find_element_by_id("phead")
driver.execute_script('arguments[0].style.position = "absolute";', header)
while True:
    try:    
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.hl_more.bg_tween:not(.nfloading)")))
        driver.execute_script("arguments[0].scrollIntoView();",loadmore)
        loadmore.click()
    except: break