how to make selenium click a button like
<a href="javascript:void(0)" id="ajax_more_button">さらに読み込む</a>
literally, have no ‘button’ type to it?
Hello, I’m trying to scrape urls from https://www.yomiuri.co.jp/editorial/, a news website.
To have access to more articles, there’s a button さらに読み込む(read more).
I tried to click the button with selenium with the following code which returned an error message below.
wait = WebDriverWait(driver, 10)
button = wait.until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "さらに読み込む")))
button.click()
the whole error message:
ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (377, 485). Other element would receive the click:
(Session info: chrome=114.0.5735.199)
Stacktrace:
Backtrace:
GetHandleVerifier [0x0072A813+48355]
(No symbol) [0x006BC4B1]
(No symbol) [0x005C5358]
(No symbol) [0x005F5062]
(No symbol) [0x005F3B98]
(No symbol) [0x005F239B]
(No symbol) [0x005F179E]
(No symbol) [0x005E9F5C]
(No symbol) [0x0060A73C]
(No symbol) [0x005E9A36]
(No symbol) [0x0060AA94]
(No symbol) [0x0061C922]
(No symbol) [0x0060A536]
(No symbol) [0x005E82DC]
(No symbol) [0x005E93DD]
GetHandleVerifier [0x0098AABD+2539405]
GetHandleVerifier [0x009CA78F+2800735]
GetHandleVerifier [0x009C456C+2775612]
GetHandleVerifier [0x007B51E0+616112]
(No symbol) [0x006C5F8C]
(No symbol) [0x006C2328]
(No symbol) [0x006C240B]
(No symbol) [0x006B4FF7]
BaseThreadInitThunk [0x75C87D59+25]
RtlInitializeExceptionChain [0x771AB74B+107]
RtlClearBits [0x771AB6CF+191]
I have no idea how to do that, maybe there’s a better library to do that in Python?
2
Answers
The さらに読み込む element is a dynamic element, so to click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using PARTIAL_LINK_TEXT:
Using CSS_SELECTOR:
Using XPATH:
Note: You have to add the following imports :
Browser snapshot:
Here’s the complete solution:
output:
Few things to note:
20 articlesunder the最新ニュースsection.さらに読み込む, it loads 10 more articles and so on.driver.execute_script("return arguments[0].click()", element)if count==100:statement or increase the count number to load a given number of articles. Please notice that as every click loads 10 more articles, the value of the variablecountwill be a multiple of 10 starting from 20. (20, 30, 40, 50,….and so on)