Question 95811 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Selenium returns an empty html body – Amazon web services

I am running selenium from AWS lambda, to automate a process, I was getting an error for element not found, and when I printed the page source it seems like it is actually empty, I have tried sleep for 30 seconds, without success, this works as expected locally

from time import sleep
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.chrome.options import Options
    from secretsx import get_secret_values



def main():
 chrome_options = Options()  
chrome_options.binary_location = '/opt/headless-chromium'
chrome_options.add_argument("--headless")  
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--single-process')
chrome_options.add_argument('--disable-dev-shm-usage')

driver = webdriver.Chrome('/opt/chromedriver',options=chrome_options)

driver.get(mypage)
driver.maximize_window()
driver.implicitly_wait(30) 

print(driver.page_source) #this returns <html xmlns="http://www.w3.org/1999/xhtml"><head></head><body></body></html>
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    the issue was actually related to SSL certs, you can setup selenium to ignore it

    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities


desired_capabilities = DesiredCapabilities.CHROME.copy()
desired_capabilities['acceptInsecureCerts'] = True

driver = webdriver.Chrome('/opt/chromedriver',options=chrome_options,desired_capabilities=desired_capabilities)

    I hope this is helpful for somebody haha since I spent 6 hours looking for an answer


  2. 0

    You saved me from spending a few hours on this problem. My code looked like this:

        options = Options()
    options.add_argument('--headless')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-gpu')
    options.add_argument('--disable-dev-shm-usage')    
    desired_capabilities = DesiredCapabilities.CHROME.copy() 
    desired_capabilities['acceptInsecureCerts'] = True
    browser = webdriver.Chrome('chromedriver', options=options, desired_capabilities=desired_capabilities)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search