Question 415961 in Html Question posted in
Some basic documentation can be found here.

Unable to get the value of an element in HTML in Python and Selenium

I’m trying to get DE(GERMANY) and IN(INDIA) printed in to a list using class="lnsTableCell lnsTableCell--left" from the html below so that I can create a CSV later.

<div role="gridcell" tabindex="0" data-test-subj="dataGridRowCell" class="euiDataGridRowCell euiDataGridRowCell--stripe euiDataGridRowCell--firstColumn" style="position: absolute; left: 0px; top: 73px; height: 34px; width: 100px;">
    <div class="euiDataGridRowCell__expandFlex">
        <div class="euiDataGridRowCell__expandContent">
            <div data-datagrid-cellcontent="true" class="euiDataGridRowCell__truncate">
                <div data-test-subj="lnsTableCellContent" class="lnsTableCell lnsTableCell--left">IN(INDIA)</div>
            </div><p class="euiScreenReaderOnly">Row: 2; Column: 1</p>
        </div>
        ...

<div role="gridcell" tabindex="0" data-test-subj="dataGridRowCell" class="euiDataGridRowCell euiDataGridRowCell--stripe euiDataGridRowCell--firstColumn" style="position: absolute; left: 0px; top: 73px; height: 34px; width: 100px;">
    <div class="euiDataGridRowCell__expandFlex">
        <div class="euiDataGridRowCell__expandContent">
            <div data-datagrid-cellcontent="true" class="euiDataGridRowCell__truncate">
                <div data-test-subj="lnsTableCellContent" class="lnsTableCell lnsTableCell--left">DE(GERMANY)</div>
            </div><p class="euiScreenReaderOnly">Row: 3; Column: 1</p>
        </div>

This is my code:

def capture_data_1():
   url_scrape = "https://kibana-xxyyzzz.com/app/dashboards#/view/8d3382c0-49cd-11ed-b4e3-3d78365d2c94?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-1w,to:now))"
        chrome_options = webdriver.ChromeOptions()
        chrome_options.add_argument('--no-sandbox')
        chrome_options.add_argument('--headless')
        chrome_options.add_argument('--disable-dev-shm-usage') # moves to /tmp
        chrome_options.add_argument('--disable-gpu')
        driver = webdriver.Chrome(options=chrome_options)
        driver.get(kibana_eu_prd.login_url)
        wait = WebDriverWait(driver, 20)
        wait.until(EC.element_to_be_clickable((By.NAME, "username"))).send_keys(kibana_user_id)
        wait.until(EC.element_to_be_clickable((By.NAME, "password"))).send_keys(kibana_user_pass)
        submit_button = wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "euiButton__text")))
        submit_button.click()
        time.sleep(5)

        driver.get(url_scrape)
      #  countries = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.lnsTableCell.lnsTableCell--left"))) # Works but only print INDIA
        countries = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="i19971251-9fb6-11ef-b33e-c91f069ea574"]/div/div/div/div[2]/div/div/div/div')))
        print(countries.text)
        for country in countries:
            print(country.text)
capture_data_1()

The error I am getting is:

Traceback (most recent call last):
  File "data_scrape.py", line 89, in capture_data_1
    countries = wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="i19971251-9fb6-11ef-b33e-c91f069ea574"]/div/div/div/div[2]/div/div/div/div')))
  File "/var/lib/jenkins/.local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 105, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Stacktrace:
#0 0x5609b1b7c10a <unknown>
#1 0x5609b1697460 <unknown>
#2 0x5609b16e3f96 <unknown>
#3 0x5609b16e4221 <unknown>
#4 0x5609b17295d4 <unknown>
#5 0x5609b170803d <unknown>
#6 0x5609b1726af6 <unknown>
#7 0x5609b1707db3 <unknown>
#8 0x5609b16d6c10 <unknown>
#9 0x5609b16d7bee <unknown>
#10 0x5609b1b4839b <unknown>
#11 0x5609b1b4c338 <unknown>
#12 0x5609b1b35ebc <unknown>
#13 0x5609b1b4ceb7 <unknown>
#14 0x5609b1b1b10f <unknown>
#15 0x5609b1b6a828 <unknown>
#16 0x5609b1b6a9f0 <unknown>
#17 0x5609b1b7af86 <unknown>
#18 0x7f40f2386609 start_thread


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "data_scrape.py", line 109, in <module>
    capture_data_1()
  File "data_scrape.py", line 104, in capture_data_1
    raise Exception(error)
Exception: Message:
Stacktrace:
#0 0x5609b1b7c10a <unknown>
#1 0x5609b1697460 <unknown>
#2 0x5609b16e3f96 <unknown>
#3 0x5609b16e4221 <unknown>
#4 0x5609b17295d4 <unknown>
#5 0x5609b170803d <unknown>
#6 0x5609b1726af6 <unknown>
#7 0x5609b1707db3 <unknown>
#8 0x5609b16d6c10 <unknown>
#9 0x5609b16d7bee <unknown>
#10 0x5609b1b4839b <unknown>
#11 0x5609b1b4c338 <unknown>
#12 0x5609b1b35ebc <unknown>
#13 0x5609b1b4ceb7 <unknown>
#14 0x5609b1b1b10f <unknown>
#15 0x5609b1b6a828 <unknown>
#16 0x5609b1b6a9f0 <unknown>
#17 0x5609b1b7af86 <unknown>
#18 0x7f40f2386609 start_thread

I have tried the following which only led to other types of errors:

#        countries = driver.find_element(By.CLASS_NAME, "lnsTableCell.lnsTableCell--left")
#        countries = driver.find_elements(By.XPATH, '//*[@id="i19971251-9fb6-11ef-b33e-c91f069ea574"]/div/div/div/div[2]/div/div/div/div')
#        countries = driver.find_elements(By.CSS_SELECTOR, ".lnsTableCell.lnsTableCell--left")
#        print(wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.lnsTableCell.lnsTableCell--left"))).text)
Tags:

2

Answers


  1. 0

    If you have only 2 countries as you named in the query, then you may use indexing or text, etc.
    Eg of indexing:

    # Germany    
"(//div[@data-test-subj='lnsTableCellContent'])[2]"

#India
"//div[@data-test-subj='lnsTableCellContent'])[1]"

    On the contrary, if you have many countries in the list, then you may loop through them.

    ls_countries = driver.find_element(By.XPATH, "//div[@data-test-subj='lnsTableCellContent']")
    listed_countries = []
    if len(ls_countries) > 0:
        for country in ls_countries:
            txt_country = country.text
            listed_coutries.append(txt_country)
    print(listed_countries)

    Hope this helps!

    Login or Signup to reply.
  2. 0

    I would like to add one note on the Anand’s smart answer.

    When you are to get a list of elements, you should use "find_elements" instead of "find_element".

    Hope this also helps!

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search