Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – Seleniumbase not logging

I am trying to capture responses using Selenium and undetected, this is working normally using those two:

import undetected_chromedriver as uc

capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}

driver = uc.Chrome(headless=False, desired_capabilities=capabilities)
driver.get(url)

time.sleep(5)

logs_raw = driver.get_log("performance")

This is working correctly and it logs all the requests and responses, however it does not work in headless mode, so I switched to use Seleniumbase

from seleniumbase import Driver

driver = Driver(uc=True, log_cdp=True, headless=True)
driver.get(url)

time.sleep(5)

logs_raw = driver.get_log("performance")
print(logs_raw)

As my understanding log_cdp is same as "goog:loggingPrefs" https://github.com/seleniumbase/SeleniumBase/issues/2220, however nothing gets logged, both in headless and non-headless and I am not sure what I am doing wrong.
The page gets loaded bypassing the captcha displaying all the data I need.

seleniumbase 4.33.7
Google Chrome 131.0.6778.85
Windows 11 using WSL Ubuntu 22.0.4 LTS
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I solved by not using Seleniumbase and instead reverting to my previous attempt by installing xvfbwrapper and xvgb

    pip install xvfbwrapper

sudo apt install xvfb

    and change my code like this:

    capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}

with Xvfb() as xvfb:
    driver = uc.Chrome(desired_capabilities=capabilities)
    driver.get(url)

 # rest of the logic

  2. 0

    There’s a bit more work involved for that, but SeleniumBase CDP Mode is the key to collecting all the requests/responses. Eg:

    import colorama
import mycdp
import sys
from seleniumbase import SB

c1 = colorama.Fore.BLUE + colorama.Back.LIGHTYELLOW_EX
c2 = colorama.Fore.BLUE + colorama.Back.LIGHTGREEN_EX
cr = colorama.Style.RESET_ALL
if "linux" in sys.platform:
    c1 = c2 = cr = ""


async def send_handler(event: mycdp.network.RequestWillBeSent):
    r = event.request
    s = f"{r.method} {r.url}"
    for k, v in r.headers.items():
        s += f"nt{k} : {v}"
    print(c1 + "*** ==> RequestWillBeSent <== ***" + cr)
    print(s)


async def receive_handler(event: mycdp.network.ResponseReceived):
    print(c2 + "*** ==> ResponseReceived <== ***" + cr)
    print(event.response)


with SB(uc=True, test=True, locale_code="en") as sb:
    sb.activate_cdp_mode("about:blank")
    sb.cdp.add_handler(mycdp.network.RequestWillBeSent, send_handler)
    sb.cdp.add_handler(mycdp.network.ResponseReceived, receive_handler)
    url = "https://seleniumbase.io/apps/calculator"
    sb.cdp.open(url)
    sb.sleep(1)

    P.S. I’m the maintainer of SeleniumBase, so I know a few tricks.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search