skip to Main Content

Well, i tried to use docker for the first time. I’ve needed it to make selenium run on server. So i just could drop this script to any server and it would work with no pain. But suddenly it just does not work, it just does not. I’ve tried to search on google, but nothing. Now i’m here searching for help. Maybe i missed something, maybe i messed up somewhere, now i see the solutiion. The solution is to download chromedriver, rejecting the idea of running selenium-standalone.

Starting selenium

options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
# options.add_argument('--verbose')
options.add_argument('--ignore-gpu-blacklist')
options.add_argument('--use-gl')
options.add_argument("--no-sandbox")
options.add_argument('--disable-web-security')
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument('--user-agent={}'.format(random.choice(headers.headers)))
driver = webdriver.Remote("http://selenium:4444/wd/hub", options=options, desired_capabilities=DesiredCapabilities.CHROME)

Dockerfile goes:

FROM python:3.8.12

ENV PYTHONBUFFERED 1
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt

RUN mkdir /app
COPY ./app /app
WORKDIR /app

Docker compose goes:

version: '3'

services:
  selenium:
    image: selenium/standalone-chrome
    ports:
      - 4444:4444
    restart: always

  app:
    build:
      context: .
    volumes:
      - ./app:/app
    command: sh -c "python3 main.py"
    depends_on:
      - selenium

Now i get the error
Selenium logs with this weird SIGTERM

Starting Selenium Grid Standalone...

18:13:41.962 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding

18:13:41.969 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing

18:13:42.973 INFO [NodeOptions.getSessionFactories] - Detected 8 available processors

18:13:43.035 INFO [NodeOptions.report] - Adding chrome for {"browserVersion": "96.0","browserName": "chrome","platformName": "Linux","se:vncEnabled": true} 1 times

18:13:43.055 INFO [Node.<init>] - Binding additional locator mechanisms: relative, name, id

18:13:43.090 INFO [LocalDistributor.add] - Added node 2100e52d-e04f-44e7-84e9-71a0e936cd32 at http://172.18.0.2:4444. Health check every 120s

18:13:43.092 INFO [GridModel.setAvailability] - Switching node 2100e52d-e04f-44e7-84e9-71a0e936cd32 (uri: http://172.18.0.2:4444) from DOWN to UP

18:13:43.286 INFO [Standalone.execute] - Started Selenium Standalone 4.1.1 (revision e8fcc2cecf): http://172.18.0.2:4444

18:14:02.359 INFO [LocalDistributor.newSession] - Session request received by the distributor: 

 [Capabilities {browserName: chrome, goog:chromeOptions: {args: [headless, --ignore-certificate-errors, --ignore-ssl-errors, --ignore-gpu-blacklist, --use-gl, --no-sandbox, --disable-web-security, --user-agent={'user-agent':...], excludeSwitches: [enable-logging], extensions: []}, pageLoadStrategy: normal}]

Starting ChromeDriver 96.0.4664.45 (76e4c1bb2ab4671b8beba3444e61c0f17584b2fc-refs/branch-heads/4664@{#947}) on port 39843

On[l1y6 4l0o2c8a3l2 4c2o.n4n3e5c]t[iSoEnVsE RaEr]e:  ablilnodw(e)d .f

Plaeialseed :s eCea nhntottp sa:s/s/icghnr ormeeqdureisvteerd. cahdrdormeisusm .(o9r9g)/

security-considerations for suggestions on keeping ChromeDriver safe.

ChromeDriver was started successfully.

18:14:03.044 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C

18:14:03.080 INFO [LocalDistributor.newSession] - Session created by the distributor. Id: 9f36e0e8d2e2417b85b035b546082c87, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.110, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /tmp/.com.google.Chrome.k2Wjlf}, goog:chromeOptions: {debuggerAddress: localhost:40301}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:cdp: ws://172.18.0.2:4444/sessio..., se:cdpVersion: 96.0.4664.110, se:vnc: ws://172.18.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}

Trapped SIGTERM/SIGINT/x so shutting down supervisord...

2021-12-23 18:14:54,146 WARN received SIGTERM indicating exit request

2021-12-23 18:14:54,147 INFO waiting for xvfb, vnc, novnc, selenium-standalone to die

2021-12-23 18:14:55,148 INFO stopped: selenium-standalone (terminated by SIGTERM)

2021-12-23 18:14:56,151 INFO stopped: novnc (terminated by SIGTERM)

2021-12-23 18:14:57,154 INFO stopped: vnc (terminated by SIGTERM)

2021-12-23 18:14:57,154 INFO waiting for xvfb to die

2021-12-23 18:14:58,156 INFO stopped: xvfb (terminated by SIGTERM)

Shutdown complete

And finally script error

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='selenium', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ece62a100>: Failed to establish a new connection: [Errno 111] Connection refused'))

I just can not tell why it crashes. Why it couldn’t make the connection.

2

Answers


  1. This error message…

    [Capabilities {browserName: chrome, goog:chromeOptions: {args: [headless, --ignore-certificate-errors, --ignore-ssl-errors, --ignore-gpu-blacklist, --use-gl, --no-sandbox, --disable-web-security, --user-agent={'user-agent':...], excludeSwitches: [enable-logging], extensions: []}, pageLoadStrategy: normal}]
    

    …implies that all the Capabilities are perfecto but the . Hence you see the MaxRetryError:

    urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='selenium', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ece62a100>: Failed to establish a new connection: [Errno 111] Connection refused'))
    

    Solution

    You need to pass a valid UserAgent through add_argument() as follows:

    from fake_useragent import UserAgent
    
    options = webdriver.ChromeOptions()
    ua = UserAgent()
    userAgent = ua.random
    print(userAgent)
    options.add_argument(f'user-agent={userAgent}')
    

    References

    You can find a couple of relevant detailed discussion in:

    Login or Signup to reply.
  2. Found an answer here:

    What is killing the pod?

    We have readiness & liveness probe (Health check) defined with timeout
    of: 5 secs & failureThreshold (retry): 3 (default) for selenium-hub.
    Sometime liveness probe didn’t response within given 5 secs for all 3
    retries which tells openshift to take care of un-healthy selenium-hub
    pod by restarting (openshift deployment/deployment-config) or moving
    it to not-ready state (openshift pod).

    What is the Fix:

    Increased probes timeout from 5 to 30 secs &
    failureThreshold from 3 to 5 as recommended in
    https://github.com/SeleniumHQ/docker-selenium/tree/selenium-3 – Adding
    a HEALTHCHECK to the Grid fixed the issue.

    If you see the fix, it’s too simple. But analyzing this issue was
    really superb experience. Once again thank you Diego Molina

    Few suggestions if you are running your test in containerized env (to
    easier the analysis if you encounter any failures):

    1. Set hub & chrome-node log level to "FINE" or "ALL" (Default: Info)
    2. Collect & store all pod logs (hub, chrome & test pod) once execution completed.
    3. Collect & store all pods event logs once execution completed.
    4. Collect & store all pods status once execution completed.
    5. Add readiness & liveness probe for both hub & node
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search