Well, i tried to use docker for the first time. I’ve needed it to make selenium run on server. So i just could drop this script to any server and it would work with no pain. But suddenly it just does not work, it just does not. I’ve tried to search on google, but nothing. Now i’m here searching for help. Maybe i missed something, maybe i messed up somewhere, now i see the solutiion. The solution is to download chromedriver, rejecting the idea of running selenium-standalone.
Starting selenium
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
# options.add_argument('--verbose')
options.add_argument('--ignore-gpu-blacklist')
options.add_argument('--use-gl')
options.add_argument("--no-sandbox")
options.add_argument('--disable-web-security')
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument('--user-agent={}'.format(random.choice(headers.headers)))
driver = webdriver.Remote("http://selenium:4444/wd/hub", options=options, desired_capabilities=DesiredCapabilities.CHROME)
Dockerfile goes:
FROM python:3.8.12
ENV PYTHONBUFFERED 1
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN mkdir /app
COPY ./app /app
WORKDIR /app
Docker compose goes:
version: '3'
services:
selenium:
image: selenium/standalone-chrome
ports:
- 4444:4444
restart: always
app:
build:
context: .
volumes:
- ./app:/app
command: sh -c "python3 main.py"
depends_on:
- selenium
Now i get the error
Selenium logs with this weird SIGTERM
Starting Selenium Grid Standalone...
18:13:41.962 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
18:13:41.969 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
18:13:42.973 INFO [NodeOptions.getSessionFactories] - Detected 8 available processors
18:13:43.035 INFO [NodeOptions.report] - Adding chrome for {"browserVersion": "96.0","browserName": "chrome","platformName": "Linux","se:vncEnabled": true} 1 times
18:13:43.055 INFO [Node.<init>] - Binding additional locator mechanisms: relative, name, id
18:13:43.090 INFO [LocalDistributor.add] - Added node 2100e52d-e04f-44e7-84e9-71a0e936cd32 at http://172.18.0.2:4444. Health check every 120s
18:13:43.092 INFO [GridModel.setAvailability] - Switching node 2100e52d-e04f-44e7-84e9-71a0e936cd32 (uri: http://172.18.0.2:4444) from DOWN to UP
18:13:43.286 INFO [Standalone.execute] - Started Selenium Standalone 4.1.1 (revision e8fcc2cecf): http://172.18.0.2:4444
18:14:02.359 INFO [LocalDistributor.newSession] - Session request received by the distributor:
[Capabilities {browserName: chrome, goog:chromeOptions: {args: [headless, --ignore-certificate-errors, --ignore-ssl-errors, --ignore-gpu-blacklist, --use-gl, --no-sandbox, --disable-web-security, --user-agent={'user-agent':...], excludeSwitches: [enable-logging], extensions: []}, pageLoadStrategy: normal}]
Starting ChromeDriver 96.0.4664.45 (76e4c1bb2ab4671b8beba3444e61c0f17584b2fc-refs/branch-heads/4664@{#947}) on port 39843
On[l1y6 4l0o2c8a3l2 4c2o.n4n3e5c]t[iSoEnVsE RaEr]e: ablilnodw(e)d .f
Plaeialseed :s eCea nhntottp sa:s/s/icghnr ormeeqdureisvteerd. cahdrdormeisusm .(o9r9g)/
security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
18:14:03.044 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
18:14:03.080 INFO [LocalDistributor.newSession] - Session created by the distributor. Id: 9f36e0e8d2e2417b85b035b546082c87, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.110, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /tmp/.com.google.Chrome.k2Wjlf}, goog:chromeOptions: {debuggerAddress: localhost:40301}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:cdp: ws://172.18.0.2:4444/sessio..., se:cdpVersion: 96.0.4664.110, se:vnc: ws://172.18.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Trapped SIGTERM/SIGINT/x so shutting down supervisord...
2021-12-23 18:14:54,146 WARN received SIGTERM indicating exit request
2021-12-23 18:14:54,147 INFO waiting for xvfb, vnc, novnc, selenium-standalone to die
2021-12-23 18:14:55,148 INFO stopped: selenium-standalone (terminated by SIGTERM)
2021-12-23 18:14:56,151 INFO stopped: novnc (terminated by SIGTERM)
2021-12-23 18:14:57,154 INFO stopped: vnc (terminated by SIGTERM)
2021-12-23 18:14:57,154 INFO waiting for xvfb to die
2021-12-23 18:14:58,156 INFO stopped: xvfb (terminated by SIGTERM)
Shutdown complete
And finally script error
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='selenium', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ece62a100>: Failed to establish a new connection: [Errno 111] Connection refused'))
I just can not tell why it crashes. Why it couldn’t make the connection.
2
Answers
This error message…
…implies that all the Capabilities are perfecto but the user-agent. Hence you see the MaxRetryError:
Solution
You need to pass a valid UserAgent through
add_argument()
as follows:References
You can find a couple of relevant detailed discussion in:
Found an answer here: