I tried to follow an example how to parse websites via python and selenium.
But I am running always into the following problem: calling the function webdriver.Firefox
opens a firefox instance, but no website via get could be called, it seems: the whole code is blocking in function Firefox (see: print("open call never reached")) The browser is opening and after ca. 30 seconds an exception causes the broswer to exit, with message:
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Possible firefox version mismatch. You must use GeckoDriver instead for Firefox 48+. Profile Dir: /tmp/tmpl5dm_azd If you specified a log_file in the FirefoxBinary constructor, check it for details
So what do I am wrong here ? How could I set the profile right ?
I tried to set marionette mode True, but got the error : "Unable to find a matching set of capabilities"
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = False
options = Options()
options.log.level = "trace"
options.headless = True
binary = FirefoxBinary("/usr/bin/firefox")
pathDriver = "./geckodriver"
testUrl="https://duckduckgo.com/"
print("will create firefox instance")
browser = webdriver.Firefox(firefox_binary=binary,options=options,capabilities=cap,executable_path=pathDriver)
print("open call never reached")
browser.get(testUrl)
webdriver.quit()
My test environment:
$ name -a
Linux 5.5.0-0.bpo.2-amd64 #1 SMP Debian 5.5.17-1~bpo10+1 (2020-04-23) x86_64 GNU/Linux
Also I downloaded the latest selenium and the geckodriver
here see what versions I do use:
$ python3 –version
Python 3.7.3
$ pip3 freeze | grep sel
selenium==3.141.0
$ geckodriver -V
geckodriver 0.27.0 (7b8c4f32cdde 2020-07-28 18:16 +0000)
$ which firefox
/usr/bin/firefox
$ firefox -v
Mozilla Firefox 68.10.0esr
2
Answers
you added the parentheses for DesiredCapabilities
or you can use webdriver_manager library which will help to get rid of a lot of headaches
and use it like this
this setup helps you have the latest browser version for selenium, your error could be caused by the unmatching versions
When using GeckoDriver to initiate/spawn a new Browsing Context i.e. Firefox Browser session with Firefox 48+ versions, you have to use Marionette mandatorily.
Solution
The solution would be either to work with default setting of marionette or turn
marionette
to True as follows: