Ubuntu – Chrome driver of selenium in headless mode not finding any element

I try to make a simple selenium test in a remote server (Ubuntu 22.04) without display mode. The chrome browser version is 120.0.6099.129 and the chrome driver version is 120.0.6099.109. The result is that the selenium is not able to find any element.
The error message is:

Traceback (most recent call last):

File "/home/japy/repos/myseleniumproject/main.py", line 37, in
username_locator = WebDriverWait(driver,100).until(EC.presence_of_element_located((By.ID, ‘fname’)))
File "/home/japy/repos/myseleniumproject/python-selenium-venv/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 105, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException

In order to get the element I tried to wait until the element is visible but then I am getting time out error.
The script I am currently using is this one:

import time
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import os
import logging

log_path = './log'
logger = logging.getLogger('selenium')
logger.setLevel(logging.DEBUG)
handler = logging.FileHandler(log_path)
logger.addHandler(handler)
logging.getLogger('selenium.webdriver.common').setLevel(logging.DEBUG)
logger.debug("this is detailed debug information:")

open(log_path, 'r')

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless=new")
driver = webdriver.Chrome(options=chrome_options)

# Go to webpage
driver.get("https://practicetestautomation.com/practice-test-login/")

#Type username student into username field
try:
    username_locator = WebDriverWait(driver,100).until(EC.presence_of_element_located((By.ID, 'username')))
finally:
    username_locator.send_keys('student')
print("username text field got")
# Type password Password12 into Password field
password_locator = driver.find_element(By.NAME, "password")
password_locator.send_keys('Password123')

# Push submit button
button = driver.find_element(By.XPATH, "//button[@class='btn']")
button.click()

The log I am getting back from the execution is the following:

this is detailed debug information:
Selenium Manager binary found at: /home/japy/repos/myseleniumproject/python-selenium-venv/lib/python3.10/site-packages/selenium/webdriver/common/linux/selenium-manager
Executing process: /home/japy/repos/myseleniumproject/python-selenium-venv/lib/python3.10/site-packages/selenium/webdriver/common/linux/selenium-manager –browser chrome –debug –output json
chromedriver not found in PATH
chrome detected at /usr/bin/google-chrome
Running command: /usr/bin/google-chrome –version
Output: "Google Chrome 120.0.6099.129 "
Detected browser: chrome 120.0.6099.129
Required driver: chromedriver 120.0.6099.109
chromedriver 120.0.6099.109 already in the cache
Driver path: /home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver
Browser path: /usr/bin/google-chrome
Using driver at: /home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver
Started executable: /home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver in a child process with pid: 3530206 using 0 to output -3
POST http://localhost:47287/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "browserVersion": null, "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/google-chrome", "args": ["–headless=new", "i", "disable-gpu", "window-size=1200,1100"]}}}}
Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"chrome","browserVersion":"120.0.6099.129","chrome":{"chromedriverVersion":"120.0.6099.109 (3419140ab665596f21b385ce136419fde0924272-refs/branch-heads/6099@{#1483})","userDataDir":"/tmp/.org.chromium.Chromium.5IAYJP"},"fedcm:accounts":true,"goog:chromeOptions":{"debuggerAddress":"localhost:45055"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"linux","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"428b77b50f61a5a83c4ff1ecc6d7371b"}} | headers=HTTPHeaderDict({‘Content-Length’: ‘850’, ‘Content-Type’: ‘application/json; charset=utf-8’, ‘cache-control’: ‘no-cache’})
Finished Request
POST http://localhost:47287/session/428b77b50f61a5a83c4ff1ecc6d7371b/url {"url": "https://www.practicetestautomation.com/practice-test-login/"}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({‘Content-Length’: ’14’, ‘Content-Type’: ‘application/json; charset=utf-8’, ‘cache-control’: ‘no-cache’})
Finished Request
GET http://localhost:47287/session/428b77b50f61a5a83c4ff1ecc6d7371b/url {}
Remote response: status=200 | data={"value":"https://www.practicetestautomation.com/practice-test-login/"} | headers=HTTPHeaderDict({‘Content-Length’: ’71’, ‘Content-Type’: ‘application/json; charset=utf-8’, ‘cache-control’: ‘no-cache’})
Finished Request
POST http://localhost:47287/session/428b77b50f61a5a83c4ff1ecc6d7371b/element {"using": "css selector", "value": "[id="username"]"}
Remote response: status=404 | data={"value":{"error":"no such element","message":"no such element: Unable to locate element: {"method":"css selector","selector":"[id="username"]"}n (Session info: chrome=120.0.6099.129)","stacktrace":"#0 0x5579fed4ef83 u003Cunknown>n#1 0x5579fea07cf7 u003Cunknown>n#2 0x5579fea5799e u003Cunknown>n#3 0x5579fea57aa1 u003Cunknown>n#4 0x5579feaa2d64 u003Cunknown>n#5 0x5579fea810dd u003Cunknown>n#6 0x5579feaa0006 u003Cunknown>n#7 0x5579fea80e53 u003Cunknown>n#8 0x5579fea48dd4 u003Cunknown>n#9 0x5579fea4a1de u003Cunknown>n#10 0x5579fed13531 u003Cunknown>n#11 0x5579fed17455 u003Cunknown>n#12 0x5579fecfff55 u003Cunknown>n#13 0x5579fed180ef u003Cunknown>n#14 0x5579fece399f u003Cunknown>n#15 0x5579fed3c008 u003Cunknown>n#16 0x5579fed3c1d7 u003Cunknown>n#17 0x5579fed4e124 u003Cunknown>n#18 0x7f1c50e94b43 u003Cunknown>n"}} | headers=HTTPHeaderDict({‘Content-Length’: ‘869’, ‘Content-Type’: ‘application/json; charset=utf-8’, ‘cache-control’: ‘no-cache’})
Finished Request

Then the last block is repeated over and over again.