Question 83401 in Docker Question posted in
The official documentation can be found here.

ImportError: cannot import name 'soft_unicode' from 'markupsafe' – Docker

I am trying to build a docker container with Airflow and Postgres nevertheless getting many errors during build as shown below. I’ve tried to downgrade markupsafe in my requirements.txt as shown below, but it didn’t help. What can I do to solve the issue?

# I tried this version:
markupsafe==2.1.1
# and then also this one:
markupsafe==2.0.1

Error message:

ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)

My docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:11.2
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"

  webserver:
    image: puckel/docker-airflow:1.10.4
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.4
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./sparkify/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      - ./sparkify/plugins:/usr/local/airflow/plugins
    ports:
      - "8081:8081"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

  requirements:
    image: python:3.8-alpine
    volumes:
      - .:/sparkify
    working_dir: /sparkify
    command: pip3 install -r requirements.txt

My requirements.txt:

markupsafe==2.0.1

Full stack trace:

Creating network "airflowwithpostgres_default" with the default driver
Creating airflowwithpostgres_postgres_1     ... done
Creating airflowwithpostgres_requirements_1 ... done
Creating airflowwithpostgres_webserver_1    ... done
Attaching to airflowwithpostgres_requirements_1, airflowwithpostgres_postgres_1, airflowwithpostgres_webserver_1
postgres_1      | The files belonging to this database system will be owned by user "postgres".
postgres_1      | This user must also own the server process.
postgres_1      | 
webserver_1     | Tue May 10 18:38:28 UTC 2022 - waiting for Postgres... 1/20
postgres_1      | The database cluster will be initialized with locale "en_US.utf8".
postgres_1      | The default database encoding has accordingly been set to "UTF8".
postgres_1      | The default text search configuration will be set to "english".
postgres_1      | 
postgres_1      | Data page checksums are disabled.
postgres_1      | 
postgres_1      | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1      | creating subdirectories ... ok
postgres_1      | selecting default max_connections ... 100
postgres_1      | selecting default shared_buffers ... 128MB
postgres_1      | selecting dynamic shared memory implementation ... posix
postgres_1      | creating configuration files ... ok
postgres_1      | running bootstrap script ... ok
postgres_1      | performing post-bootstrap initialization ... ok
postgres_1      | syncing data to disk ... 
postgres_1      | WARNING: enabling "trust" authentication for local connections
postgres_1      | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1      | --auth-local and --auth-host, the next time you run initdb.
postgres_1      | ok
postgres_1      | 
postgres_1      | Success. You can now start the database server using:
postgres_1      | 
postgres_1      |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1      | 
postgres_1      | waiting for server to start....2022-05-10 18:38:29.428 UTC [44] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1      | 2022-05-10 18:38:29.441 UTC [45] LOG:  database system was shut down at 2022-05-10 18:38:29 UTC
postgres_1      | 2022-05-10 18:38:29.446 UTC [44] LOG:  database system is ready to accept connections
postgres_1      |  done
postgres_1      | server started
postgres_1      | CREATE DATABASE
postgres_1      | 
postgres_1      | 
postgres_1      | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1      | 
postgres_1      | 2022-05-10 18:38:29.859 UTC [44] LOG:  received fast shutdown request
postgres_1      | waiting for server to shut down....2022-05-10 18:38:29.861 UTC [44] LOG:  aborting any active transactions
postgres_1      | 2022-05-10 18:38:29.865 UTC [44] LOG:  background worker "logical replication launcher" (PID 51) exited with exit code 1
postgres_1      | 2022-05-10 18:38:29.865 UTC [46] LOG:  shutting down
postgres_1      | 2022-05-10 18:38:29.882 UTC [44] LOG:  database system is shut down
postgres_1      |  done
postgres_1      | server stopped
postgres_1      | 
postgres_1      | PostgreSQL init process complete; ready for start up.
postgres_1      | 
postgres_1      | 2022-05-10 18:38:29.974 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgres_1      | 2022-05-10 18:38:29.974 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1      | 2022-05-10 18:38:29.976 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1      | 2022-05-10 18:38:29.989 UTC [62] LOG:  database system was shut down at 2022-05-10 18:38:29 UTC
postgres_1      | 2022-05-10 18:38:29.994 UTC [1] LOG:  database system is ready to accept connections
requirements_1  | Collecting markupsafe==2.1.1
requirements_1  |   Downloading MarkupSafe-2.1.1-cp38-cp38-musllinux_1_1_x86_64.whl (29 kB)
requirements_1  | Installing collected packages: markupsafe
requirements_1  | Successfully installed markupsafe-2.1.1
requirements_1  | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
airflowwithpostgres_requirements_1 exited with code 0
postgres_1      | 2022-05-10 18:38:33.377 UTC [69] LOG:  incomplete startup packet
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
postgres_1      | 2022-05-10 18:38:35.020 UTC [70] LOG:  incomplete startup packet
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1     |     from airflow import configuration
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module>
webserver_1     |     from airflow.models import DAG
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module>
webserver_1     |     from airflow.models.baseoperator import BaseOperator  # noqa: F401
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module>
webserver_1     |     import jinja2
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module>
webserver_1     |     from jinja2.environment import Environment, Template
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module>
webserver_1     |     from jinja2 import nodes
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module>
webserver_1     |     from jinja2.utils import Markup
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module>
webserver_1     |     from markupsafe import Markup, escape, soft_unicode
webserver_1     | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py)
airflowwithpostgres_webserver_1 exited with code 1
Tags:

12

Answers


  1. 0

    It turns out markupsafe removed soft_unicode which is causing this error! https://github.com/pallets/markupsafe/issues/304

    Add MarkupSafe==2.0.1 to your PYTHON_DEPS like so:

      webserver:
    image: puckel/docker-airflow:1.10.4
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.4
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: MarkupSafe==2.0.1

    Then run docker-compose down then docker-compose up -d --build and all services should be running!

    If you get another importerror, you can add the packagae to PYTHON_DEPS like so:

    PYTHON_DEPS: MarkupSafe==2.0.1 example==1.1.9
    Login or Signup to reply.
  2. 0

    Downgrade markupsafe to 2.0.1

    pip install markupsafe==2.0.1

    Login or Signup to reply.
  3. 0

    I am using Python 3.7.6 on Linux, in my case, I just upgrade the flask version and everything works again:

    pip install -U flask
    Login or Signup to reply.
  4. 0

    This did not fix the issue alone for me.

    pip install markupsafe==2.0.1

    The following did:

    1. I uninstalled and reinstalled zmq.
    2. I installed markupsafe 2.0.1 version.
    3. Open jupyter notebook
    pip uninstall zmq
pip install zmq
pip install markupsafe==2.0.1
jupyter notebook
    Login or Signup to reply.
  5. 0

    In Mac OS X marports with Python 3.8 I solved it downgrading this package with:

    sudo port activate py38-markupsafe @2.0.1_0
    Login or Signup to reply.
  6. 0

    for anyone who downgraded and still faced issue.

    go into directory where already exists delete it, the reinstall downgraded version.

    pip install markupsafe==2.0.1 itself will not fix issue unless already existing file for markup is deleted.

    Login or Signup to reply.
  7. 0

    As seen from your logs the root of the cause is jinja2 (older?) being incompatible with its dependency markupsafe (newer?). Something keeps jinja2 pinned at an older, incompatible version. You can either downgrade the package which is "too new" (as most answers typically suggest as a workaround in such cases, thus creating tech debt that will come back to bite you), or (better) upgrade the package that is "too old", here: jinja2, to the latest version.

    For instance, this bug can be reproduced by simply pinning jinja
    to an incompatible version (e.g. jinja2==2.11.3 – a very popular "latest working" version pin:), while keeping other connected (i.e. reverse dependent, also recursively) packages at their latest versions (e.g. Flask==2.2.2, itsdangerous==2.1.2, Werkzeug==2.2.2, …). Unpinning jinja2 and letting it be upgraded (to 3.1.2) solves the ImportError.

    Notice also that if you print reverse dependencies of MarkupSafe, like that:

    # pip install pipdeptree
pipdeptree -r -p markupsafe

    you will see which minimum versions of this package are expected by Jinja2:

    # for pinned (obsolete) Jinja2: `MarkupSafe` has been automatically
# downgraded  by `pip` to `2.0.1` - hence the other answers 
# to this question:
MarkupSafe==2.0.1
  - Jinja2==2.11.3 [requires: MarkupSafe>=0.23]

#...versus unpinned:

# for unpinned (latest) Jinja2: using latest version of `MarkupSafe` 
# (in the same simple container as above)
MarkupSafe==2.1.1
  - Jinja2==3.1.2 [requires: MarkupSafe>=2.0]
    Login or Signup to reply.
  8. 0

    I was troubleshooting my own project after reinstalling my requirements and instead of simply downgrading that package, you can also fix the application, if you are willing to do so.

    As seen here, the solution simply seems to be to rename soft_unicode into soft_str.

    Login or Signup to reply.
  9. 0

    Solution 1: Upgrading markupsafe module

    pip install MarkupSafe

    Solution 2: Downgrading markupsafe module to 2.0.1 version

    pip install markupsafe==2.0.1

    Solution 3: Upgrading aws-sam-cli module to Latest Version

    pip install --user --upgrade aws-sam-cli

    Solution 4: Downgrading aws-sam-cli module to Latest Version

    pip install --user aws-sam-cli ==1.37.0`
    Login or Signup to reply.
  10. 0

    This worked in MAC and it will auto uninstall the 2.1.1 and again install 2.0.1 :

    pip install markupsafe==2.0.1
    Login or Signup to reply.
  11. 0

    Looks like 2.0.1 is not working right now, downgrading to 2.0.0 worked for me

    pip install markupsafe==2.0.0
    Login or Signup to reply.
  12. 0

    Whoever is facing this problem in 2023 when opening Jupyter Notebook, just upgrade Jinja2

    pip install --upgrade Jinja2

    downgrading MarkeupSafe did not work because other packages need its latest version

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search