Apache Beam not recognizing custom container arguments – Docker

I’m trying to use a local docker image to run a Beam pipeline, but it looks like this image has not been recognized even after I follow the Beam documentation suggested steps (https://beam.apache.org/documentation/runtime/environments/).

I performed the following steps:

1. Created a Dockerfile with my custom dependencias (pypostal and fuzzywuzzy):

Dockerfile

FROM apache/beam_python3.7_sdk:2.25.0

## System Dependencies
RUN apt-get update && apt-get upgrade -y && apt-get clean

ENV TZ=America
RUN DEBIAN_FRONTEND="noninteractive" apt-get -y install tzdata

# Python package management and basic dependencies
RUN apt-get install -y curl python3.7 python3.7-dev python3.7-distutils build-essential graphviz git-all
# Register the version in alternatives
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.7 1
## Create User Directory
RUN mkdir -p /home/user


# LIBPOSTAL
# Install Libpostal dependencies
RUN apt-get update &&
    apt-get install -y 
        git 
        make 
        curl 
        autoconf 
        automake 
        libtool 
        pkg-config

# Download libpostal source to /usr/local/libpostal
RUN cd /usr/local && 
    git clone https://github.com/openvenues/libpostal

# Create Libpostal data directory at /var/libpostal/data
RUN cd /var && 
    mkdir libpostal && 
    cd libpostal && 
    mkdir data

# Install Libpostal from source
RUN cd /usr/local/libpostal && 
    ./bootstrap.sh && 
    ./configure --datadir=/var/libpostal/data && 
    make -j4 && 
    make install && 
  ldconfig

# Python Packages
COPY requirements.txt /requirements.txt 

# Install Pip Requirements
RUN pip install -r requirements.txt


ENV PYTHONPATH "${PYTHONPATH}:/home/user"

WORKDIR /home/user

requirements.txt

fuzzywuzzy
postal

2. Created a pipeline.py file with the following Beam pipeline code:

import apache_beam as beam
import argparse
from apache_beam.options.pipeline_options import PipelineOptions
from apache_beam.options.pipeline_options import SetupOptions

from postal.expand import expand_address
from postal.parser import parse_address
from fuzzywuzzy import fuzz

def run(argv=None, save_main_session=True):
    parser = argparse.ArgumentParser()
    known_args, pipeline_args = parser.parse_known_args(argv)
    pipeline_options = PipelineOptions(pipeline_args)
    pipeline_options.view_as(SetupOptions).save_main_session = save_main_session


    addresses_examples = [
        "465 windward pkwy, alpharetta, georgia, u.s.a.",
        "2018 colby taylor drive, 40475, richmond, usa",
        "19-21 city road, chester ,chester ch1 3ae"
        "no.12 lishi hutong, chaoyangmen ; nei nanxiaoj",
        "building b, 25 yuan da road haidian district,"
    ]

    class ParseAddress(beam.DoFn):

        def process(self, text):
            yield parse_address(text)


    with beam.Pipeline(options=pipeline_options) as p:
        plants = (
          p
          | 'Adresses' >> beam.Create(addresses_examples)
          | 'Parser' >> beam.ParDo(ParseAddress())
          | beam.Map(print))

if __name__ == '__main__':
    run()

3. Ran the script using the command:

python3 -m pipeline --runner=PortableRunner --environment_type="DOCKER" --environment_config="beam-text:0.1" --job_endpoint=embed

(beam-text:0.1 is my image name)

But I still receiving the error message:

No module named 'apache_beam'

It sounds that Beam is ignoring my custom container arguments.