skip to Main Content

I have the following .gitlab-ci.yml file :

image: python:3.8

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths:
    - .cache/pip
    - venv/

before_script:
  - python -V  # Print out python version for debugging
  - pip install --upgrade pip
  - pip install virtualenv
  - virtualenv venv
  - source venv/bin/activate
  - pip install -r requirements.txt
  - cp properties_ci properties
  - apt-get update
  - apt-get install net-tools

stages:
  - build
  - test
  - security
  - leanix

include:
  - ...

test:unit:
  stage: ...

test:integration:
    stage: test
    script:
        - echo "0"
        - python app.py &
        - curl 127.0.0.1:8126
        - py.test tests/integration/test_integration.py
    services:
        - name: cassandra:3.11

When I launch my python application by using the command python app.py I can see the following output :

$ python app.py
WARNING:cassandra.cluster:Cluster.__init__ called with contact_points specified, but no load_balancing_policy. In the next major version, this will raise an error; please specify a load-balancing policy. (contact_points = ['cassandra'], lbp = None)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
Creating keyspace if not exist...
Creating tables if not exist...
 * Serving Flask app 'src' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:8126

So connecting to Cassandra and creating/inserting data works well, but the integration tests cannot access the python application on localhost.

In my integration tests I call http://127.0.0.1:8126 but it says the following :

Creating keyspace if not exist...
=========================== short test summary info ============================
ERROR tests/integration/test_integration.py - requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8126): Max retries exceeded with url: /getCSRFToken (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f316d3df700>: Failed to establish a new connection: [Errno 111] Connection refused'))
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 3.05s =============================== 

Any ideas how to call localhost inside the docker container so that I can access my application ? I tried also 127.0.0.1 and it does not work.

EDIT 1

The script test_integration.py contains as a small example the following portion of code :

...
url = "http://localhost:8126/series/"

import zipfile
with zipfile.ZipFile("tests/integration/20230323.zip", mode="r") as archive:
  #Iterate through files in zip file
  for zipfilename in archive.filelist:
    txtdata = archive.read(zipfilename)
    if (zipfilename.filename == 'OP_lastchange_CRUDE_2023-03-23.json'):
      payload2 = json.dumps(json.loads(txtdata))
      response = requests.request("POST", url, headers=headers, data=payload2)
      assert response.status_code == 204, "Santa Claus is not coming this year !"
      
... 

EDIT 2

The solution is given by @KamilCuk – setting the python app to run in the background in the section before_script like python app.py & and sleeping for several seconds makes my python application available to be called like http://localhost:8126.

2

Answers


  1. Chosen as BEST ANSWER

    One of the working solutions, in my case, proposed by KamilCuk, is the following :

    image: python:3.8
    
    variables:
      PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
    
    cache:
      paths:
        - .cache/pip
        - venv/
    
    before_script:
      - ...
      - python app.py &
      - sleep 20 # wait for application to start-up
    
    stages:
      - build
      - run
      - test
      - security
      - leanix
    
    include:
      - project: ...
    
    services:
      - name: cassandra:3.11
        alias: cassandra
    
    test:unit:
      stage: ...
    
    test:integration:
      stage: test
      script:
        - echo "0"
        - py.test tests/integration/test_series.py
    

    The solution is given by @KamilCuk - setting the python app to run in the background in the section before_script like python app.py & and sleeping for several seconds sleep 20 makes my python application available to be called like http://localhost:8126.

    I need to investigate the other solution proposed by KamilCuk using fixtures.


  2. Instead consider fixtures. Put the following in your conftest.py:

    @pytest.fixture(scope="session")
    def app(request):
       pp = subprocess.popen("python app.py")
       time.sleep(5) # TODO: replace with waiting for open port
       request.addfinalizer(lambda: (pp.terminate(), pp.wait()))
       return pp
    

    Then in tests use:

    def test_something(app):
         pass
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search