skip to Main Content

I am very new to Gitlab CI/CD and I have read its documentation very carefully about creating a new CI/CD process using .gitlab-ci.yml file. As I have found out in order to have Continuous Deployment (also known as CD), it is needed to register a new gitlab-runner on my linux server.

Explanation

Here is my .gitlab-ci.yml file:

stages:
  - build
  - deploy

docker-build:
  image: docker:stable
  services:
    - docker:dind
  stage: build
  only:
    refs:
      - ci-test
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build --pull -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
    - echo $CI_REGISTRY_IMAGE
    - echo $CI_COMMIT_SHORT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

deploy:
  image: docker:stable
  stage: deploy
  services:
    - docker:dind
  only:
    refs: 
      - ci-test
  when: manual
  except:
    changes:
      - "*.md"
  script:
    - docker build --pull -t $CI_REGISTRY_IMAGE$CI_COMMIT_SHORT_SHA .
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

and this is my Dockerfile:

version: '3'

services:
  app:
#    build: .
    environment:
      - DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
#    command: uwsgi --http :8000 --socket /socket/api.sock --chmod-socket=666 --module _base.wsgi --master --processes 5 --threads 2
    command: uwsgi --http :8000 --module _base.wsgi --master --processes 5 --threads 2
#    command: python manage.py runserver 0.0.0.0:8000
    depends_on:
      - redis
    ports:
      - 8000:8000
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
    restart: on-failure
    network_mode: host

  redis:
    image: redis:6.0-alpine
    volumes:
      - /var/lib/redis/redis.dump:
    ports:
      - 6379:6379
#
  celery-beat:
    environment:
      - DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base beat -l info
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}

  celery-worker-default:
    environment:
      - DEFAULT_DATABASE_HOST=172.17.0.1
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base worker -l INFO -Q clearsight-default --concurrency 1 -n clearsight-default
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}


  celery-worker-aws:
    environment:
      - DEFAULT_DATABASE_HOST=172.17.0.1
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base worker -l INFO -Q clearsight-aws --concurrency 1 -n clearsight-default
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}

  watchtower:
    image: containrrr/watchtower:1.3.0
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ~/.docker/config.json:/config.json
    restart: always
    command: --interval 60

Problem
I have tried the CI pipeline and it works properly with shared runners:

passed pipeline

So, I think there isn’t any problem in CI side. Therefore, I registered a new gitlab-runner (docker+machine) in the server and it can be seen in the repository CI/CD runners, but as it can be seen in the below picture, it is not connected!

runner not connected

Question How can I resolve the runner issue and make the runner connect to the jobs?

4

Answers


  1. Chosen as BEST ANSWER

    The problem was solved after running the gitlab-runner verify command.

    ~ ➤ gitlab-runner -h # shows the help and other commands of gitlab-runner
    NAME:
       gitlab-runner - a GitLab Runner
    
    USAGE:
       gitlab-runner [global options] command [command options] [arguments...]
    
    VERSION:
       13.11.0 (7f7a4bb0)
    
    AUTHOR:
       GitLab Inc. <[email protected]>
    
    COMMANDS:
         exec                  execute a build locally
         list                  List all configured runners
         run                   run multi runner service
         register              register a new runner
         install               install service
         uninstall             uninstall service
         start                 start service
         stop                  stop service
         restart               restart service
         status                get status of a service
         run-single            start single runner
         unregister            unregister specific runner
         verify                verify all registered runners
         artifacts-downloader  download and extract build artifacts (internal)
         artifacts-uploader    create and upload build artifacts (internal)
         cache-archiver        create and upload cache artifacts (internal)
         cache-extractor       download and extract cache artifacts (internal)
         cache-init            changed permissions for cache paths (internal)
         health-check          check health for a specific address
         read-logs             reads job logs from a file, used by kubernetes executor (internal)
         help, h               Shows a list of commands or help for one command
    
    GLOBAL OPTIONS:
       --cpuprofile value           write cpu profile to file [$CPU_PROFILE]
       --debug                      debug mode [$DEBUG]
       --log-format value           Choose log format (options: runner, text, json) [$LOG_FORMAT]
       --log-level value, -l value  Log level (options: debug, info, warn, error, fatal, panic) [$LOG_LEVEL]
       --help, -h                   show help
       --version, -v                print the version
    

    Now it works fine:

    gitlab-runner registered

    Thanks @WytrzymałyWiktor for his comment on this post. I didn't find anything helpful other than his comment.

    P.S After performing the above steps, you may need to run gitlab-runner start in order to resolve your problem!


  2. For those that are using GitLab Runner under Docker, use the equivalent command to verify your new instance as provided by @MostafaGhadimi above:

    docker exec <container name> /bin/sh -c "gitlab-runner verify"
    
    Login or Signup to reply.
  3. For me sudo gitlab-runner verify worked. Second command to be executed is sudo gitlab-runner run. For custom runner, it is important to manually execute the runner.

    Login or Signup to reply.
  4. Running

    gitlab-runner verify

    makes it connected again.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search