skip to Main Content

Modifying an app based on this example, I am having a problem when deploying my updated Heroku app during initialization of the web.1 dyno.

Everything is working fine in my development envr; when I start the ‘app’ and ‘worker’ executables directly, I can access the working app from 127.0.0.1:5000

I already have the postgresql and redistogo addons created as part of the app…

heroku config file

=== app Config Vars
APP_SETTINGS:  config.StagingConfig
DATABASE_URL:  postgres://[points-to-AWS-EC2].compute-1.amazonaws.com:[port]/[identifier]
REDISTOGO_URL: redis://redistogo:[identifier]@pike.redistogo.com:[port]/

Procfile

web: bash heroku.sh

heroku.sh

#!/bin/bash
gunicorn app:app --daemon
python worker.py

app.py

from collections import Counter
from datetime import datetime
import json
import re
import operator
import os

from flask import Flask, render_template, request, jsonify
from flask_sqlalchemy import SQLAlchemy
from requests import get, exceptions as rx
import nltk
from rq import Queue
from rq.job import Job
from validators import url as vurl

from worker import conn

app = Flask(__name__, template_folder='app-templates/')
app.config.from_object(os.environ['APP_SETTINGS'])

db = SQLAlchemy(app)    
q = Queue(connection=conn)


def worker_task(url):
    '''
    takes website url and ...

worker.py

import os
import redis
from rq import Worker, Queue, Connection

listen = ['default']
redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
conn = redis.from_url(redis_url)


if __name__ == '__main__':
    # for tracking worker job in a terminal
    # assumptions: redis-server is running
    try:
        with Connection(conn):
            worker = Worker(list(map(Queue, listen)))
--->>>      worker.work()         <<<---
    except Exception as e:
        print(f"{str(e)}")

During the dyno startup, I get the following errors:

heroku[web.1]: State changed from crashed to starting

heroku[web.1]: Starting process with command bash heroku.sh

app[web.1]: Command # 2 (HSET rq:worker:2bd1d3d3a7f04d2fb7a09b34786689bf
birth 2020-05-23T11:28:04.485474Z
last_heartbeat 2020-05-23T11:28:04.485474Z
queues default pid 11
hostname b428750b-5d7f-4837-b9c2-b2195e234978
version 1.4.1 python_version 3.7.4 (default, Sep 12 2019, 01:19:52)

app[web.1]: [GCC 7.4.0]) of pipeline caused error: wrong number of arguments for ‘hset’ command

heroku[web.1]: Process exited with status 0

heroku[web.1]: State changed from starting to crashed

Im fairly confident that the error is in the highlighted worker.work() line from the worker.py file, because I get the same error when I run heroku run python worker.py (or directly copying those lines into the heroku run python REPL interpreter)

2

Answers


  1. Chosen as BEST ANSWER

    SOLVED was an issue with my package versions...

    the original example I was using had the following package version dependencies:

    redis = "==3.4.1"
    rq = "==1.2.2"
    

    but I was just using the most recently updated packages = "*" for both, and when I originally deployed the package in Heroku, it installed RQ version "1.4.0", but then (apparently) when my new app was deployed, the Heroku installer used the latest 1.4.1 package, which broke during the worker.work() method...

    Worker RQ v 1.4.0 output:

    {
      'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=10038,db=0>>>, 
    'hostname': 'e868b6d3-70b4-4e17-98c2-b96aae04d9a8', 
    'pid': 39, 
    'job_class': <class 'rq.job.Job'>, 
    'queue_class': <class 'rq.queue.Queue'>, 'version': '1.4.0',
    **'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) n[GCC 7.4.0]',** 
    **'serializer': <module 'pickle' from '/app/.heroku/python/lib/python3.7/pickle.py'>,**
    'name': '05c7af34c21f44f0b4374b40525779af', 
    'queues': [Queue('default')], 
    '_exc_handlers': [], ... ... 
    }
    

    Worker RQ v1.4.1 output:

    {
    'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=11022,db=0>>>,
    'hostname': '19a476fa-b4e6-4e63-b40e-ac8779ae0f9e', 
    'pid': 39, 
    'job_class': <class 'rq.job.Job'>, 
    'queue_class': <class 'rq.queue.Queue'>, 
    'version': '1.4.1', 
    **'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) n[GCC 7.4.0]',**
    **'serializer': <class 'rq.serializers.DefaultSerializer'>,**
    'name': 'b63389b3499f4d73be889a33f3777b46', 
    'queues': [Queue('default')], 
    '_exc_handlers': [], ... ... 
    }
    

    You can see a difference in the 'serializer' key between the two, possibly causing the HSET exception that was being throw...

    Reverting to RQ v1.2.2 solved the issue for me; I probably could have used the same v1.4.0...


  2. I had a similar problem, and I fixed it by downgrading my version of rq from 1.4.1 to 1.3.0.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search