Memcached – Flask-SocketIO + uWSGI. Emit with Mule worker

Recently I’m implementing WebSocket support in my Flask app.

Setup looks as follow:
1. Python 2.7
2. Flask + Flask-SocketIo
3. SocketIO (Client side)
4. NGINX as proxy
5. uWSGI 2.0.13 + it’s WebSocket with Gevent (Monkey-patched)

I’ve manage to enable socket connection but I’m struggling on synchronization of emits between UWSGI Workers, single mule especially.

I have dedicated UWSGI mule to monit some stuff and reports the state every 30 secs, broadcasting it to connected clients. If I allow my task to run on any available worker (not only mule dedicated) emit reaches only to client that was previously server by exact same process e.g:
Client request was served by Worker #6 with (400 PID) and established Socket connection.
Later one mule task was served by very the same Worker so client received the Emit) If any other worker processed that task, emit is not cached by Client.

With single worker mode everything works fine, but that is obviously not acceptable solution.

Here is some tech details:
uwsgi.ini

[uwsgi]
#application's base folder
base = /home/ubuntu/application_test

#python module to import
app = manage
module = %(app)

home = %(base)/venv
virtualenv = %(base)/venv
pythonpath = %(base)

#socket file's location
socket = %(base)/application_test.sock

#permissions for the socket file
chmod-socket = 666

#the variable that holds a flask application inside the module imported at 
line #6
callable = app

#location of log files
logto = /var/log/uwsgi/%n.log

processes = 10

#WebSocket
http-websockets = true

gevent = 1000
enable-threads = true
die-on-therm = true

vacuum = true

mule=%(base)/application_test/uwsgi_mules/metrics_mule.py

Flask App

(...)
from flask_socketio import SocketIO
app = Flask(__name__)
socketio = SocketIO(app,message_queue='redis://')
(...)

Worker Emit Code

socket = SocketIO(message_queue='redis://')

def broad_cast_server_info(state, health):
    socket.emit('sys_state', health)
    socket.emit('system_snapshot', state)

Client Side

        socket.on('sys_state', function(data){
            (...)
            }
        });

If I use client side socket.emit with acknowledgment response, everything works fine.

When In mule I simply import socketio and call emit on it I face problem described above. However If I follow:
https://flask-socketio.readthedocs.io/en/latest/#emitting-from-an-external-process

I’m getting an redis error every time mule runs its job.

File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/flask_socketio/init.py”, line 365, in emit
skip_sid=skip_sid, callback=callback, **kwargs)
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/socketio/server.py”, line 228, in emit
**kwargs)
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/socketio/pubsub_manager.py”, line 66, in emit
‘skip_sid’: skip_sid, ‘callback’: callback})
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/socketio/redis_manager.py”, line 62, in _publish
return self.redis.publish(self.channel, pickle.dumps(data))
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/redis/client.py”, line 2034, in publish
return self.execute_command(‘PUBLISH’, channel, message)
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/redis/client.py”, line 673, in execute_command
connection.send_command(*args)
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/redis/connection.py”, line 610, in send_command
self.send_packed_command(self.pack_command(*args))
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/redis/connection.py”, line 585, in send_packed_command
self.connect()
File “/home/ubuntu/application_test/venv/local/lib/python2.7/site-packages/redis/connection.py”, line 489, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error -2 connecting to redispass:6379. Name or service not known.

PIP FREEZE:

alabaster==0.7.9
amqp==2.1.1
aniso8601==1.2.0
appdirs==1.4.3
astroid==1.4.8
Babel==2.3.4
beautifulsoup4==4.5.1
certifi==2017.11.5
chardet==3.0.4
click==6.7
decorator==4.0.11
defusedxml==0.5.0
dict2xml==1.5
dicttoxml==1.7.4
docopt==0.6.2
enum-compat==0.0.2
enum34==1.1.6
eventlet==0.21.0
Flask==0.12
Flask-Login==0.4.0
Flask-RESTful==0.3.5
flask-restful-swagger-2==0.33
Flask-Script==2.0.5
Flask-SocketIO==2.9.2
flask-swagger-ui==0.0.3
Flask-WTF==0.14.2
gevent==1.2.2
greenlet==0.4.12
html5lib==1.0b8
hurry.filesize==0.9
hypchat==0.21
idna==2.6
infinity==1.4
intervals==0.8.0
itsdangerous==0.24
Jinja2==2.9.5
jira==1.0.10
lazy-object-proxy==1.2.2
lxml==3.8.0
MarkupSafe==1.0
monotonic==1.4
oauthlib==2.0.2
olefile==0.44
ordereddict==1.1
packaging==16.8
pbr==3.0.1
pdfkit==0.6.1
Pillow==4.0.0
pql==0.4.3
psutil==5.4.1
py==1.4.34
pymongo==3.4.0
pyodbc==4.0.15
pyparsing==2.2.0
PyPDF2==1.26.0
pypyodbc==1.3.4
python-dateutil==2.6.0
python-engineio==2.0.1
python-memcached==1.58
python-socketio==1.8.3
pytz==2016.10
reportlab==3.4.0
requests==2.13.0
requests-oauthlib==0.8.0
requests-toolbelt==0.8.0
six==1.10.0
SQLAlchemy==1.1.6
SQLAlchemy-Utils==0.32.14
suds==0.4
urllib3==1.22
validators==0.11.3
vine==1.1.3
webencodings==0.5.1
Werkzeug==0.12
wrapt==1.10.10
WTForms==2.1
xmltodict==0.10.2
WTForms-Components==0.10.3

I’ve red about following cases:
– monkey-patch Gevent
– early-monkey-patch Gevent via uwsgi (suppose to work on uWSGI 2.1)
– add Redis queue

Is there any other method to make Flask-SocketIO works in sync with uWSGI workers?