Can't emit without request context

[tbsuht]

Hi,

I want to emit from a callback function which is invoked by another python module.

@app.get("/create-event")
def create_event():
    socketio.emit("YANG", {"path": "def", "value": "abc"}, to="ND")
    return "Event created"


@socketio.event
def connect():
    join_room("ND")
    emit("WELCOME!")

This example is working fine.

def change_clb(event, req_id, changes, private_data):
    try:
        for change in changes:
            socketio.emit("YANG", {"path": "test", "value": "test"}, to="ND")
            app.logger.info("sending change to ND: " + change.xpath)
    except Exception as e:
        app.logger.error(e)

change_clb is a function which I pass to another python module. I can see the log messages, but not the messages in the websocket.

Any idea what I'm doing wrong or what needs to be considered when emitting from outside a request context?

[tbsuht]

I tried the same as in this example with a background thread and using the same emit as above:
https://github.com/miguelgrinberg/Flask-SocketIO/blob/main/example/app.py#L19
This is working.

Could the invocation of the callback function have something to do with it?

[miguelgrinberg]

You need to pass the namespace explicitly, because when you don't pass it the emit tries to get it from the context. Basically, add namespace='/' to your emit. If this does not help, then please copy the stack trace of the error you are receiving, that should give me more information about the issue.

[miguelgrinberg]

I'm sorry but I don't understand what does the context to do with any of this. Sounds like you just assumed this is related to contexts, without knowing for sure?

I suggest that you create a small self-contained application that demonstrates the problem and send it to me. I'll review it and let you know if there is anything wrong.

Edit: if you are using eventlet or gevent, one possible issue here is that your callback is being invoked in another physical thread to the one used by eventlet/gevent.

[Tobulus]

Sorry for the confusion... Your edit is probably right.

Is there a "standard" way to coordinate eventlet and such threads?

I'm using nginx, gunicorn and eventlet currently.

[miguelgrinberg]

The standard way is to use greenlets, not threads. This is sometimes difficult or impossible if the threads are created by packages you don't have control of. Both eventlet and gevent offer a monkey patching feature that modifies the threading classes to use greenlets, but gunicorn enables that by default so you are probably using that already. So your other option would be to drop greenlets and use standard threads for Socket.IO. This is the "threading" async_mode, which should work just fine with callbacks running on other threads.

[tbsuht]

Thanks a lot for your feedback and support, now its working!

Actually I dropped eventlet for the threading mode, as I don't need it and it creates more problems than benefits.

Two small questions remain:

1. I'm starting gunicorn with the --threads param as shown in the docs. If I set the number of threads to e.g. 5 and try to open 6 connections, one of them will not be served, is what I can observe. My understanding is that this is due to the blocking way of handling the connections in threading mode. Am I correct? If I set the number of threads to 1000 are they spawned on demand? I'm running on a system with restricted ressources, so I don't want to set this to a too high number.

2. Is their a technical reason why socketio.emit doesn't support the broadcast=True parameter? I'm currently setting up a room and broadcast there, but in my case the room is redundant.

[miguelgrinberg]

The threads are created on demand by gunicorn, depending on load. If you set a maximum that is too high the process may run out of RAM and die, so you'd want to pick a number that can be tolerated in your system.

The socketio.emit function broadcasts to everyone by default. There is no need to use broadcast=True.