2014-02-10T03:46:26Z

Easy WebSockets with Flask and Gevent

This weekend I decided to take a short vacation from my book writing effort and spend time on a project I wanted to work on for a long time. The result of this effort is a brand new Flask extension that I think is pretty cool.

I'm happy to introduce Flask-SocketIO, a very easy to use extension that enables WebSocket communications in Flask applications.

What is WebSocket?

WebSocket is a new communication protocol introduced with HTML5, mainly to be implemented by web clients and servers, though it can also be implemented outside of the web.

Unlike HTTP connections, a WebSocket connection is a permanent, bi-directional communication channel between a client and the server, where either one can initiate an exchange. Once established, the connection remains available until one of the parties disconnects from it.

WebSocket connections are useful for games or web sites that need to display live information with very low latency. Before this protocol existed there were other much less efficient approaches to achieve the same result such as Comet.

The following web browsers support the WebSocket protocol:

  • Chrome 14
  • Safari 6
  • Firefox 6
  • Internet Explorer 10

What is SocketIO?

SocketIO is a cross-browser Javascript library that abstracts the client application from the actual transport protocol. For modern browsers the WebSocket protocol is used, but for older browsers that don't have WebSocket SocketIO emulates the connection using one of the older solutions, the best one available for each given client.

The important fact is that in all cases the application uses the same interface, the different transport mechanisms are abstracted behind a common API, so using SocketIO you can be pretty much sure that any browser out there will be able to connect to your application, and that for every browser the most efficient method available will be used.

What about Flask-Sockets?

A while ago Kenneth Reitz published Flask-Sockets, another extension for Flask that makes the use of WebSocket accessible to Flask applications.

The main difference between Flask-Sockets and Flask-SocketIO is that the former wraps the native WebSocket protocol (through the use of the gevent-websocket project), so it can only be used by the most modern browsers that have native support. Flask-SocketIO transparently downgrades itself for older browsers.

Another difference is that Flask-SocketIO implements the message passing protocol exposed by the SocketIO Javascript library. Flask-Sockets just implements the communication channel, what is sent on it is entirely up to the application.

Flask-SocketIO also creates an environment for event handlers that is close to that of regular view functions, including the creation of application and request contexts. There are some important exceptions to this explained in the documentation, however.

A Flask-SocketIO Server

Installation of Flask-SocketIO is very simple:

$ pip install flask-socketio

Below is an example Flask application that implements Flask-SocketIO:

from flask import Flask, render_template
from flask_socketio import SocketIO, emit

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)

@app.route('/')
def index():
    return render_template('index.html')

@socketio.on('my event')
def test_message(message):
    emit('my response', {'data': message['data']})

@socketio.on('my broadcast event')
def test_message(message):
    emit('my response', {'data': message['data']}, broadcast=True)

@socketio.on('connect')
def test_connect():
    emit('my response', {'data': 'Connected'})

@socketio.on('disconnect')
def test_disconnect():
    print('Client disconnected')

if __name__ == '__main__':
    socketio.run(app)

The extension is initialized in the usual way, but to simplify the start up of the server a custom run() method is used instead of flask run or app.run(). This method starts the eventlet or gevent servers if they are installed. Using gunicorn with the eventlet or gevent workers should also work. The run() method takes optional host and port arguments, but by default it will listen on localhost:5000 like Flask's development web server.

The only traditional route in this application is /, which serves index.html, a web document that contains the client implementation of this example.

To receive WebSocket messages from the client the application defines event handlers using the socketio.on decorator.

The first argument to the decorator is the event name. Event names 'connect', 'disconnect', 'message' and 'json' are special events generated by SocketIO. Any other event names are considered custom events.

The 'connect' and 'disconnect' events are self-explanatory. The 'message' event delivers a payload of type string, and the 'json' and custom events deliver a JSON payload, in the form of a Python dictionary.

To send events a Flask server can use the send() and emit() functions provided by Flask-SocketIO. The send() function sends a standard message of string or JSON type to the client. The emit() function sends a message under a custom application-defined event name.

Messages are sent to the connected client by default, but when including the broadcast=True optional argument all clients connected to the namespace receive the message.

A SocketIO Client

Ready to try your hand at some Javascript? The index.html page used by the example server contains a little client application that uses jQuery and SocketIO. The relevant code is shown below:

$(document).ready(function(){
    var socket = io();
    socket.on('my response', function(msg) {
        $('#log').append('<p>Received: ' + msg.data + '</p>');
    });
    $('form#emit').submit(function(event) {
        socket.emit('my event', {data: $('#emit_data').val()});
        return false;
    });
    $('form#broadcast').submit(function(event) {
        socket.emit('my broadcast event', {data: $('#broadcast_data').val()});
        return false;
    });
});

The socket variable is initialized with a SocketIO connection to the server. If the Socket.IO server is hosted at a different URL than the HTTP server, then you can pass a connection URL as an argument to io().

The socket.on() syntax is used in the client side to define an event handler. In this example a custom event with name 'my response' is handled by adding the data attribute of the message payload to the contents of a page element with id log. This element is defined in the HTML portion of the page.

The next two blocks override the behavior of two form submit buttons so that instead of submitting a form over HTTP they trigger the execution of a callback function.

For the form with id emit the submit handler emits a message to the server with name 'my event' that includes a JSON payload with a data attribute set to the value of the text field in that form.

The second form, with id broadcast does the same thing, but sends the data under a different event name called 'my broadcast event'.

If you now go back to the server code you can review the handlers for these two custom events. For 'my event' the server just echoes the payload back to the client in a message sent under event name 'my response', which is handled by showing the payload in the page. The event named 'my broadcast event' does something similar, but instead of echoing back to the client alone it broadcasts the message to all connected clients, also under the 'my response' event.

You can view the complete HTML document in the GitHub repository.

Running the Example

To run this example you first need to download the code from GitHub. For this you have two options:

The example application is in the example directory, so cd to it to begin.

To keep your global Python interpreter clean it is a good idea to make a virtual environment:

$ virtualenv venv
$ . venv/bin/activate

Then you need to install the dependencies:

(venv) $ pip install -r requirements.txt

Finally you can run the application:

(venv) $ python app.py

Now open your web browser and navigate to http://localhost:5000 and you will get a page with two forms as shown in the following screenshot:

Any text you submit from any of the two text fields will be sent to the server over the SocketIO connection, and the server will echo it back to the client, which will append the message to the "Receive" part of the page, where you can already see the message sent by the 'connect' event handler from the server.

Things get much more interesting if you connect a second browser to the application. In my case I'm testing this with Firefox and Chrome, but any two browsers that you run on your machine will do. If you prefer to access the server from multiple machines you can do that too, but you first need to change the start up command to socketio.run(app, host='0.0.0.0') so that the server listens on the public network interface.

With two or more clients when you submit a text from the form on the left only the client that submitted the message gets the echoed response. If you submit from the form on the right the server broadcasts the message to all connected clients, so all get the reply.

If a client disconnects (for example if you close the browser window) the server will detect it a few seconds later and send a disconnect event to the application. The console will print a message to that effect.

Final Words

For a more complete description of this extension please read the documentation. If you want to make improvements to it feel free to fork it and then submit a pull request.

I hope you make cool applications with this extension. I can tell you that I had a lot of fun implementing this extension.

If you make something with it feel free to post links in the comments below.

Miguel

495 comments

  • #426 Miguel Grinberg said 2019-10-23T08:08:32Z

    @Yasmin: I have shown a few example apps at conferences over the years. For example:

    Fun with WebSocket and Flask: https://www.youtube.com/watch?v=Jwux1TPZUwg Look Ma, No HTTP!: https://www.youtube.com/watch?v=C-Ltp1D_m3c&t=18081s

  • #427 Jython said 2019-11-17T21:13:41Z

    This is my setup but I get a 404 when using default path socket.io. When I change the path parameter to an anything custom, I get this error on the server side: https://pastebin.com/n96FfXBu

    And on client side (testing with browser, I get a connection reset).

    REQUIREMENTS.txt (Ignore some of them)

    billiard==3.6.0.0 celery==4.3.0 certifi==2019.6.16 cffi==1.12.3 chardet==3.0.4 Click==7.0 contextlib2==0.5.5 cryptography==2.6.1 Django==2.2.3 ecdsa==0.13.2 fabric==2.4.0 Flask==1.1.1 Flask-Migrate==2.5.2 Flask-SocketIO==4.2.1 Flask-SQLAlchemy==2.4.0 future==0.17.1 gevent==1.4.0 greenlet==0.4.15 gunicorn==19.9.0 httplib2==0.14.0 idna==2.8 invoke==1.2.0 itsdangerous==1.1.0 Jinja2==2.10.1 kombu==4.6.3 Mako==1.1.0 MarkupSafe==1.1.1 paramiko==2.6.0 phonenumbers==8.10.16 psycopg2==2.8.3 pyasn1==0.4.5 pycparser==2.19 PyJWT==1.7.1 PyNaCl==1.3.0 pyotp==2.3.0 python-dateutil==2.8.0 python-dotenv==0.10.3 python-editor==1.0.4 python-engineio==3.10.0 python-socketio==4.3.1 pytz==2019.1 PyYAML==5.1.1 recordtype==1.3 redis==3.3.11 requests==2.22.0 rsa==4.0 schema==0.7.0 six==1.12.0 SQLAlchemy==1.3.6 sqlparse==0.3.0 style==1.1.0 update==0.0.1 urllib3==1.25.3 vine==1.3.0 virtualenv==16.7.2 Werkzeug==0.15.5

  • #428 Miguel Grinberg said 2019-11-17T23:21:54Z

    @Jython: In the vast majority of cases you do not need to know about the path, the default works fine. That means that on the server you don't set a path, and on the client you don't specify the path either. Changing the path in either the server or the client is uncommon, you must have a good reason to do it.

  • #429 ashutosh said 2020-01-29T05:47:43Z

    hello thanks you for your all amazing socket.io program, i am having issue with the disconnect event in socket.io as in is it possible for a code to trigger disconnect event on server side when client dc's but not by traditional way as in by clicking a logout button but more random way like closing a tab or closing a browser

  • #430 Miguel Grinberg said 2020-01-29T10:53:13Z

    @ashutosh: the client is automatically disconnected if they close the tab or browser. No need to do anything.

  • #431 Drinian said 2020-02-29T00:08:55Z

    Hey Miguel,

    I am having an issue where when I use the socketio.run(app) it is breaking my jinja extension that I set in my create_app function --> app.jinja_options['extensions'].append('jinja2_time.TimeExtension'). When I check in my app it still shows that those extensions still are in the jinja_options, but they aren't recognized when I use them in my templates.

  • #432 Miguel Grinberg said 2020-02-29T17:48:25Z

    @Drinian: impossible for me to tell with such little information. Can you build a small self-contained app that shows this issue? Open an issue on the GitHub repo with it and I'll have a look.

  • #433 Sanya prosto CaHek said 2020-03-04T14:05:34Z

    Good afternoon. I don't understand how gunicorn works. I have windows, and my experiments with the test server knowingly failed. Because I run an app on HEROKU using a server GUNICORN, even a simple app that uses a socket doesn't work correctly for me.

    LInk --- https://flask-socket.herokuapp.com/ problem --- bad requests, websocket fail, and etc. code server --- any fragment as is yours example procfile --- web: gunicorn -k eventlet main_app:app (all other options do not work) python 3.8.2 help please thanks for atention

  • #434 Miguel Grinberg said 2020-03-04T19:41:13Z

    @Sanya: I can't really do much. You have access to your logs, so you can see what's failing.

  • #435 Andre Queiroz said 2020-03-06T11:33:11Z

    Hello. your work in sharing your code has helped me a lot. thanks. I added flask-mqtt to your code, so everything was unstable. Do you recommend using a websoket server?

  • #436 Miguel Grinberg said 2020-03-06T11:49:54Z

    @Andre: sorry, not sure what you mean with "unstable" and have no idea how your project is structured or what it does, so I can't really answer your question.

  • #437 Dheeraj Gupta said 2020-04-05T02:10:07Z

    Is there anyway we can identify that our socket is converted to websocket or not ? My application is sending the emit before its converted to webSocket which creating error on Frontend.

  • #438 Miguel Grinberg said 2020-04-06T22:25:35Z

    @Dheeraj: there shouldn't be any difference, the conversion from polling to websocket is transparent. But if you want to remove that step you can always ask your client to connect directly with websocket.

  • #439 utkarsh said 2020-05-12T16:35:33Z

    Hey Miguel, Socketio doesn't seem to work for 'https' protocol for my case. Quite Confused.

  • #440 Miguel Grinberg said 2020-05-12T19:33:22Z

    @utkarsh: encryption is a function of the web server that you use, not of flask-socketio, which has nothing to do with networking or web servers.

  • #441 Hector Rojas said 2020-05-18T12:30:01Z

    Miguel, I really like the way you integrated your package with Flask. I would like to use your package in a small project in where I need to implement a websocket reverse-proxy in python within the Flask framework. That means that I would also need the client running in python. Security is not a concern for this project. Using something like NGINX to do the proxy will be an overkill. Could you recommend what package(s) would do this similar to what you have done with Flask-socketio? Great work by the way!! Thanks, Hector

  • #442 Miguel Grinberg said 2020-05-18T17:28:52Z

    @Hector: the python-socketio package has a Socket.IO client that you can use to connect to the Socket.IO server.

  • #443 Zakaria Boualaid said 2020-06-06T22:41:56Z

    Hi, thank you fro this article. Is it possible to run socketio.emit from inside a background job function that is picked up by rq/rqscheduler? My first thought is that socketio is not known inside the background job function, but I tried to get it there and I wasn't successful. Any idea please?

  • #444 Doesn't Matter said 2020-06-07T07:40:33Z

    I am trying flask socketio. Sometimes it works sometimes it doesn't. I am copying the code from your tutorial it run without error but doesn't response anything including it sometimes tell user is connected sometimes not. Why?

  • #445 Miguel Grinberg said 2020-06-07T10:00:42Z

    @Zakaria: see the "Emitting from an External Process" section of the documentation: https://flask-socketio.readthedocs.io/en/latest/#emitting-from-an-external-process.

  • #446 Miguel Grinberg said 2020-06-07T10:02:25Z

    @Doesn't: can't really answer with the little information you provide. Please look for errors in the browser or the server, log messages, etc, and provide those.

  • #447 Blue Fang said 2020-07-10T06:27:16Z

    Hi, I have a question about this flask project. The startup method is "socketio.run(app)", but the normal startup method is "app.run()". Is it just passing video streams as a separate WebSocket project?That is, I should have another project. When I need to use the camera function, the web page should visit this WebSocket project to implement the function of my other project. Is that right?

  • #448 Miguel Grinberg said 2020-07-10T08:54:57Z

    @Blue: I'm not sure what do you mean by video streams. Are you talking about a specific project? In any case, socketio.run() replaces and expands app.run() to also handle the Socket.IO events. If you use socketio.run() you do not need to run app.run().

  • #449 Paul Chaffee said 2020-07-12T06:47:37Z

    Hi Miguel,

    Thanks for all your work. I'm having trouble accessing objects in a dict on the client side. In main.py I have a data structure like this: p1_sees = {'player1': ('p1_card1.jpg', 'p1_card2.jpg'), 'player2': ('p3_card1.jpg', 'p2_card2.jpg')} and my decorator: @socketio.on('my_room_event', namespace='/test') def send_room_message(): emit('my_response', {'data': p1_sees, 'count': session['receive_count']}, room='player1')

    On the client side I catch the data payload with socket.on('my_response', function(msg, cb) { $('#log').append('<br>' + $('<div/>').text('Received #' + msg.count + ': ' + msg.data["player1"][0]).html()); if (cb) cb(); }); but this doesn't work. The first item in p1_sees is a tuple. I just want to be able to access the tuple elements individually but can't figure out how to grab them on the client side. For example above, I assumed msg.data["player1"][0] would get the first tuple element for "player1" in the payload, but this code produces nothing. If I just leave the 0 index element off, the value shown in the brower is p1_card1.jpg, p1_card2.jpg.

    I don't understand this. Thanks much!

  • #450 Miguel Grinberg said 2020-07-12T22:31:01Z

    @Paul: why don't you log the entire data structure in the browser's console to see what data did you get?

Leave a Comment