2014-02-10T03:46:26Z

Easy WebSockets with Flask and Gevent

This weekend I decided to take a short vacation from my book writing effort and spend time on a project I wanted to work on for a long time. The result of this effort is a brand new Flask extension that I think is pretty cool.

I'm happy to introduce Flask-SocketIO, a very easy to use extension that enables WebSocket communications in Flask applications.

What is WebSocket?

WebSocket is a new communication protocol introduced with HTML5, mainly to be implemented by web clients and servers, though it can also be implemented outside of the web.

Unlike HTTP connections, a WebSocket connection is a permanent, bi-directional communication channel between a client and the server, where either one can initiate an exchange. Once established, the connection remains available until one of the parties disconnects from it.

WebSocket connections are useful for games or web sites that need to display live information with very low latency. Before this protocol existed there were other much less efficient approaches to achieve the same result such as Comet.

The following web browsers support the WebSocket protocol:

  • Chrome 14
  • Safari 6
  • Firefox 6
  • Internet Explorer 10

What is SocketIO?

SocketIO is a cross-browser Javascript library that abstracts the client application from the actual transport protocol. For modern browsers the WebSocket protocol is used, but for older browsers that don't have WebSocket SocketIO emulates the connection using one of the older solutions, the best one available for each given client.

The important fact is that in all cases the application uses the same interface, the different transport mechanisms are abstracted behind a common API, so using SocketIO you can be pretty much sure that any browser out there will be able to connect to your application, and that for every browser the most efficient method available will be used.

What about Flask-Sockets?

A while ago Kenneth Reitz published Flask-Sockets, another extension for Flask that makes the use of WebSocket accessible to Flask applications.

The main difference between Flask-Sockets and Flask-SocketIO is that the former wraps the native WebSocket protocol (through the use of the gevent-websocket project), so it can only be used by the most modern browsers that have native support. Flask-SocketIO transparently downgrades itself for older browsers.

Another difference is that Flask-SocketIO implements the message passing protocol exposed by the SocketIO Javascript library. Flask-Sockets just implements the communication channel, what is sent on it is entirely up to the application.

Flask-SocketIO also creates an environment for event handlers that is close to that of regular view functions, including the creation of application and request contexts. There are some important exceptions to this explained in the documentation, however.

A Flask-SocketIO Server

Installation of Flask-SocketIO is very simple:

$ pip install flask-socketio

Below is an example Flask application that implements Flask-SocketIO:

from flask import Flask, render_template
from flask_socketio import SocketIO, emit

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)

@app.route('/')
def index():
    return render_template('index.html')

@socketio.on('my event')
def test_message(message):
    emit('my response', {'data': message['data']})

@socketio.on('my broadcast event')
def test_message(message):
    emit('my response', {'data': message['data']}, broadcast=True)

@socketio.on('connect')
def test_connect():
    emit('my response', {'data': 'Connected'})

@socketio.on('disconnect')
def test_disconnect():
    print('Client disconnected')

if __name__ == '__main__':
    socketio.run(app)

The extension is initialized in the usual way, but to simplify the start up of the server a custom run() method is used instead of flask run or app.run(). This method starts the eventlet or gevent servers if they are installed. Using gunicorn with the eventlet or gevent workers should also work. The run() method takes optional host and port arguments, but by default it will listen on localhost:5000 like Flask's development web server.

The only traditional route in this application is /, which serves index.html, a web document that contains the client implementation of this example.

To receive WebSocket messages from the client the application defines event handlers using the socketio.on decorator.

The first argument to the decorator is the event name. Event names 'connect', 'disconnect', 'message' and 'json' are special events generated by SocketIO. Any other event names are considered custom events.

The 'connect' and 'disconnect' events are self-explanatory. The 'message' event delivers a payload of type string, and the 'json' and custom events deliver a JSON payload, in the form of a Python dictionary.

To send events a Flask server can use the send() and emit() functions provided by Flask-SocketIO. The send() function sends a standard message of string or JSON type to the client. The emit() function sends a message under a custom application-defined event name.

Messages are sent to the connected client by default, but when including the broadcast=True optional argument all clients connected to the namespace receive the message.

A SocketIO Client

Ready to try your hand at some Javascript? The index.html page used by the example server contains a little client application that uses jQuery and SocketIO. The relevant code is shown below:

$(document).ready(function(){
    var socket = io();
    socket.on('my response', function(msg) {
        $('#log').append('<p>Received: ' + msg.data + '</p>');
    });
    $('form#emit').submit(function(event) {
        socket.emit('my event', {data: $('#emit_data').val()});
        return false;
    });
    $('form#broadcast').submit(function(event) {
        socket.emit('my broadcast event', {data: $('#broadcast_data').val()});
        return false;
    });
});

The socket variable is initialized with a SocketIO connection to the server. If the Socket.IO server is hosted at a different URL than the HTTP server, then you can pass a connection URL as an argument to io().

The socket.on() syntax is used in the client side to define an event handler. In this example a custom event with name 'my response' is handled by adding the data attribute of the message payload to the contents of a page element with id log. This element is defined in the HTML portion of the page.

The next two blocks override the behavior of two form submit buttons so that instead of submitting a form over HTTP they trigger the execution of a callback function.

For the form with id emit the submit handler emits a message to the server with name 'my event' that includes a JSON payload with a data attribute set to the value of the text field in that form.

The second form, with id broadcast does the same thing, but sends the data under a different event name called 'my broadcast event'.

If you now go back to the server code you can review the handlers for these two custom events. For 'my event' the server just echoes the payload back to the client in a message sent under event name 'my response', which is handled by showing the payload in the page. The event named 'my broadcast event' does something similar, but instead of echoing back to the client alone it broadcasts the message to all connected clients, also under the 'my response' event.

You can view the complete HTML document in the GitHub repository.

Running the Example

To run this example you first need to download the code from GitHub. For this you have two options:

The example application is in the example directory, so cd to it to begin.

To keep your global Python interpreter clean it is a good idea to make a virtual environment:

$ virtualenv venv
$ . venv/bin/activate

Then you need to install the dependencies:

(venv) $ pip install -r requirements.txt

Finally you can run the application:

(venv) $ python app.py

Now open your web browser and navigate to http://localhost:5000 and you will get a page with two forms as shown in the following screenshot:

Any text you submit from any of the two text fields will be sent to the server over the SocketIO connection, and the server will echo it back to the client, which will append the message to the "Receive" part of the page, where you can already see the message sent by the 'connect' event handler from the server.

Things get much more interesting if you connect a second browser to the application. In my case I'm testing this with Firefox and Chrome, but any two browsers that you run on your machine will do. If you prefer to access the server from multiple machines you can do that too, but you first need to change the start up command to socketio.run(app, host='0.0.0.0') so that the server listens on the public network interface.

With two or more clients when you submit a text from the form on the left only the client that submitted the message gets the echoed response. If you submit from the form on the right the server broadcasts the message to all connected clients, so all get the reply.

If a client disconnects (for example if you close the browser window) the server will detect it a few seconds later and send a disconnect event to the application. The console will print a message to that effect.

Final Words

For a more complete description of this extension please read the documentation. If you want to make improvements to it feel free to fork it and then submit a pull request.

I hope you make cool applications with this extension. I can tell you that I had a lot of fun implementing this extension.

If you make something with it feel free to post links in the comments below.

Miguel

495 comments

  • #351 Rashmi Ashok said 2017-03-15T17:44:49Z

    I am trying to use socketio in google app scripts to send some data from google spread sheet to python code on my local machine. Can you tell me if socketio is supported on google app scripts?

  • #352 Miguel Grinberg said 2017-03-15T19:02:42Z

    @Rashmi: I'm not sure. I assume you are going to use the Socket.IO server presented here on your Python machine, and intend to use a JavaScript socket.io client connecting to it? You need to find out of GAS supports the Socket.IO client.

  • #353 gurkan said 2017-03-16T21:50:01Z

    hi Miguel , so i'am playing with flask socketio , i would like to know , if the current version of flask socketio supports , binary stream , as blob ? i need blob to transfert jpeg image.

    thank you

  • #354 Miguel Grinberg said 2017-03-16T22:02:44Z

    @gurkan: Yes, current versions of Flask-SocketIO support binary payloads. If you use Python 3 then it just works, but if you use Python 2, binary support needs to be enabled, since there is no easy way to separate text vs binary on that version.

  • #355 Aswin said 2017-04-05T21:03:22Z

    Hi Miguel, I'm using docker and gunicorn. I have this line in my dockerfile - CMD gunicorn -b 0.0.0.0:9000

    Can you please help me on how to get the gevent worker running?

  • #356 Miguel Grinberg said 2017-04-05T21:10:36Z

    @Aswin: add gevent and gevent-websocket to your requirements file, then add -k gevent to your gunicorn command.

  • #357 Aswin said 2017-04-05T21:40:31Z

    Hi Miguel, Thanks for your response. I tried as you suggested but its not working.

    I'm using this this example app - https://github.com/miguelgrinberg/Flask-SocketIO-Chat within my application. This is what's happening in my browser - https://drive.google.com/file/d/0B9NWUoqSQfxqMDJITWpmdkN3Z1k/view?usp=sharing

    I'm a beginner and I appreciate your help

  • #358 Miguel Grinberg said 2017-04-06T02:42:05Z

    @Aswin: Make sure you start a single gunicorn worker, you can't use more than one with Socket.IO, as explained in the documentation. If that doesn't help, then I guess you need to look for more clues in the errors that correspond to those 400s. Look in the server side log for stack traces.

  • #359 kimi said 2017-04-20T15:40:32Z

    Hi Miguel, Thanks for your great job. But how can I deployment a app when using redis as a message queue? I am using gunicorn.

  • #360 Miguel Grinberg said 2017-04-20T16:43:25Z

    @kimi: Have you seen my "Flask At Scale" class on youtube? I cover an application that uses Flask-SocketIO, Celery and Redis. Here is the link: https://www.youtube.com/watch?v=tdIIJuPh3SI

  • #361 Robert Fletcher said 2017-05-23T22:16:00Z

    In order to make this work as expected it needs to be updated a little. firstly you need to install eventlet

    pip install eventlet

    next go to the top of the application.py file and add the following

    import eventlet eventlet.monkey_patch()

    next change the line 'from flask.ext.socketio import SocketIO' to: from flask_socketio import SocketIO, emit

    once that is done open the index.html file from the templates folder and change the source of the socketio javascript file to a up to date link. (google socketio CDN)

    once I did this the app works perfectly.

    Thanks for posting this Miguel!

  • #362 Jordi said 2017-07-06T19:17:58Z

    Hello, Thank you for this excellent tutorial that I recently found. It's a bit old, but I allowed myself to post my question because I did not find a satisfactory answer on the web. I work on a home automation project with raspberry pi3 and flask. I would like to be able to send real-time info that will be displayed on a web page (html + javascript). For this I use the extension flask-socketio but it does not seem to work. Here is an excerpt of my code:

    from flask import Flask, render_template, send_from_directory, jsonify, request from flask_socketio import SocketIO, emit, send import time import RPi.GPIO as GPIO GPIO.setmode(GPIO.BCM) PIN=4 GPIO.setup(PIN, GPIO.IN)

    app = Flask(name, static_url_path='/static') app.config['SECRET_KEY'] = 'secret' socketio = SocketIO(app)

    @app.route('/') def static_file(path): return app.send_static_file(path)

    @app.route('/') def index(): return render_template ('index.html')

    @app.route('/_startMotion') def startMotion(): try: motion = False while True: if GPIO.input(PIN): if motion == False: test_message() motion = True else: motion = False time.sleep(.3) except: pass return 'ok'

    @socketio.on('motion') def test_message(): send('motion detected')

    if name == 'main': socketio.run(app, host='0.0.0.0', port=5000, debug=True) And on the client side here is my javascript code:

    $(document).ready(function(){ var socket = io.connect(); socket.on('motion', function(msg){ $('#message').text(msg.data); $('#message').fadeIn().delay(3000).fadeOut(); }); });

    I would like to be able to post a message on my web page, whenever a movement is detected. Then an idea of ​​what does not work in my code?

  • #363 Miguel Grinberg said 2017-07-07T22:30:22Z

    @Jordi: Are you using eventlet or gevent? If you are, you can't use time.sleep, or any other calls that are blocking.

  • #364 Jordi said 2017-07-09T19:11:40Z

    Hello, I actually use gevent. But I would like to understand, at what level the code blocks in its execution? Because if I put a print("something") just before "emit", when I call the "test_message" function, the print shows "something" in my console, but the "emit" does not send anything to the client. Is the code blocked before or after the execution of "emit"? Also, I retrieved the file socket.io.js in the official repository on github, and I copied it into my static directory to reference it in my html code. Is it sufficient? Thank you

  • #365 Miguel Grinberg said 2017-07-09T21:42:57Z

    @Jordi: the emit function does not really emit to the client, it just queues the event to go out. If your function does not release the CPU, then other background tasks from Socket.IO will not be able to get the event out. The time.sleep() function blocks, so no other tasks can run during the sleep period. The gevent.sleep() function is non-blocking, meaning that other tasks do get to run during the sleep.

  • #366 Jordi said 2017-07-10T15:13:38Z

    Thank you for this simple and clear explanation. But I still encounter difficulties on my project and I really need your help. Overall, I can see in my console that the connection is established between my client and my server ((1917) accepted ('192.168.43.98', 54509) 192.168.43.98 - [10 / Jul / 2017 14:57:49] "POST /socket.io/?EIO=3&transport=polling&t=1499698663923 ...). In my javascript console, no error. So in principle everything works correctly. But the client still can not receive messages from the server. So, can you give me a simple example where the server is the author of the message and sending it to the client? I tried all sorts of configuration but none seems to work in my case. Thank you in advance. P.S: I switched from gevent to eventlet

  • #367 Miguel Grinberg said 2017-07-11T21:43:15Z

    @Jordi: the example in the Flask-SocketIO repository starts a background thread that sends messages to clients.

  • #368 ivan said 2017-07-22T07:46:15Z

    Hi Miguel, I just got a weird problem here, When I installing Flask-socketIO, I got some invalid syntax on python file. Below is one of the syntax error.

    FIle "/usr/local/lib/python2.7/dist-packages/socketio/asyncio_pubsub_manager.py", line 39 async def emit(self, event, data, namespace=None, room=None, skip_sid=None,

    And, there are 5 errors. All of them sintax error on def.

    I cannot find the same problem as mine. Hope you can Answer this.

  • #369 Miguel Grinberg said 2017-07-22T17:37:57Z

    The asyncio modules in this package are compatible with Python 3.5 and up. You are using 2.7, so you should use the WSGI options, not asyncio.

  • #370 Ed Wachtel said 2017-07-23T23:14:31Z

    I have a socketio connection per tab on my browser. When I receive a message is there a simple way to emit to only that tab?

    @socketio.on('CheckAlignComplete', namespace='/test') def complete_message(job_id): (emitMsg,ret) = alignJobs.getState(job_id) socketio.emit(emitMsg, job_id, namespace='/test') #TODO emit to only the tab that sent this message

  • #371 Miguel Grinberg said 2017-07-24T00:14:01Z

    @Ed: you want to use "emit()" instead of "socketio.emit()". Import it from flask_socketio.

  • #372 tsanglemon said 2017-08-11T03:32:15Z

    Excuse me? How can I use socketio in aspscheduler ? I set a timed task with aspcheduler. And in this task i excute"socketio.emit(...)",but it dosen't work. I have no idea about that. Is there any other timed way to use socketio? Thanks a lot!

    scheduler.add_job(sockettest, 'interval', seconds = 10)

    def sockettest(): socketio.emit('my_response', {'data': '!!!!!', 'count': 0}, namespace='/test')

  • #373 Miguel Grinberg said 2017-08-11T05:14:58Z

    @tsanglemon: Are you using eventlet or gevent? Maybe appscheduler is incompatible with them. If you didn't monkey patch the std library you should try that. If you want another way to send stuff at regular intervals, see the example in the Flask-SocketIO repository. I do it with a background thread: https://github.com/miguelgrinberg/Flask-SocketIO/blob/master/example/app.py#L17-L25

  • #374 tsanglemon said 2017-08-11T08:20:41Z

    Thanks for your response! I have used eventlet.I guess it's because aspscheduler is synchronous and socketio is async. I have tried your background thread to set up a corn job. That is great ,except that I have to calculate the number of the seconds.Can I use the Celery?I just learn about that it can support eventlet or gevent

  • #375 Miguel Grinberg said 2017-08-11T23:21:15Z

    @tsanglemon: yes, Celery should work. In my opinion, though, that is probably overkill. If all you need is to run something at regular intervals you can do that without any external packages. See the accepted answer on this Stack Overflow question, for example: https://stackoverflow.com/questions/3393612/run-certain-code-every-n-seconds

Leave a Comment