2013-11-28T07:33:19Z

RESTful Authentication with Flask

This article is the fourth in my series on RESTful APIs. Today I will be showing you a simple, yet secure way to protect a Flask based API with password or token based authentication.

This article stands on its own, but if you feel you need to catch up here are the links to the previous articles:

Example Code

The code discussed in the following sections is available for you to try and hack. You can find it on GitHub: REST-auth. Note that the GitHub repository likely has code that is newer than what I'm going to show in this article. If you want to see the version of the code featured in this article use this link.

The User Database

To give this example some resemblance to a real life project I'm going to use a Flask-SQLAlchemy database to store users.

The user model will be very simple. For each user a username and a password_hash will be stored.

class User(db.Model):
    __tablename__ = 'users'
    id = db.Column(db.Integer, primary_key = True)
    username = db.Column(db.String(32), index = True)
    password_hash = db.Column(db.String(128))

For security reasons the original password will not be stored, after the hash is calculated during registration it will be discarded. If this user database were to fall in malicious hands it would be extremely hard for the attacker to decode the real passwords from the hashes.

Passwords should never be stored in the clear in a user database.

Password Hashing

To create the password hashes I'm going to use PassLib, a package dedicated to password hashing.

PassLib provides several hashing algorithms to choose from. The custom_app_context object is an easy to use option based on the sha256_crypt hashing algorithm.

To add password hashing and verification two new methods are added to the User model:

from passlib.apps import custom_app_context as pwd_context

class User(db.Model):
    # ...

    def hash_password(self, password):
        self.password_hash = pwd_context.encrypt(password)

    def verify_password(self, password):
        return pwd_context.verify(password, self.password_hash)

The hash_password() method takes a plain password as argument and stores a hash of it with the user. This method is called when a new user is registering with the server, or when the user changes the password.

The verify_password() method takes a plain password as argument and returns True if the password is correct or False if not. This method is called whenever the user provides credentials and they need to be validated.

You may ask how can the password be verified if the original password was thrown away and lost forever after it was hashed.

Hashing algorithms are one-way functions, meaning that they can be used to generate a hash from a password, but they cannot be used in the reverse direction. But these algorithms are deterministic, given the same inputs they will always generate the same output. All PassLib needs to do to verify a password is to hash it with the same function that was used during registration, and then compare the resulting hash against the one stored in the database.

User Registration

In this example, a client can register a new user with a POST request to /api/users. The body of the request needs to be a JSON object that has username and password fields.

The implementation of the Flask route is shown below:

@app.route('/api/users', methods = ['POST'])
def new_user():
    username = request.json.get('username')
    password = request.json.get('password')
    if username is None or password is None:
        abort(400) # missing arguments
    if User.query.filter_by(username = username).first() is not None:
        abort(400) # existing user
    user = User(username = username)
    user.hash_password(password)
    db.session.add(user)
    db.session.commit()
    return jsonify({ 'username': user.username }), 201, {'Location': url_for('get_user', id = user.id, _external = True)}

This function is extremely simple. The username and password arguments are obtained from the JSON input coming with the request and then validated.

If the arguments are valid then a new User instance is created. The username is assigned to it, and the password is hashed using the hash_password() method. The user is finally written to the database.

The body of the response shows the user representation as a JSON object, with a status code of 201 and a Location header pointing to the URI of the newly created user.

Note: the implementation of the get_user endpoint is now shown here, you can find it in the full example on github.

Here is an example user registration request sent from curl:

$ curl -i -X POST -H "Content-Type: application/json" -d '{"username":"miguel","password":"python"}' http://127.0.0.1:5000/api/users
HTTP/1.0 201 CREATED
Content-Type: application/json
Content-Length: 27
Location: http://127.0.0.1:5000/api/users/1
Server: Werkzeug/0.9.4 Python/2.7.3
Date: Thu, 28 Nov 2013 19:56:39 GMT

{
  "username": "miguel"
}

Note that in a real application this would be done over secure HTTP. There is no point in going through the effort of protecting the API if the login credentials are going to travel through the network in clear text.

Password Based Authentication

Now let's assume there is a resource exposed by this API that needs to be available only to registered users. This resource is accessed at the /api/resource endpoint.

To protect this resource I'm going to use HTTP Basic Authentication, but instead of implementing this protocol by hand I'm going to let the Flask-HTTPAuth extension do it for me.

Using Flask-HTTPAuth an endpoint is protected by adding the login_required decorator to it:

from flask_httpauth import HTTPBasicAuth
auth = HTTPBasicAuth()

@app.route('/api/resource')
@auth.login_required
def get_resource():
    return jsonify({ 'data': 'Hello, %s!' % g.user.username })

But of course Flask-HTTPAuth needs to be given some more information to know how to validate user credentials, and for this there are several options depending on the level of security implemented by the application.

The option that gives the maximum flexibility (and the only that can accomodate PassLib hashes) is implemented through the verify_password callback, which is given the username and password and is supposed to return True if the combination is valid or False if not. Flask-HTTPAuth invokes this callback function whenever it needs to validate a username and password pair.

An implementation of the verify_password callback for the example API is shown below:

@auth.verify_password
def verify_password(username, password):
    user = User.query.filter_by(username = username).first()
    if not user or not user.verify_password(password):
        return False
    g.user = user
    return True

This function finds the user by the username, then verifies the password using the verify_password() method. If the credentials are valid then the user is stored in Flask's g object so that the view function can use it.

Here is an example curl request that gets the protected resource for the user registered above:

$ curl -u miguel:python -i -X GET http://127.0.0.1:5000/api/resource
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 30
Server: Werkzeug/0.9.4 Python/2.7.3
Date: Thu, 28 Nov 2013 20:02:25 GMT

{
  "data": "Hello, miguel!"
}

If an incorrect login is used, then this is what happens:

$ curl -u miguel:ruby -i -X GET http://127.0.0.1:5000/api/resource
HTTP/1.0 401 UNAUTHORIZED
Content-Type: text/html; charset=utf-8
Content-Length: 19
WWW-Authenticate: Basic realm="Authentication Required"
Server: Werkzeug/0.9.4 Python/2.7.3
Date: Thu, 28 Nov 2013 20:03:18 GMT

Unauthorized Access

Once again I feel the need to reiterate that in a real application the API should be available on secure HTTP only.

Token Based Authentication

Having to send the username and the password with every request is inconvenient and can be seen as a security risk even if the transport is secure HTTP, since the client application must have those credentials stored without encryption to be able to send them with the requests.

An improvement over the previous solution is to use a token to authenticate requests.

The idea is that the client application exchanges authentication credentials for an authentication token, and in subsequent requests just sends this token.

Tokens are usually given out with an expiration time, after which they become invalid and a new token needs to be obtained. The potential damage that can be caused if a token is leaked is much smaller due to their short life span.

There are many ways to implement tokens. A straightforward implementation is to generate a random sequence of characters of certain length that is stored with the user and the password in the database, possibly with an expiration date as well. The token then becomes sort of a plain text password, in that can be easily verified with a string comparison, plus a check of its expiration date.

A more elaborated implementation that requires no server side storage is to use a cryptographically signed message as a token. This has the advantage that the information related to the token, namely the user for which the token was generated, is encoded in the token itself and protected against tampering with a strong cryptographic signature.

Flask uses a similar approach to write secure cookies. This implementation is based on a package called itsdangerous, which I will also use here.

The token generation and verification can be implemented as additional methods in the User model:

from itsdangerous import (TimedJSONWebSignatureSerializer
                          as Serializer, BadSignature, SignatureExpired)

class User(db.Model):
    # ...

    def generate_auth_token(self, expiration = 600):
        s = Serializer(app.config['SECRET_KEY'], expires_in = expiration)
        return s.dumps({ 'id': self.id })

    @staticmethod
    def verify_auth_token(token):
        s = Serializer(app.config['SECRET_KEY'])
        try:
            data = s.loads(token)
        except SignatureExpired:
            return None # valid token, but expired
        except BadSignature:
            return None # invalid token
        user = User.query.get(data['id'])
        return user

In the generate_auth_token() method the token is an encrypted version of a dictionary that has the id of the user. The token will also have an expiration time embedded in it, which by default will be of ten minutes (600 seconds).

The verification is implemented in a verify_auth_token() static method. A static method is used because the user will only be known once the token is decoded. If the token can be decoded then the id encoded in it is used to load the user, and that user is returned.

The API needs a new endpoint that the client can use to request a token:

@app.route('/api/token')
@auth.login_required
def get_auth_token():
    token = g.user.generate_auth_token()
    return jsonify({ 'token': token.decode('ascii') })

Note that this endpoint is protected with the auth.login_required decorator from Flask-HTTPAuth, which requires that username and password are provided.

What remains is to decide how the client is to include this token in a request.

The HTTP Basic Authentication protocol does not specifically require that usernames and passwords are used for authentication, these two fields in the HTTP header can be used to transport any kind of authentication information. For token based authentication the token can be sent as a username, and the password field can be ignored.

This means that now the server can get some requests authenticated with username and password, while others authenticated with an authentication token. The verify_password callback needs to support both authentication styles:

@auth.verify_password
def verify_password(username_or_token, password):
    # first try to authenticate by token
    user = User.verify_auth_token(username_or_token)
    if not user:
        # try to authenticate with username/password
        user = User.query.filter_by(username = username_or_token).first()
        if not user or not user.verify_password(password):
            return False
    g.user = user
    return True

This new version of the verify_password callback attempts authentication twice. First it tries to use the username argument as a token. If that doesn't work, then username and password are verified as before.

The following curl request gets an authentication token:

$ curl -u miguel:python -i -X GET http://127.0.0.1:5000/api/token
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 139
Server: Werkzeug/0.9.4 Python/2.7.3
Date: Thu, 28 Nov 2013 20:04:15 GMT

{
  "token": "eyJhbGciOiJIUzI1NiIsImV4cCI6MTM4NTY2OTY1NSwiaWF0IjoxMzg1NjY5MDU1fQ.eyJpZCI6MX0.XbOEFJkhjHJ5uRINh2JA1BPzXjSohKYDRT472wGOvjc"
}

Now the protected resource can be obtained authenticating with the token:

$ curl -u eyJhbGciOiJIUzI1NiIsImV4cCI6MTM4NTY2OTY1NSwiaWF0IjoxMzg1NjY5MDU1fQ.eyJpZCI6MX0.XbOEFJkhjHJ5uRINh2JA1BPzXjSohKYDRT472wGOvjc:unused -i -X GET http://127.0.0.1:5000/api/resource
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 30
Server: Werkzeug/0.9.4 Python/2.7.3
Date: Thu, 28 Nov 2013 20:05:08 GMT

{
  "data": "Hello, miguel!"
}

Note that in this last request the password is written as the word unused. The password in this request can be anything, since it isn't used.

OAuth Authentication

When talking about RESTful authentication the OAuth protocol is usually mentioned.

So what is OAuth?

OAuth can be many things. It is most commonly used to allow an application (the consumer) to access data or services that the user (the resource owner) has with another service (the provider), and this is done in a way that prevents the consumer from knowing the login credentials that the user has with the provider.

For example, consider a website or application that asks you for permission to access your Facebook account and post something to your timeline. In this example you are the resource holder (you own your Facebook timeline), the third party application is the consumer and Facebook is the provider. Even if you grant access and the consumer application writes to your timeline, it never sees your Facebook login information.

This usage of OAuth does not apply to a client/server RESTful API. Something like this would only make sense if your RESTful API can be accessed by third party applications (consumers).

In the case of a direct client/server communication there is no need to hide login credentials, the client (curl in the examples above) receives the credentials from the user and uses them to authenticate requests with the server directly.

OAuth can do this as well, and then it becomes a more elaborated version of the example described in this article. This is commonly referred to as the "two-legged OAuth", to contrast it to the more common "three-legged OAuth".

If you decide to support OAuth there are a few implementations available for Python listed in the OAuth website.

Conclusion

I hope this article helped you understand how to implement user authentication for your API.

Once again, you can download and play with a fully working implementation of the server described above. You can find the software on my GitHub site: REST-auth. Once again keep in mind that from time to time I update the code on GitHub. Use this link to access the code version featured in this article.

If you have any questions or found any flaws in the solution I presented please let me know below in the comments.

Miguel

207 comments

  • #126 Miguel Grinberg said 2016-05-12T18:14:03Z

    @Christoph: No, an anonymous user is not allowed to post data, because the request needs to carry user credentials. Either a username/password pair, or maybe a token. See the authentication section in this article.

  • #127 Miguel Grinberg said 2016-05-12T18:24:39Z

    @Matt: the easiest way to handle token expiration is to require that the client gets a brand new token when the current token expires and it starts getting 401s. You can use some mechanism by which the token expiration is extended each time a request comes in, but that prevents you from storing the expiration inside the token, you'll need to put it in the user database. In general, REST favors stateless services, so my first approach would be the one I would call more "RESTful".

  • #128 Evan B said 2016-05-22T11:27:17Z

    This is a cool article, do you have any on OAuth authentication for python applications? Been reading the docs but can't seem to find anything with solid documentation on how something like this should be implemented. Going to give the methods in here a try and see if I can secure my app.

  • #129 Miguel Grinberg said 2016-05-23T06:12:23Z

  • #130 Black said 2016-05-25T07:26:59Z

    Thanks for the series of articles on Flask, Miguel, they were very helpful for me to build API server.

    I have one question on API design, and it would be great if you have time to give it a look. I am currently building an Android app, it is trivial news app which needs to retrieve news from APIs. I've build initial version of API server using Flask, so far it is working good. Now I would like to add some level of security for APIs (for sure, there will be HTTPS), so that only the app can access it. The app will be public, and there is no need for user authentication. Initially, I was thinking of using token based authentication, but as I don't keep user data, this might be an over-engineering. Now I am leaning towards using Basic Auth, however my concern is how to define some (text) value that can be used as authentication, and also can be generated/verified in both server/client. By that I mean, for example, client should be able to create some value and use it in Authentication header, and server needs to understand that the header is valid one. Hopefully I could deliver my message, sorry if not. Thanks in advance.

  • #131 Miguel Grinberg said 2016-06-02T01:48:49Z

    @Black: I would just use a random string for a token. The server and client must know this token in advance. Unfortunately if your client is a JavaScript app running on the browser there isn't a good way to hide the token, as it will have to appear in your client-side code.

  • #132 Dennis Zhu said 2016-06-08T08:39:25Z

    Thanks for the greate tutorial! I got a question,how can I invalid a token by manually instead by time expiration in section 'Token Based Authentication'?

  • #133 Miguel Grinberg said 2016-06-09T15:41:24Z

    @Dennis: With this type of token, the only way to invalidate one is by having a database table of revoked tokens. When you get a token you first make sure it isn't marked as revoked by checking this table.

  • #134 Jack said 2016-06-12T23:36:30Z

    Great article! I downloaded the api.py and ran your example curl commands using Windows. Using Windows, I ran into trouble when I tried to register a new user. I had to escape the JSON code differently: curl -i -X POST -H "Content-Type: application/json" -d "{"""username""":"""miguel""","""password""":"""p ython"""}" http://127.0.0.1:5000/api/users.

    It took me a few hours to figure that one out. The rest of the curl commands ran fine.

    Thanks, Jack

  • #135 cecemel said 2016-09-01T06:20:35Z

    Thx for the article.

    How would you implement with this token based mechanism, a log-out functionality?

  • #136 Miguel Grinberg said 2016-09-10T17:47:33Z

    @cecemel: when you work with a stateless API there is really no concept of login or logout. The client requests a token and this token is used every time a request is made, so essentially, the server logs the user in and out with every request. If you want a token to stop working, then typically applications do so by revoking the token. In this case, revoking a token can be implemented by having a database table with revoked tokens, which is checked every time a client sends a token.

  • #137 Jason said 2016-09-16T19:23:58Z

    Excellent article. We have implemented this for our API, but need to decide how our client persists the token. We are developing a single-page app, and would like to do everything (including authentication) through the API. In terms of persistency, it seems we have the following options.

    1) send the token as a cookie, which the app can use until expiration, even if app is closed/re-opened. 2) return a token which is stored as a variable in the app. Token will persist until it expires or app is closed. 3) return a new token with every request. Token will persist until it expires or the app is closed.

    Which of these do you believe to be the most secure and most efficient? Are any of these more vulnerable to XSS or CSRF attacks?

    Also, what are the best ways to protect the API against XSS/CSRF attacks?

    Thank you! I also own your Flask book which is excellent!

  • #138 Miguel Grinberg said 2016-09-21T19:32:29Z

    @Jason: not sure I understand your #1, #2 and #3 options, they do not seem to be describing solutions to the same problem. #1 is about how to send the token, #2 and #3 are more about how a token is stored or implemented.

    The client normally sends the token in a header, either the standard Authorization header, a custom header or a cookie (which is also a header). If you use SSL then all those are secure and prevent third parties to capture the token in transit.

    The client application can store the token in a variable, or maybe in local storage. The important thing is that the token needs to be available every time a request is sent, because it needs to be inserted in the corresponding header.

    Your #3 is not clear to me. Changing the token every time a request is issued is essentially the same as using a very short token expiration time. Nothing wrong with it, but your application will have to have the logic to deal with constant token changes.

  • #139 Odko said 2016-10-11T11:57:40Z

    Hi, I want to send request with token in POSTMAN. How should I do that? I just wrote token in Authentication field in POSTMAN. But it responded 'Unauthorized Access'. Please help me

  • #140 Miguel Grinberg said 2016-10-12T00:28:05Z

    @Odko: you need to put the token in the username field, and leave the password blank.

  • #141 Ajay rawat said 2016-10-26T06:15:05Z

    I am getting success when hitting logout, i am using session.pop() for users token, but actually its logging out the user. user object is still there in the session

  • #142 Doug C said 2017-01-03T16:37:08Z

    Miguel, thanks for your articles. I have a few questions for you. I've read all your rest articles but been struggling to implement the token auth and would love to see an example of the js/client specifically for tokens.

    1) When going directly to a route and getting the "error": "Unauthorized access" message should I be re-directing them (via htaccess or something) to a page to enter a username / password?

    2) If I am making an Ajax request using the before Send xhr.setRequestHeader("Authorization","Basic " + btoa(this.username + ":" + this.password)); would I somehow want to change this to send the token instead after being authenticated the first time?

    3) Is the normal flow to have the user enter a username / password for the first time > go to api/token to get the token > store the token in js somehow then redirect to the page the user wants to see?

    Thanks.

  • #143 Miguel Grinberg said 2017-01-03T21:54:07Z

    @Doug: answers you your questions:

    1) Redirecting to the login page is a good solution. But I would do it on the status code of the response (401) and not on the text of the error message.

    2) Yes. Once you obtain a token, you will be sending that token with every request. Typically, the authorization header for a token uses the format "Bearer ".

    3) I'm not sure there is a "normal" flow. A style that in my opinion makes sense is to separate username/password auth from the actual application. When the app needs to let a user in, it redirects to the user subsystem, which prompts for username and password, and when the user is accepted, redirects back to the original app with a token. Your application never sees username and password, the only thing it ever knows is the token. this is more or less how OAuth works.

  • #144 Alan said 2017-01-11T00:18:27Z

    About generating signed access tokens, a problem I see is that it would be impossible to invalidate once it's been generated (other than letting it expire naturally, or changing the app secret, but that would kill all tokens). The only option I can see is adding a 'valid' flag on the database for each token, but that kind of defeats the purpose of signing.

    So, if for example I want to expire a specific session, or generate a one-time access token, this approach would be a no-no.

    Am I right or missing something?

    Thanks!

  • #145 Miguel Grinberg said 2017-01-11T06:18:28Z

    @Alan: The implementation that I've seen to revoke JWT tokens uses a database table to store revoked tokens. Every time the server receives a token, it needs to check that the token hasn't been revoked by searching for it in this table.

  • #146 Doug C said 2017-01-12T18:29:25Z

    I'm sending my headers like this in sessionStorage for now:

    headers: { 'Authorization': sessionStorage.getItem('token') + ':unused' },

    I've also tried headers: { 'Authorization': 'Bearer ' + sessionStorage.getItem('token') + ':unused' },

    It looks like this in the Chrome dev tool request (xxxx being the current token): Authorization:xxxx:unused

    The token is correct and if I send it in curl it works but I still get a 403 from the browser. Any ideas? Thanks.

  • #147 Miguel Grinberg said 2017-01-12T23:23:32Z

    @Doug: not sure what's the relation to sessionStorage here. In any case, the Authorization header needs to have the scheme as a first word. That typically is "Basic" or "Digest", sometimes "Bearer". It depends on the API. For the example in this article, you can see above that the format is "Basic TOKEN:unused"

  • #148 Doug said 2017-01-13T05:25:55Z

    I must be doing something wrong. I don't know why this works in curl:

    curl -u eyJhbGciOiJIUzI1NiIsImV4cCI6MTQ4NDI4NTMwMywiaWF0IjoxNDg0Mjg0NzAzfQ.eyJpZCI6MX0.G-zcBdgjm7sGHY5cXDMzPwYeaV-aOgHU6ffl-J79zo0:unused -i -X GET http://127.0.0.1:5000/api/pins

    but the same token in a header (shown from Chrome Dev tools) returns a 500 error using 'Basic ' before the token. Without 'Basic ' or with 'Bearer ' it's a 403. My Python code is copied straight from the tutorial.

    From Chrome: Authorization:Basic eyJhbGciOiJIUzI1NiIsImV4cCI6MTQ4NDI4NTMwMywiaWF0IjoxNDg0Mjg0NzAzfQ.eyJpZCI6MX0.G-zcBdgjm7sGHY5cXDMzPwYeaV-aOgHU6ffl-J79zo0:unused

  • #149 Miguel Grinberg said 2017-01-13T07:24:05Z

    @Doug: And did you investigate what error caused the 500 error?

  • #150 Doug said 2017-01-13T18:28:15Z

    It's working now using 'Basic ' in the header. Also, I had removed the btoa method from the token before as I thought it was causing me problems. So this is working. Thanks for your quick responses and everything you have done to help people learn Flask and Python. ... authorization: "Basic " + btoa(sessionStorage.getItem("token") + ":unused") ...

    In $.ajax:

    headers: { "Authorization": this.authorization },

Leave a Comment