2013-07-28T19:11:38Z

Designing a RESTful API using Flask-RESTful

This is the third article in which I explore different aspects of writing RESTful APIs using the Flask microframework. Here is the first, and the second.

The example RESTful server I wrote before used only Flask as a dependency. Today I will show you how to write the same server using Flask-RESTful, a Flask extension that simplifies the creation of APIs.

The RESTful server

As a reminder, here is the definition of the ToDo List web service that has been serving as an example in my RESTful articles:

HTTP MethodURIAction
GEThttp://[hostname]/todo/api/v1.0/tasksRetrieve list of tasks
GEThttp://[hostname]/todo/api/v1.0/tasks/[task_id]Retrieve a task
POSThttp://[hostname]/todo/api/v1.0/tasksCreate a new task
PUThttp://[hostname]/todo/api/v1.0/tasks/[task_id]Update an existing task
DELETEhttp://[hostname]/todo/api/v1.0/tasks/[task_id]Delete a task

The only resource exposed by this service is a "task", which has the following data fields:

  • uri: unique URI for the task. String type.
  • title: short task description. String type.
  • description: long task description. Text type.
  • done: task completion state. Boolean type.

Routing

In my first RESTful server example (source code here) I have used regular Flask view functions to define all the routes.

Flask-RESTful provides a Resource base class that can define the routing for one or more HTTP methods for a given URL. For example, to define a User resource with GET, PUT and DELETE methods you would write:

from flask import Flask
from flask_restful import Api, Resource

app = Flask(__name__)
api = Api(app)

class UserAPI(Resource):
    def get(self, id):
        pass

    def put(self, id):
        pass

    def delete(self, id):
        pass

api.add_resource(UserAPI, '/users/<int:id>', endpoint = 'user')

The add_resource function registers the routes with the framework using the given endpoint. If an endpoint isn't given then Flask-RESTful generates one for you from the class name, but since sometimes the endpoint is needed for functions such as url_for I prefer to make it explicit.

My ToDo API defines two URLs: /todo/api/v1.0/tasks for the list of tasks, and /todo/api/v1.0/tasks/<int:id> for an individual task. Since Flask-RESTful's Resource class can wrap a single URL this server will need two resources:

class TaskListAPI(Resource):
    def get(self):
        pass

    def post(self):
        pass

class TaskAPI(Resource):
    def get(self, id):
        pass

    def put(self, id):
        pass

    def delete(self, id):
        pass

api.add_resource(TaskListAPI, '/todo/api/v1.0/tasks', endpoint = 'tasks')
api.add_resource(TaskAPI, '/todo/api/v1.0/tasks/<int:id>', endpoint = 'task')

Note that while the method views of TaskListAPI receive no arguments the ones in TaskAPI all receive the id, as specified in the URL under which the resource is registered.

Request Parsing and Validation

When I implemented this server in the previous article I did my own validation of the request data. For example, look at how long the PUT handler is in that version:

@app.route('/todo/api/v1.0/tasks/<int:task_id>', methods = ['PUT'])
@auth.login_required
def update_task(task_id):
    task = filter(lambda t: t['id'] == task_id, tasks)
    if len(task) == 0:
        abort(404)
    if not request.json:
        abort(400)
    if 'title' in request.json and type(request.json['title']) != unicode:
        abort(400)
    if 'description' in request.json and type(request.json['description']) is not unicode:
        abort(400)
    if 'done' in request.json and type(request.json['done']) is not bool:
        abort(400)
    task[0]['title'] = request.json.get('title', task[0]['title'])
    task[0]['description'] = request.json.get('description', task[0]['description'])
    task[0]['done'] = request.json.get('done', task[0]['done'])
    return jsonify( { 'task': make_public_task(task[0]) } )

Here I have to make sure the data given with the request is valid before using it, and that makes the function pretty long.

Flask-RESTful provides a much better way to handle this with the RequestParser class. This class works in a similar way as argparse for command line arguments.

First, for each resource I define the arguments and how to validate them:

from flask_restful import reqparse

class TaskListAPI(Resource):
    def __init__(self):
        self.reqparse = reqparse.RequestParser()
        self.reqparse.add_argument('title', type = str, required = True,
            help = 'No task title provided', location = 'json')
        self.reqparse.add_argument('description', type = str, default = "", location = 'json')
        super(TaskListAPI, self).__init__()

    # ...

class TaskAPI(Resource):
    def __init__(self):
        self.reqparse = reqparse.RequestParser()
        self.reqparse.add_argument('title', type = str, location = 'json')
        self.reqparse.add_argument('description', type = str, location = 'json')
        self.reqparse.add_argument('done', type = bool, location = 'json')
        super(TaskAPI, self).__init__()

    # ...

In the TaskListAPI resource the POST method is the only one the receives arguments. The title argument is required here, so I included an error message that Flask-RESTful will send as a response to the client when the field is missing. The description field is optional, and when it is missing a default value of an empty string will be used. One interesting aspect of the RequestParser class is that by default it looks for fields in request.values, so the location optional argument must be set to indicate that the fields are coming in request.json.

The request parser for the TaskAPI is constructed in a similar way, but has a few differences. In this case it is the PUT method that will need to parse arguments, and for this method all the arguments are optional, including the done field that was not part of the request in the other resource.

Now that the request parsers are initialized, parsing and validating a request is pretty easy. For example, note how much simpler the TaskAPI.put() method becomes:

    def put(self, id):
        task = filter(lambda t: t['id'] == id, tasks)
        if len(task) == 0:
            abort(404)
        task = task[0]
        args = self.reqparse.parse_args()
        for k, v in args.iteritems():
            if v != None:
                task[k] = v
        return jsonify( { 'task': make_public_task(task) } )

A side benefit of letting Flask-RESTful do the validation is that now there is no need to have a handler for the bad request code 400 error, this is all taken care of by the extension.

Generating Responses

My original REST server generates the responses using Flask's jsonify helper function. Flask-RESTful automatically handles the conversion to JSON, so instead of this:

        return jsonify( { 'task': make_public_task(task) } )

I can do this:

        return { 'task': make_public_task(task) }

Flask-RESTful also supports passing a custom status code back when necessary:

        return { 'task': make_public_task(task) }, 201

But there is more. The make_public_task wrapper from the original server converted a task from its internal representation to the external representation that clients expected. The conversion included removing the id field and adding a uri field in its place. Flask-RESTful provides a helper function to do this in a much more elegant way that not only generates the uri but also does type conversion on the remaining fields:

from flask_restful import fields, marshal

task_fields = {
    'title': fields.String,
    'description': fields.String,
    'done': fields.Boolean,
    'uri': fields.Url('task')
}

class TaskAPI(Resource):
    # ...

    def put(self, id):
        # ...
        return { 'task': marshal(task, task_fields) }

The task_fields structure serves as a template for the marshal function. The fields.Url type is a special type that generates a URL. The argument it takes is the endpoint (recall that I have used explicit endpoints when I registered the resources specifically so that I can refer to them when needed).

Authentication

The routes in the REST server are all protected with HTTP basic authentication. In the original server the protection was added using the decorator provided by the Flask-HTTPAuth extension.

Since the Resouce class inherits from Flask's MethodView, it is possible to attach decorators to the methods by defining a decorators class variable:

from flask_httpauth import HTTPBasicAuth
# ...
auth = HTTPBasicAuth()
# ...

class TaskAPI(Resource):
    decorators = [auth.login_required]
    # ...

class TaskAPI(Resource):
    decorators = [auth.login_required]
    # ...

Conclusion

The complete server implementation based on Flask-RESTful is available in my REST-tutorial project on github. The file with the Flask-RESTful server is rest-server-v2.py.

You can also download the entire project including both server implementations and a javascript client to test it:

Download REST-tutorial project.

Let me know if you have any questions in the comments below.

Miguel

188 comments

  • #76 Miguel Grinberg said 2014-10-06T00:36:38Z

    @spitz: why do you need templates in your API? Templates are normally used to render HTML, but the responses from API routes are generated from JSON or XML, so they do not use templates.

  • #77 Victor said 2014-11-08T05:24:24Z

    Miguel, thanks for the fantastic tutorial. Really well done!

    I am trying to wrap my brain around how to separate files for a large project and still access things I need. So I have a main.py that instantiates everything in, including all the HTTPBasicAuth auth routines. Then in foo.py I have the actual class with the get/put methods defined there, and in my main.py I register foo.py via api.add_resource.

    So far so good. But how do I use the @auth.login_required decorator in foo.py since the @auth.get_password function is defined in main.py? Does that make sense?

    Thanks again for a wonderful tutorial

  • #78 Miguel Grinberg said 2014-11-08T07:36:32Z

    @Victor: one way to do it would be to move all the authentication related things to a auth.py module, which both main.py and foo.py can import.

  • #79 Victor said 2014-11-08T08:10:18Z

    Miguel, I tried exactly that actually ;)

    Does the 'auth = HTTPBasicAuth()' definition live centrally in the auth.py module? Or does each module need to instantiate it separately? I thought I had a pretty good grasp on Python scoping issues until I ran into this ;)

  • #80 Victor said 2014-11-08T08:22:41Z

    Figured it out, 30 seconds after posting :)

    In my auth.py file I just did: def get_auth(): return auth

    And in foo.py did: myauth = auth.get_auth()

    And it all works. Thank you again for the tutorial and help!

  • #81 Miguel Grinberg said 2014-11-09T02:17:20Z

    @Victor: your solution is fine. Another way to achieve the same thing is to add "from auth import auth" at the top of your foo.py module.

  • #82 Bhargav said 2014-12-07T17:07:37Z

    Hey, I just wanted to say thanks for the amazing tutorials. It is really helping me understand and learn flask better.

  • #83 John said 2014-12-20T23:47:33Z

    Hi Miguel,

    Thanks for the Tutorial. What is the best way to handle db.relationship things inside the Flask-Restful API framework?

    I was thinking of different ways to do it. Does it make sense just return the unique ID or is there some better solution? If so, should I do the same thing for the table and not use relationships but instead a series of joins in flask sql alchemy?

    Thanks

  • #84 Miguel Grinberg said 2014-12-21T03:10:03Z

    @John: For the API to be RESTful you need to return URL links. See the 2nd chapter of the RESTful API training video I did for O'Reilly for some examples: http://player.oreilly.com/videos/9781491911938?toc_id=190358 (this chapter is available for free!)

  • #85 John said 2014-12-21T19:57:14Z

    Thanks for response. Website seems pretty useful, I would love access to it. However I am not able to see anything beyond the intro without paying. I'm trying to program something on the side and as such, don't have a corporate account to pay for any of these expenses. $90 bucks everytime I need a tutorial is pretty expensive for me.

  • #86 Miguel Grinberg said 2014-12-22T03:01:56Z

    @John: the first two chapter are freely available. I gave you a direct link to the 2nd chapter, which shows how I work with links in a demonstration.

  • #87 John said 2014-12-23T09:23:33Z

    Got it, I see what you are saying now.

    This is awesome, thanks

  • #88 Devan said 2014-12-31T16:31:20Z

    Great tutorial, thanks for putting this together.

    I was hoping to create separate modules for parts of the RESTful api and then tie them together in a file where the app is declared. Every way I look at it I'm finding myself stuck in a circular reference back to the HTTPBasicAuth() object. Is it still possible to use the HTTPAuth extension in this model?

    Thanks, Devan

  • #89 Miguel Grinberg said 2015-01-01T01:21:13Z

    @Devan: Have a look at the example project I created for my PyCon 2014 talk on REST APIs with Flask. https://github.com/miguelgrinberg/api-pycon2014

  • #90 Flasker said 2015-01-19T15:53:19Z

    Thanks for a great tutorial!

    How would you recommend extending this example and add a mongoDB database to manage all the data?

  • #91 Miguel Grinberg said 2015-01-19T22:42:34Z

    @Flasker: I don't think the example requires much changes, just replace the in-memory data structure with your database calls, be it Mongo or any other. Is there anything in particular that isn't clear on how to migrate to a real database?

  • #92 Flasker said 2015-01-20T06:58:00Z

    I'm wondering what's the best approach to embed a mongo DB, the example doesn't require many changes, I'm not sure how and where to insert the DB connection and what is the best approach to manage the DB scheme in a way that supports more complicated requests (e.g. sort by one of the dates fields, get part of a document).

    Can you refer me to a good reference on that?

    BTW, I'm also thinking about the client side, the current code display the items using a for loop, what if I want to add the option to sort the items or add sub-pages but still use an api? lets say I would want to view more details on a task and would want to click on it and move to the task page with all of its info, can it be done with the current page structure? or should I use the template system?

    Basically I'm going to save many documents (objects) to a mongo DB and need to update the status for each document stored (this would have been another table in a rational db), I would want to show the documents by the status fields, filter by date etc'.

    Thanks!

  • #93 Miguel Grinberg said 2015-01-20T19:59:04Z

    @Flasker: you may want to watch my PyCon 2014 talk on REST APIs. I'm using SQLAlchemy there, but adapting to Mongo should be easy. I also show how to implement decorators to add more features. I don't specifically show sorting, but the techniques are similar to what I show in this talk.

  • #94 Marcel Maré said 2015-01-31T16:02:30Z

    Hi Miguel

    I was wondering why you created two Resources (TaskApi and TaskListApi) instead of just TaskApi and registering it with two endpoints: api.add_resource(TaskAPI, '/todo/api/v1.0/tasks/', '/todo/api/v1.0/tasks', endpoint = 'tasks')

    In the GET method (def get(self, id=None)) you could just test for the presence of an id to decide whether to handle the case of a single task or all tasks.

  • #95 Miguel Grinberg said 2015-02-01T03:33:25Z

    @Marcel: I prefer to keep the different URLs separate, I'm not a fan of using the same function to handle two completely different behaviors.

  • #96 Brett Gerry said 2015-02-06T21:16:13Z

    Thanks again Miguel for a great tutorial. I am curious about the use of decorators=[..] vs. method_decorators=[..] for authentication and other uses. In my API I am doing a further check beyond basic auth for access to account-level endpoints such as /Accounts//MyThings to make sure the user is authorized to access to that account-level endpoint.

    I use a custom decorator to do so, but ran into a problem when I tried to add it to the decorators=[..] list because the flask.g globals I set where not in context within my decorator function. What I needed to do was add my decorator to method_decorators=[..] which will attach it to all resource method functions. See http://flask-restful.readthedocs.org/en/0.3.1/extending.html#resource-method-decorators.

    Can you comment on the right time to use decorators vs method_decorators in flask-restful?

  • #97 Miguel Grinberg said 2015-02-07T00:47:04Z

    @Brett: the "decorators" option comes from the class based views from Flask, which Flask-RESTful inherits from. I'm surprised you couldn't get Flask.g to work, the context should be set by the time these decorators are invoked. The "method_decorators" is an alternative solution provided directly by Flask-RESTful. It is implemented in a slightly different way, as these decorates the actual methods such as get(), post(), etc., while the Flask decorators are applied to wrapper functions. The difference is subtle, the method_decorators will see the "self" argument, because they decorate a method, while the Flask decorators will not have that. Unless the decorator needs to use the arguments sent to the view function both options should give you the same result.

  • #98 Jia said 2015-02-27T16:34:54Z

    Hi Miguel,

    Thank you for this tutorial series, very helpful. I have one question about the reqparse module. My intention is to allow users to search my task list using /todo/api/v1.0/tasks?title='something'&description='something else' so to begin with, I tried to modify the TaskListAPI.get() to return self.reqparse.parse_args(), but the result tells me neither title nor description has been assigned. Could you let me know why? I tried with the other version that doesn't use the flask restful api extension module and it works well. Thank you very much for your help in advance.

  • #99 Miguel Grinberg said 2015-03-01T01:52:44Z

    @Jia: do you have your code somewhere I can review it? I think if you define the title and description arguments with add_argument() then they should be parsed and returned by parse_args().

  • #100 Jia said 2015-03-05T15:57:53Z

    @Miguel: shown below is my code that attempts to use the API for setting up an API for clinical trial manager. The URL I tried in accessing the get function is http://localhost:5000/TrialManager/TrialDetails?nctid=10101010&phase=1 Your input would be greatly appreciated.

    !flask/bin/python

    from flask import Flask, jsonify, abort, make_response from flask.ext.restful import Api, Resource from flask.ext.restful import reqparse from flask.ext.restful import fields, marshal from flask.ext.httpauth import HTTPBasicAuth

    app = Flask(name) api = Api(app) auth = HTTPBasicAuth()

    @auth.get_password def get_password(username): password = None if username == 'user': password = 'password' return password

    @auth.error_handler def unauthorized(): return make_response(jsonify({'message': 'Unauthorized access'}), 403)

    trial_fields = { 'nctid': fields.String, 'phase': fields.Integer }

    class TrialManagerAPI(Resource): decorators = [auth.login_required]

    def __init__(self): self.reqparse = reqparse.RequestParser() self.reqparse.add_argument('nctid', type=str, location='json') self.reqparse.add_argument('phase', type=int, location='json') super(TrialManagerAPI, self).__init__() def get(self): args = self.reqparse.parse_args() return args

    api.add_resource(TrialManagerAPI, '/TrialManager/TrialDetails', endpoint='TrialDetails')

    if name == 'main': # the argument configuration for the app.run function is super important for # allowing the API to be accessible externally using your IP #app.run(host='0.0.0.0', debug=False) app.run(debug=True)

Leave a Comment