2018-01-23T20:01:33Z

The Flask Mega-Tutorial Part VIII: Followers

This is the eighth installment of the Flask Mega-Tutorial series, in which I'm going to tell you how to implement a "followers" feature similar to that of Twitter and other social networks.

For your reference, below is a list of the articles in this series.

In this chapter I am going to work on the application's database some more. I want users of the application to be able to easily choose which other users they want to follow. So I'm going to be expanding the database so that it can keep track of who is following who, which is trickier than you may think.

The GitHub links for this chapter are: Browse, Zip, Diff.

Database Relationships Revisited

I said above that I want to maintain a list of "followed" and "follower" users for each user. Unfortunately, a relational database does not have a list type that I can use for these lists, all there is are tables with records and relationships between these records.

The database has a table that represents users, so what's left is to come up with the proper relationship type that can model the follower/followed link. This is a good time to review the basic database relationship types:

One-to-Many

I have already used a one-to-many relationship in Chapter 4. Here is the diagram for this relationship:

One-to-many Relationship

The two entities linked by this relationship are users and posts. I say that a user has many posts, and a post has one user (or author). The relationship is represented in the database with the use of a foreign key on the "many" side. In the relationship above, the foreign key is the user_id field added to the posts table. This field links each post to the record of its author in the user table.

It is pretty clear that the user_id field provides direct access to the author of a given post, but what about the reverse direction? For the relationship to be useful I should be able to get the list of posts written by a given user. The user_id field in the posts table is also sufficient to answer this question, as databases have indexes that allow for efficient queries such us "retrieve all posts that have a user_id of X".

Many-to-Many

A many-to-many relationship is a bit more complex. As an example, consider a database that has students and teachers. I can say that a student has many teachers, and a teacher has many students. It's like two overlapped one-to-many relationships from both ends.

For a relationship of this type I should be able to query the database and obtain the list of teachers that teach a given student, and the list of students in a teacher's class. This is actually non-trivial to represent in a relational database, as it cannot be done by adding foreign keys to the existing tables.

The representation of a many-to-many relationship requires the use of an auxiliary table called an association table. Here is how the database would look for the students and teachers example:

many-to-many

While it may not seem obvious at first, the association table with its two foreign keys is able to efficiently answer all the queries about the relationship.

Many-to-One and One-to-One

A many-to-one is similar to a one-to-many relationship. The difference is that this relationship is looked at from the "many" side.

A one-to-one relationship is a special case of a one-to-many. The representation is similar, but a constraint is added to the database to prevent the "many" side to have more than one link. While there are cases in which this type of relationship is useful, it isn't as common as the other types.

Representing Followers

Looking at the summary of all the relationship types, it is easy to determine that the proper data model to track followers is the many-to-many relationship, because a user follows many users, and a user has many followers. But there is a twist. In the students and teachers example I had two entities that were related through the many-to-many relationship. But in the case of followers, I have users following other users, so there is just users. So what is the second entity of the many-to-many relationship?

The second entity of the relationship is also the users. A relationship in which instances of a class are linked to other instances of the same class is called a self-referential relationship, and that is exactly what I have here.

Here is a diagram of the self-referential many-to-many relationship that keeps track of followers:

many-to-many

The followers table is the association table of the relationship. The foreign keys in this table are both pointing at entries in the user table, since it is linking users to users. Each record in this table represents one link between a follower user and a followed user. Like the students and teachers example, a setup like this one allows the database to answer all the questions about followed and follower users that I will ever need. Pretty neat.

Database Model Representation

Let's add followers to the database first. Here is the followers association table:

app/models.py: Followers association table

followers = db.Table('followers',
    db.Column('follower_id', db.Integer, db.ForeignKey('user.id')),
    db.Column('followed_id', db.Integer, db.ForeignKey('user.id'))
)

This is a direct translation of the association table from my diagram above. Note that I am not declaring this table as a model, like I did for the users and posts tables. Since this is an auxiliary table that has no data other than the foreign keys, I created it without an associated model class.

Now I can declare the many-to-many relationship in the users table:

app/models.py: Many-to-many followers relationship

class User(UserMixin, db.Model):
    # ...
    followed = db.relationship(
        'User', secondary=followers,
        primaryjoin=(followers.c.follower_id == id),
        secondaryjoin=(followers.c.followed_id == id),
        backref=db.backref('followers', lazy='dynamic'), lazy='dynamic')

The setup of the relationship is non-trivial. Like I did for the posts one-to-many relationship, I'm using the db.relationship function to define the relationship in the model class. This relationship links User instances to other User instances, so as a convention let's say that for a pair of users linked by this relationship, the left side user is following the right side user. I'm defining the relationship as seen from the left side user with the name followed, because when I query this relationship from the left side I will get the list of followed users (i.e those on the right side). Let's examine all the arguments to the db.relationship() call one by one:

  • 'User' is the right side entity of the relationship (the left side entity is the parent class). Since this is a self-referential relationship, I have to use the same class on both sides.
  • secondary configures the association table that is used for this relationship, which I defined right above this class.
  • primaryjoin indicates the condition that links the left side entity (the follower user) with the association table. The join condition for the left side of the relationship is the user ID matching the follower_id field of the association table. The value of this argument is followers.c.follower_id, which qreferences the follower_id column of the association table.
  • secondaryjoin indicates the condition that links the right side entity (the followed user) with the association table. This condition is similar to the one for primaryjoin, with the only difference that now I'm using followed_id, which is the other foreign key in the association table.
  • backref defines how this relationship will be accessed from the right side entity. From the left side, the relationship is named followed, so from the right side I am going to use the name followers to represent all the left side users that are linked to the target user in the right side. The additional lazy argument indicates the execution mode for this query. A mode of dynamic sets up the query to not run until specifically requested, which is also how I set up the posts one-to-many relationship.
  • lazy is similar to the parameter of the same name in the backref, but this one applies to the left side query instead of the right side.

Don't worry if this is hard to understand. I will show you how to work with these queries in a moment, and then everything will become clearer.

The changes to the database need to be recorded in a new database migration:

(venv) $ flask db migrate -m "followers"
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.autogenerate.compare] Detected added table 'followers'
  Generating /home/miguel/microblog/migrations/versions/ae346256b650_followers.py ... done

(venv) $ flask db upgrade
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 37f06a334dbf -> ae346256b650, followers

Adding and Removing "follows"

Thanks to the SQLAlchemy ORM, a user following another user can be recorded in the database working with the followed relationship as if it was a list. For example, if I had two users stored in user1 and user2 variables, I can make the first follow the second with this simple statement:

user1.followed.append(user2)

To stop following the user, then I could do:

user1.followed.remove(user2)

Even though adding and removing followers is fairly easy, I want to promote reusability in my code, so I'm not going to sprinkle "appends" and "removes" through the code. Instead, I'm going to implement the "follow" and "unfollow" functionality as methods in the User model. It is always best to move the application logic away from view functions and into models or other auxiliary classes or modules, because as you will see later in this chapter, that makes unit testing much easier.

Below are the changes in the user model to add and remove relationships:

app/models.py: Add and remove followers

class User(UserMixin, db.Model):
    #...

    def follow(self, user):
        if not self.is_following(user):
            self.followed.append(user)

    def unfollow(self, user):
        if self.is_following(user):
            self.followed.remove(user)

    def is_following(self, user):
        return self.followed.filter(
            followers.c.followed_id == user.id).count() > 0

The follow() and unfollow() methods use the append() and remove() methods of the relationship object as I have shown above, but before they touch the relationship they use the is_following() supporting method to make sure the requested action makes sense. For example, if I ask user1 to follow user2, but it turns out that this following relationship already exists in the database, I do not want to add a duplicate. The same logic can be applied to unfollowing.

The is_following() method issues a query on the followed relationship to check if a link between two users already exists. You have seen me use the filter_by() method of the SQLAlchemy query object before, for example to find a user given its username. The filter() method that I'm using here is similar, but lower level, as it can include arbitrary filtering conditions, unlike filter_by() which can only check for equality to a constant value. The condition that I'm using in is_following() looks for items in the association table that have the left side foreign key set to the self user, and the right side set to the user argument. The query is terminated with a count() method, which returns the number of results. The result of this query is going to be 0 or 1, so checking for the count being 1 or greater than 0 is actually equivalent. Other query terminators you have seen me use in the past are all() and first().

Obtaining the Posts from Followed Users

Support for followers in the database is almost complete, but I'm actually missing one important feature. In the index page of the application I'm going to show blog posts written by all the people that are followed by the logged in user, so I need to come up with a database query that returns these posts.

The most obvious solution is to run a query that returns the list of followed users, which as you already know, it would be user.followed.all(). Then for each of these returned users I can run a query to get the posts. Once I have all the posts I can merge them into a single list and sort them by date. Sounds good? Well, not really.

This approach has a couple of problems. What happens if a user is following a thousand people? I would need to execute a thousand database queries just to collect all the posts. And then I will need to merge and sort the thousand lists in memory. As a secondary problem, consider that the application's home page will eventually have pagination implemented, so it will not display all the available posts but just the first few, with a link to get more if desired. If I'm going to display posts sorted by their date, how can I know which posts are the most recent of all followed users combined, unless I get all the posts and sort them first? This is actually an awful solution that does not scale well.

There is really no way to avoid this merging and sorting of blog posts, but doing it in the application results in a very inefficient process. This kind of work is what relational databases excel at. The database has indexes that allow it to perform the queries and the sorting in a much more efficient way that I can possibly do from my side. So what I really want is to come up with a single database query that defines the information that I want to get, and then let the database figure out how to extract that information in the most efficient way.

Below you can see this query:

app/models.py: Followed posts query

class User(UserMixin, db.Model):
    #...
    def followed_posts(self):
        return Post.query.join(
            followers, (followers.c.followed_id == Post.user_id)).filter(
                followers.c.follower_id == self.id).order_by(
                    Post.timestamp.desc())

This is by far the most complex query I have used on this application. I'm going to try to decipher this query one piece at a time. If you look at the structure of this query, you are going to notice that there are three main sections designed by the join(), filter() and order_by() methods of the SQLAlchemy query object:

Post.query.join(...).filter(...).order_by(...)

Joins

To understand what a join operation does, let's look at an example. Let's assume that I have a User table with the following contents:

id username
1 john
2 susan
3 mary
4 david

To keep things simple I am not showing all the fields in the user model, just the ones that are important for this query.

Let's say that the followers association table says that user john is following users susan and david, user susan is following mary and user mary is following david. The data that represents the above is this:

follower_id followed_id
1 2
1 4
2 3
3 4

Finally, the posts table contains one post from each user:

id text user_id
1 post from susan 2
2 post from mary 3
3 post from david 4
4 post from john 1

This table also omits some fields that are not part of this discussion.

Here is the join() call that I defined for this query once again:

Post.query.join(followers, (followers.c.followed_id == Post.user_id))

I'm invoking the join operation on the posts table. The first argument is the followers association table, and the second argument is the join condition. What I'm saying with this call is that I want the database to create a temporary table that combines data from posts and followers tables. The data is going to be merged according to the condition that I passed as argument.

The condition that I used says that the followed_id field of the followers table must be equal to the user_id of the posts table. To perform this merge, the database will take each record from the posts table (the left side of the join) and append any records from the followers table (the right side of the join) that match the condition. If multiple records in followers match the condition, then the post entry will be repeated for each. If for a given post there is no match in followers, then that post record is not part of the join.

With the example data I defined above, the result of the join operation is:

id text user_id follower_id followed_id
1 post from susan 2 1 2
2 post from mary 3 2 3
3 post from david 4 1 4
3 post from david 4 3 4

Note how the user_id and followed_id columns are equal in all cases, as this was the join condition. The post from user john does not appear in the joined table because there are no entries in followers that have john as a followed user, or in other words, nobody is following john. And the post from david appears twice, because that user is followed by two different users.

It may not be immediately clear what do I gain by creating this join, but keep reading, as this is just one part of the bigger query.

Filters

The join operation gave me a list of all the posts that are followed by some user, which is a lot more data that I really want. I'm only interested in a subset of this list, the posts followed by a single user, so I need trim all the entries I don't need, which I can do with a filter() call.

Here is the filter portion of the query:

filter(followers.c.follower_id == self.id)

Since this query is in a method of class User, the self.id expression refers to the user ID of the user I'm interested in. The filter() call selects the items in the joined table that have the follower_id column set to this user, which in other words means that I'm keeping only the entries that have this user as a follower.

Let's say the user I'm interested in is john, which has its id field set to 1. Here is how the joined table looks after the filtering:

id text user_id follower_id followed_id
1 post from susan 2 1 2
3 post from david 4 1 4

And these are exactly the posts that I wanted!

Remember that the query was issued on the Post class, so even though I ended up with a temporary table that was created by the database as part of this query, the result will be the posts that are included in this temporary table, without the extra columns added by the join operation.

Sorting

The final step of the process is to sort the results. The part of the query that does that says:

order_by(Post.timestamp.desc())

Here I'm saying that I want the results sorted by the timestamp field of the post in descending order. With this ordering, the first result will be the most recent blog post.

Combining Own and Followed Posts

The query that I'm using in the followed_posts() function is extremely useful, but has one limitation. People expect to see their own posts included in their timeline of followed users, and the query as it is does not have that capability.

There are two possible ways to expand this query to include the user's own posts. The most straightforward way is to leave the query as it is, but make sure all users are following themselves. If you are your own follower, then the query as shown above will find your own posts along with those of all the people you follow. The disadvantage of this method is that it affects the stats regarding followers. All follower counts are going to be inflated by one, so they'll have to be adjusted before they are shown. The second way to do this is by create a second query that returns the user's own posts, and then use the "union" operator to combine the two queries into a single one.

After considering both options I decided to go with the second one. Below you can see the followed_posts() function after it has been expanded to include the user's posts through a union:

app/models.py: Followed posts query with user's own posts.

    def followed_posts(self):
        followed = Post.query.join(
            followers, (followers.c.followed_id == Post.user_id)).filter(
                followers.c.follower_id == self.id)
        own = Post.query.filter_by(user_id=self.id)
        return followed.union(own).order_by(Post.timestamp.desc())

Note how the followed and own queries are combined into one, before the sorting is applied.

Unit Testing the User Model

While I don't consider the followers implementation I have built a "complex" feature, I think it is also not trivial. My concern when I write non-trivial code, is to ensure that this code will continue to work in the future, as I make modifications on different parts of the application. The best way to ensure that code you have already written continues to work in the future is to create a suite of automated tests that you can re-run each time changes are made.

Python includes a very useful unittest package that makes it easy to write and execute unit tests. Let's write some unit tests for the existing methods in the User class in a tests.py module:

tests.py: User model unit tests.

from datetime import datetime, timedelta
import unittest
from app import app, db
from app.models import User, Post

class UserModelCase(unittest.TestCase):
    def setUp(self):
        app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
        db.create_all()

    def tearDown(self):
        db.session.remove()
        db.drop_all()

    def test_password_hashing(self):
        u = User(username='susan')
        u.set_password('cat')
        self.assertFalse(u.check_password('dog'))
        self.assertTrue(u.check_password('cat'))

    def test_avatar(self):
        u = User(username='john', email='john@example.com')
        self.assertEqual(u.avatar(128), ('https://www.gravatar.com/avatar/'
                                         'd4c74594d841139328695756648b6bd6'
                                         '?d=identicon&s=128'))

    def test_follow(self):
        u1 = User(username='john', email='john@example.com')
        u2 = User(username='susan', email='susan@example.com')
        db.session.add(u1)
        db.session.add(u2)
        db.session.commit()
        self.assertEqual(u1.followed.all(), [])
        self.assertEqual(u1.followers.all(), [])

        u1.follow(u2)
        db.session.commit()
        self.assertTrue(u1.is_following(u2))
        self.assertEqual(u1.followed.count(), 1)
        self.assertEqual(u1.followed.first().username, 'susan')
        self.assertEqual(u2.followers.count(), 1)
        self.assertEqual(u2.followers.first().username, 'john')

        u1.unfollow(u2)
        db.session.commit()
        self.assertFalse(u1.is_following(u2))
        self.assertEqual(u1.followed.count(), 0)
        self.assertEqual(u2.followers.count(), 0)

    def test_follow_posts(self):
        # create four users
        u1 = User(username='john', email='john@example.com')
        u2 = User(username='susan', email='susan@example.com')
        u3 = User(username='mary', email='mary@example.com')
        u4 = User(username='david', email='david@example.com')
        db.session.add_all([u1, u2, u3, u4])

        # create four posts
        now = datetime.utcnow()
        p1 = Post(body="post from john", author=u1,
                  timestamp=now + timedelta(seconds=1))
        p2 = Post(body="post from susan", author=u2,
                  timestamp=now + timedelta(seconds=4))
        p3 = Post(body="post from mary", author=u3,
                  timestamp=now + timedelta(seconds=3))
        p4 = Post(body="post from david", author=u4,
                  timestamp=now + timedelta(seconds=2))
        db.session.add_all([p1, p2, p3, p4])
        db.session.commit()

        # setup the followers
        u1.follow(u2)  # john follows susan
        u1.follow(u4)  # john follows david
        u2.follow(u3)  # susan follows mary
        u3.follow(u4)  # mary follows david
        db.session.commit()

        # check the followed posts of each user
        f1 = u1.followed_posts().all()
        f2 = u2.followed_posts().all()
        f3 = u3.followed_posts().all()
        f4 = u4.followed_posts().all()
        self.assertEqual(f1, [p2, p4, p1])
        self.assertEqual(f2, [p2, p3])
        self.assertEqual(f3, [p3, p4])
        self.assertEqual(f4, [p4])

if __name__ == '__main__':
    unittest.main(verbosity=2)

I have added four tests that exercise the password hashing, user avatar and followers functionality in the user model. The setUp() and tearDown() methods are special methods that the unit testing framework executes before and after each test respectively. I have implemented a little hack in setUp(), to prevent the unit tests from using the regular database that I use for development. By changing the application configuration to sqlite:// I get SQLAlchemy to use an in-memory SQLite database during the tests. The db.create_all() call creates all the database tables. This is a quick way to create a database from scratch that is useful for testing. For development and production use I have already shown you how to create database tables through database migrations.

You can run the entire test suite with the following command:

(venv) $ python tests.py
test_avatar (__main__.UserModelCase) ... ok
test_follow (__main__.UserModelCase) ... ok
test_follow_posts (__main__.UserModelCase) ... ok
test_password_hashing (__main__.UserModelCase) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.494s

OK

From now on, every time a change is made to the application, you can re-run the tests to make sure the features that are being tested have not been affected. Also, each time another feature is added to the application, a unit test should be written for it.

Integrating Followers with the Application

The support of followers in the database and models is now complete, but I don't have any of this functionality incorporated into the application, so I'm going to add that now.

Because the follow and unfollow actions introduce changes in the application, I'm going to implement them as POST requests, which are triggered from the web browser as a result of submitting a web form. It would be easier to implement these routes as GET requests, but then they could be exploited in CSRF attacks. Because GET requests are harder to protect against CSRF, they should only be used on actions that do not introduce state changes. Implementing these as a result of a form submission is better because then a CSRF token can be added to the form.

But how can a follow or unfollow action be triggered from a web form when the only thing the user needs to do is click on "Follow" or "Unfollow", without submitting any data? To make this work, the form is going to be empty. The only elements in the form are going to be the CSRF token, which is implemented as a hidden field and added automatically by Flask-WTF, and a submit button, which is going to be what the user needs to click to trigger the action. Since the two actions are almost identical I'm going to use the same form for both. I'm going to call this form EmptyForm.

app/forms.py: Empty form for following and unfollowing.

class EmptyForm(FlaskForm):
    submit = SubmitField('Submit')

Let's add two new routes in the application to follow and unfollow a user:

app/routes.py: Follow and unfollow routes.

from app.forms import EmptyForm

# ...

@app.route('/follow/<username>', methods=['POST'])
@login_required
def follow(username):
    form = EmptyForm()
    if form.validate_on_submit():
        user = User.query.filter_by(username=username).first()
        if user is None:
            flash('User {} not found.'.format(username))
            return redirect(url_for('index'))
        if user == current_user:
            flash('You cannot follow yourself!')
            return redirect(url_for('user', username=username))
        current_user.follow(user)
        db.session.commit()
        flash('You are following {}!'.format(username))
        return redirect(url_for('user', username=username))
    else:
        return redirect(url_for('index'))

@app.route('/unfollow/<username>', methods=['POST'])
@login_required
def unfollow(username):
    form = EmptyForm()
    if form.validate_on_submit():
        user = User.query.filter_by(username=username).first()
        if user is None:
            flash('User {} not found.'.format(username))
            return redirect(url_for('index'))
        if user == current_user:
            flash('You cannot unfollow yourself!')
            return redirect(url_for('user', username=username))
        current_user.unfollow(user)
        db.session.commit()
        flash('You are not following {}.'.format(username))
        return redirect(url_for('user', username=username))
    else:
        return redirect(url_for('index'))

The form handling in these routes is simpler, because we only have to implement the submission part. Unlike other forms such as the login and edit profile forms, these two forms do not have their own pages, the forms will be rendered by the user() route and will appear in the user's profile page. The only reason why the validate_on_submit() call can fail is if the CSRF token is missing or invalid, so in that case I just redirect the application back to the home page.

If the form validation passes, I do some error checking before actually carrying out the follow or unfollow action. This is to prevent unexpected issues, and to try to provide a useful message to the user when a problem has occurred.

To render the follow or unfollow button, I need to instantiate an EmptyForm object and pass it to the user.html template. Because these two actions are mutually exclusive, I can pass a single instance of this generic form to the template:

app/routes.py: Follow and unfollow routes.

@app.route('/user/<username>')
@login_required
def user(username):
    # ...
    form = EmptyForm()
    return render_template('user.html', user=user, posts=posts, form=form)

I can now add the follow or unfollow forms in the profile page of each user:

app/templates/user.html: Follow and unfollow links in user profile page.

        ...
        <h1>User: {{ user.username }}</h1>
        {% if user.about_me %}<p>{{ user.about_me }}</p>{% endif %}
        {% if user.last_seen %}<p>Last seen on: {{ user.last_seen }}</p>{% endif %}
        <p>{{ user.followers.count() }} followers, {{ user.followed.count() }} following.</p>
        {% if user == current_user %}
        <p><a href="{{ url_for('edit_profile') }}">Edit your profile</a></p>
        {% elif not current_user.is_following(user) %}
        <p>
            <form action="{{ url_for('follow', username=user.username) }}" method="post">
                {{ form.hidden_tag() }}
                {{ form.submit(value='Follow') }}
            </form>
        </p>
        {% else %}
        <p>
            <form action="{{ url_for('unfollow', username=user.username) }}" method="post">
                {{ form.hidden_tag() }}
                {{ form.submit(value='Unfollow') }}
            </form>
        </p>
        {% endif %}
        ...

The changes to the user profile template add a line below the last seen timestamp that shows how many followers and followed users this user has. And the line that has the "Edit" link when you are viewing your own profile now can have one of three possible links:

  • If the user is viewing his or her own profile, the "Edit" link shows as before.
  • If the user is viewing a user that is not currently followed, the "Follow" form shows.
  • If the user is viewing a user that is currently followed, the "Unfollow" form shows.

To reuse the EmptyForm() instance for both the follow and unfollow forms, I pass a value argument when rendering the submit button. In a submit button, the value attribute defines the label, so with this trick I can change the text in the submit button depending on the action that I need to present to the user.

At this point you can run the application, create a few users and play with following and unfollowing users. The only thing you need to remember is to type the profile page URL of the user you want to follow or unfollow, since there is currently no way to see a list of users. For example, if you want to follow a user with the susan username, you will need to type http://localhost:5000/user/susan in the browser's address bar to access the profile page for that user. Make sure you check how the followed and follower user counts change as you issue follows or unfollows.

I should be showing the list of followed posts in the index page of the application, but I don't have all the pieces in place to do that yet, since users cannot write blog posts yet. So I'm going to delay this change until that functionality is in place.

345 comments

  • #226 Joshua Muwanguzi said 2020-04-23T11:22:24Z

    Thanks for the course Miguel! I'm encountering an error..looks like my database is messed up after doing the unittesting. What does the error mean and how do I fix it?

    $ flask run * Serving Flask app "microblog.py" * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off [2020-04-23 14:06:38,651] INFO in init: Microblog startup * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET / HTTP/1.1" 302 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET / HTTP/1.1" 302 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET /login?next=%2F HTTP/1.1" 200 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET /login?next=%2F HTTP/1.1" 200 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET /login?next=%2F HTTP/1.1" 200 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET /favicon.ico HTTP/1.1" 404 - 127.0.0.1 - - [23/Apr/2020 14:06:43] "GET /login?next=%2F HTTP/1.1" 200 - [2020-04-23 14:06:52,061] ERROR in app: Exception on /login [POST] Traceback (most recent call last): File "c:\users\dell e7240\desktop\microblog\env\lib\site-packages\sqlalchemy\engine\base.py", line 1247, in _execute_context self.dialect.do_execute( File "c:\users\dell e7240\desktop\microblog\env\lib\site-packages\sqlalchemy\engine\default.py", line 590, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: user

  • #227 Mostafa Abobakr said 2020-04-23T21:32:10Z

    shouldn't we add follow/unfollow links as forms, so they are accessed by post methods only? also, we will avoid the ifs in the view functions.

  • #228 Glen Veigas said 2020-04-24T07:45:47Z

    Hi Miguel I have implemented the list followers and following functionality that displays the list of users following a particular user and list of users that the user is following respectively

    I have done the following in my routes.py file for displaying list of followers

    @app.route("/user//userListFollowers") @login_required def listFollowers(username): selected_user = User.query.filter_by(username=username).first_or_404() selected_user = selected_user.followers.all() return render_template('userListFollowers.html', listFollowers=selected_user)

    this is my userListFollowers.html template that displays the followers list {%extends "layout.html" %} {% block body %} Followers {% for users in listFollowers%} {{users.username}} {% endfor %} {% endblock %}

    But nothing gets displayed here

    Also after I read the block related to followed_posts functionality I tried doing this in my models.py file

    class User ..... ....... def follower(self): follr = User.query.join(followers, (followers.c.followed_id == User.id)).filter( followers.c.follower_id == self.id) return follr

    and then changed the routes.py file to this def listFollowers(username): selected_user = User.query.filter_by(username=username).first_or_404() return render_template('userListFollowers.html', listFollowers=selected_user.follower())

    but still didn't get the required result nothing was being displayed Can you help me with this?

  • #229 Miguel Grinberg said 2020-04-24T14:18:07Z

    @Joshua: the error means that the database does not have the tables created. If you failed to configure your tests to use their own database, then it is possible that the tests used your main development database, and at the end the tables were removed during test cleanup. Delete the sqlite file, then run "flask db upgrade" and then you will have a brand new database ready to use during development. Also fix your tests to change the DATABASE_URL, so that this does not happen the next time you run the tests.

  • #230 Miguel Grinberg said 2020-04-24T14:24:38Z

    @Glen: this is much easier for you to debug than me, because I do not have the code. Print the result of the query to see if that part works or not. I noticed you are not issuing an all() call to resolve the query, maybe that's the problem, but as I said, testing is the easiest and fastest way to figure this out.

  • #231 Miguel Grinberg said 2020-04-24T14:26:24Z

    @Mostafa: Yes, you could make these POST requests to make these options more secure.

  • #232 Xingran said 2020-04-27T20:08:45Z

    Hi, thank you for this article. I am using mysql database, for unittest, how can I setup database on mysql? app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://' is not working.

  • #233 Miguel Grinberg said 2020-04-27T21:58:49Z

    @Xingran: mysql does not have an in-memory database like sqlite does. For MySQL you have to use a real database and configure the URL exactly like you do for the development or production databases.

  • #234 ed said 2020-05-06T19:06:26Z

    I have multiple schemas and tried doing this (the followers and users same schema) and I get sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'follower s.follower_id' could not find table 'users' with which to generate a foreign key to target column 'id'

    followers = db.Table('followers', db.Model.metadata, db.Column('follower_id', db.Integer, db.ForeignKey('users.id')), db.Column('followed_id', db.Integer, db.ForeignKey('users.id')), schema='avengers_user', ) class User(UserMixin, db.Model): __tablename__ = 'users' __bind_key__ = 'avengers' __table_args__ = {"schema": "avengers_user"} id = db.Column(db.Integer, autoincrement=True, primary_key=True, unique=True) followed = db.relationship( 'User', secondary=followers, primaryjoin=(followers.c.follower_id == id), secondaryjoin=(followers.c.followed_id == id), backref=db.backref('followers', lazy='dynamic'), lazy='dynamic')
  • #235 Miguel Grinberg said 2020-05-07T11:29:36Z

    @ed: have you tried specifying the schema when you refer to a table? How would SQLAlchemy know which schema if not told? Maybe db.ForeignKey('avengers_user.users.id')?

  • #236 Chetan said 2020-05-08T07:49:57Z

    Hey Miguel, was studying flask using your tutorial but when I ran tests.py I got 2 errors

    ``` python tests.py test_avatar (main.UserModelCase) ... ok test_follow (main.UserModelCase) ... ERROR test_follow_posts (main.UserModelCase) ... ERROR test_password_hashing (main.UserModelCase) ... ok

    ====================================================================== ERROR: test_follow (main.UserModelCase)

    Traceback (most recent call last): File "D:\python_projects\MegaFlaskTutorial\Project1\venv\lib\site-package s\sqlalchemy\util_collections.py", line 210, in getattr return self._data[key] KeyError: 'followed_id'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "tests.py", line 38, in test_follow u1.follow(u2) File "D:\python_projects\MegaFlaskTutorial\Project1\app\models.py", line 42, in follow if not self.is_following(user): File "D:\python_projects\MegaFlaskTutorial\Project1\app\models.py", line 51, in is_following followers.c.followed_id == user.id).count() > 0 File "D:\python_projects\MegaFlaskTutorial\Project1\venv\lib\site-package s\sqlalchemy\util_collections.py", line 212, in getattr raise AttributeError(key) AttributeError: followed_id

    ====================================================================== ERROR: test_follow_posts (main.UserModelCase)

    Traceback (most recent call last): File "D:\python_projects\MegaFlaskTutorial\Project1\venv\lib\site-package s\sqlalchemy\util_collections.py", line 210, in getattr return self._data[key] KeyError: 'followed_id'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "tests.py", line 74, in test_follow_posts u1.follow(u2) # john follows susan File "D:\python_projects\MegaFlaskTutorial\Project1\app\models.py", line 42, in follow if not self.is_following(user): File "D:\python_projects\MegaFlaskTutorial\Project1\app\models.py", line 51, in is_following followers.c.followed_id == user.id).count() > 0 File "D:\python_projects\MegaFlaskTutorial\Project1\venv\lib\site-package s\sqlalchemy\util_collections.py", line 212, in getattr raise AttributeError(key) AttributeError: followed_id

    Ran 4 tests in 0.910s

    FAILED (errors=2)

    ```

  • #237 Miguel Grinberg said 2020-05-08T22:52:25Z

    @Chetan: review your model definitions, I believe the tests detected a problem, you do not have the models defined correctly.

  • #238 Ben Minahan said 2020-05-09T15:42:16Z

    Miguel, thank you for putting this series together, I have learned so much thus far!

    One design choice I am curious about: shouldn't the follow and unfollow be POST requests since it's manipulating data? I noticed there's not a way in HTML spec to post on anchor tags, so was wondering how you might achieve this or if there was a reason I'm not understanding to do GETs.

    Thanks!

  • #239 Liam Zhou said 2020-05-13T03:48:30Z

    Hey Miguel, thank you for such an awesome tutorial which is free. It benefits me so much, so I bought your course just to support you.

    Now I'm doing my own project and I'm running into a problem in unit testing. I want to insert some data to the test database before each test, but I don't know what's the most elegant way to do that. Is it possible to let db object executes raw sql commands from a .sql file? Or you have any other suggestions?

  • #240 Miguel Grinberg said 2020-05-14T10:03:12Z

    @Liam: If you already work with raw SQL in your app, then I see no problem in also doing so for the tests. If your app only uses models, I would personally work in the same way for the tests. You can create an auxiliary function or method accessible to your tests that creates this test data. Or if you want the data inserted into every test then you can add it in the setUp() method for your test case.

  • #241 Luciano Pastine said 2020-05-19T16:41:52Z

    Congratulations for this work Miguel, still in 2020, I believe is one of the best resources to learn Flask. Thank you so much!

  • #242 Daniel De Leon said 2020-05-20T13:26:15Z

    Hey there! Is this tutorial the same as the book? I just purchased the book not too long ago and find that the code looks much better on here as opposed to the kindle version. I just want to know so that I can just follow this instead of the book. Thanks!

  • #243 Miguel Grinberg said 2020-05-20T15:16:25Z

    @Daniel: I wrote two Flask books, which one do you have? The tutorial in this blog matches the book titled "The New and Improved Flask Mega-Tutorial".

  • #244 Adi said 2020-05-21T21:16:02Z

    For getting posts by all followed users, would it not be more efficient to first filter out all the users that the particular user follows and then apply the join to the resulting table. i.e., filter(followers.c.follower_id == self.id) then join this with the post table.

  • #245 Miguel Grinberg said 2020-05-22T08:35:52Z

    @Adi: only way to know is to try it on a specific database to see how the query is executed. In my experience, the order in which you provide your filters and joins does not matter, the database always optimizes to do the least amount of work.

  • #246 Vinícius Melo said 2020-05-28T00:46:59Z

    Hi Miguel, and someone who i hope will be reading this, i encounter a problem with the last version of SQLAlchemy when creating the relationship, after a few hours searching for the solution, i found that it is necessary to change the primary and secondary join to a string based expression. Hope that this will help someone in the future, i love this tutorial and your work, congrats man, keep going.

    The solution:

    followed = db.relationship( 'User', secondary=followers, primaryjoin=("followers.c.follower_id == User.id"), secondaryjoin=("followers.c.followed_id == User.id"), backref="followers", )

  • #247 Miguel Grinberg said 2020-05-28T10:18:46Z

    @Vinícius: what is the error that you get? I have retested with the latest SQLAlchemy release and found everything to work as before.

  • #248 shafayat said 2020-05-29T10:50:50Z

    I keep getting the following problem when trying to update the database to add followers. I don't understand what to do.

    INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. ERROR [root] Error: Can't locate revision identified by '1d40f7482134'

  • #249 Miguel Grinberg said 2020-05-29T21:40:33Z

    @shafayat: this happens when you delete a migration script that is in use. If you know the state in which the database is in, you can use the flask db stamp <revision> command to set the correct migration in the database. If you don't know the state the database is in and you don't have anything important stored in it, the easiest solution is to delete the database and then do a flask db upgrade to make a new one.

  • #250 Rostislav said 2020-05-31T14:30:44Z

    This has been a super helpful series for me. I've only been learning how to code for basically a few months and am trying to build a small prototype for a startup idea I have. I keep going back and forth between continuing with the tutorial and then modifying it and applying it to my own context.

    I've made it this far and have run into some trouble with the tests.py portion. 2/4 of my tests fail because it says "no such table user" the app itself works fine with flask run.

    admittedly my code isn't identical to yours because I also implemented google login, but again it all works live, but fails in the tests.

    code says it fails at db.commit()

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: user [SQL: INSERT INTO user (googid, username, email, password_hash, about_me, last_seen) VALUES (?, ?, ?, ?, ?, ?)] [parameters: (None, 'john', 'john@example.com', None, None, '2020-05-31 14:15:05.917427')]

    I've checked the code in the github link against mine and can't seem to find the issue.

    Any idea what might be the cause?

Leave a Comment