Creating and populating a non-nullable field in Django

Hey, what’s up guys, that’s another quick post! I’ll show you how to create a new non-nullable field in Django and how to populate it using Django migrations.

SAY WUUUUT?????

Here’s the thing, do you know when you have your website in production and everything set in order and then some guy (there’s always some guy) appears with a new must-have mandatory field that nobody, neither the client nor the PO, no one, thought about it? That’s the situation.

But it happens that you use Django Migrations and you want to add those baby fields and run your migrations back and forth, right?

For this example, I decided to use a random project from the web. I chose this Django Polls on Django 1.10.

So, as usual, clone and create your virtual environment.

git clone [email protected]:garmoncheg/django-polls_1.10.git
cd django-polls_1.10/
mkvirtualenv --python=/usr/bin/python3 django-polls 
pip install django==1.10  # Nesse projeto o autor não criou um requirements.txt
python manage.py migrate  # rodando as migrações existentes
python manage.py createsuperuser 
python manage.py runserver

Note: This project has one missing migration, so if you’re following this step-by-step run python manage.py makemigrations to create the migration 0002 (that’s just a minor change on a verbose_name)

Now, access the admin website and add a poll

Alright, you can go to the app and see your poll there, answer it and whatever. Until now we did nothing.

The idea is to create more questions with different pub_dates to get the party started.

After you use your Polls app a little you’ll notice that any poll stay forever on your website, i.e., you never close it.

So, our update on this project will be that: From now on, all polls will have an expiration date. When the user creates a poll, he/she must enter the expiration date. That’s a non-nullable, mandatory field. For the polls that already exists in our database, we will arbitrarily decide they will have a single month to expire from the publication date.

Before migrations exist, it was done through SQL, you had to add a DateField that allowed NULL, then you’d create a query to populate this field and finally another ALTER TABLE to turn that column into a mandatory field. With the migrations, it works in the same way.

So, let’s add the expires_date field to models.py

expires_date = models.DateTimeField('expires at', null=True)

The whole models:

class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
    expires_date = models.DateTimeField('expires at', null=True)

    def __str__(self):
        return self.question_text

    def was_published_recently(self):
        now = timezone.now()
        return now - datetime.timedelta(days=1) <= self.pub_date <= now
    was_published_recently.admin_order_field = 'pub_date'
    was_published_recently.boolean = True
    was_published_recently.short_description = 'Published recently?'

It’s time to make migrations:

python manage.py makemigrations

This is going to generate the 0003_question_expires_date migration, like this:

class Migration(migrations.Migration):

    dependencies = [
        ('polls', '0002_auto_20170429_2220'),
    ]

    operations = [
        migrations.AddField(
            model_name='question',
            name='expires_date',
            field=models.DateTimeField(null=True, verbose_name='expires at'),
        ),
    ]

Let’s alter this migration’s code, NO PANIC!

Populating the new field

First of all, create a function to populate the database with the expires dates:

def populate_expires_date(apps, schema_editor):
    """
    Populates expire_date fields for polls already on the database.
    """
    from datetime import timedelta

    db_alias = schema_editor.connection.alias
    Question = apps.get_model('polls', 'Question')

    for row in Question.objects.using(db_alias).filter(expires_date__isnull=True):
        row.expires_date = row.pub_date + timedelta(days=30)
        row.save()

Originally, I’ve used this code in a project with multiple databases, so I needed to use db_alias and I think it’s interesting to show it here.

Inside a migration, you’ll find a operations list. On that list, we’ll add the commands to run our populate_expires_date function and after that, we’ll alter this field to make it non-nullable.

operations = [
    migrations.AddField(
        model_name='question',
        name='expires_date',
        field=models.DateTimeField(null=True, verbose_name='expires at'),
    ),
    migrations.RunPython(populate_expires_date, reverse_code=migrations.RunPython.noop),
    migrations.AlterField(
        model_name='question',
        name='expires_date',
        field=models.DateTimeField(verbose_name='expires at'),
    )
]

You can see that we used migrations.RunPython to run our function during the migration. The reverse_code is for cases of unapplying a migration. In this case, the field didn’t exist before, so we’ll do nothing.

Right after, we’ll add the migration to alter the field and turn null=True. We also could have done that just removing that from the model and running makemigrations again. ( Now, we have to remove it from the model, anyway).

models.py

class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
    expires_date = models.DateTimeField('expires at')

    def __str__(self):
        return self.question_text

    def was_published_recently(self):
        now = timezone.now()
        return now - datetime.timedelta(days=1) <= self.pub_date <= now
    was_published_recently.admin_order_field = 'pub_date'
    was_published_recently.boolean = True
    was_published_recently.short_description = 'Published recently?'

And we’re ready to run the migrations

python mange.py migrate

Done! To see this working I’ll add this field to admin.py:

class QuestionAdmin(admin.ModelAdmin):
    fieldsets = [
        (None,               {'fields': ['question_text']}),
        ('Date information', {'fields': ['pub_date', 'expires_date'], 'classes': ['collapse']}),
    ]
    inlines = [ChoiceInline]
    list_display = ('question_text', 'pub_date', 'expires_date', 'was_published_recently')
    list_filter = ['pub_date']
    search_fields = ['question_text']

And voilá, all Questions you had on polls now have an expires_date, mandatory and with 30 days by default for the old ones.

That’s it, the field we wanted! The modified project is here on my GitHub: https://github.com/ffreitasalves/django-polls_1.10

If you like it, share it and leave a comment, if you didn’t, just leave the comment.

XD

 

References:

http://stackoverflow.com/questions/28012340/django-1-7-makemigration-non-nullable-field

http://stackoverflow.com/questions/29217706/django-view-sql-query-without-publishinhg-migrations

https://realpython.com/blog/python/data-migrations/

https://docs.djangoproject.com/en/1.10/howto/writing-migrations/#non-atomic-migrations