Ultimate Guide to Securely Deploy Django at Scale on AWS ECS [Part 1]

April 14, 2021



NOTE: This is the first part of a blog series.

Part 1: Coding the XKCD Django App

Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache

Part 3: AWS: Deploying XKCD App to Elastic Container Service

Overview

In this blog post, we will create a small Django application and dockerize it. Then we will be using the docker image to deploy the application to AWS Elastic Container Service with Auto Scaling in Parts 2 and 3 of this blog series.

Django’s sessions will be stored on an AWS ElastiCache Redis server to avoid hitting the database too much. We will also be using AWS Systems Manager Parameter Store to store our application secrets such as database credentials. We will start with coding a sample Django application called XKCD App.

Prerequisites

Index

Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache

Configuring AWS

  1. AWS RDS - Relational Database Service
    1. Configuring RDS Security Group
    2. Creating a Postgresql Database on RDS
    3. Updating Django settings to use the PostgreSQL Database Backend
      1. Install the PostgreSQL library
      2. Update the settings.py
  2. AWS Systems Manager Parameter Store
    1. Adding our secrets to Parameter Store
    2. Configuring Django App to use AWS Parameter Store
      1. Install AWS SDK for Python: boto3
      2. Update the settings.py file
    3. Migrating Django models to RDS instance
    4. Build & run the docker image with the aws credentials
    5. Creating Parameter Store IAM Role
    6. Creating a super user
  3. ElastiCache Redis
    1. Creating the Security Group: XKCDAppElastiCacheSecurityGroup
    2. Creating the ElastiCache Redis Instance
    3. Adding ElastiCache endpoint to Parameter Store
    4. Installing django-redis package
    5. Updating Django settings to use Redis as Session Storage

Part 3: AWS: Deploying XKCD App to Elastic Container Service

  1. Elastic Container Registry
    1. Uploading XKCD Apps Docker Image to ECR
  2. Elastic Load Balancing
    1. ELB Security Group Creation
    2. ELB Creation
      1. Step 1: Configure Load Balancer
      2. Step 2: Security Settings
      3. Step 3: Security Groups
      4. Step 4: Routing: Target Group Creation
      5. Step 5: Registering Targets to Target Group
      6. Step 6: Review
      7. Step 7: Forward traffic from port 80 to port 8000
  3. Elastic Container Service
    1. Creating an ECS Task Execution Role: XKCDAppECSTaskExecutionRole
    2. Creating ECS security group: XKCDAppECSSecurityGroup
    3. Creating Task Definition: XKCDAppTaskDefinition
      1. Adding a container to XKCDAppTaskDefinition
    4. Creating Cluster Service
      1. Step 1: Configure Service
      2. Step 2: Configure Network
      3. Step 3: Set Auto Scaling
      4. Step 4: Review
    5. Load Testing our App with Hey
    6. Creating Auto Scaling for XKCDAppClusterService
      1. Testing the Auto Scaling Policy
  4. Updating Security Groups
    1. Updating XKCDAppElasticLoadBalancerSecurityGroup
    2. Updating XKCDAppECSSecurityGroup
    3. Updating XKCDAppElastiCacheSecurityGroup
    4. Updating XKCDAppRDSSecurityGroup

About AWS Security Groups & IAM Roles

In this blog post, whenever we are going to create an AWS Service, we will be creating that service’s Security Group or IAM Role beforehand.

All security groups will have fully open inbound/outbound rules at first, and when we are done with the application setup on AWS, we will go and revise the inbound/outbound rules of all security groups.

Coding the XKCD Django App

For this blog’s demo, we can create a very small XKCD Comic Viewer Django application. It’s going to only have a homepage view that will show a random comic using python module XKCD and will have a button for another comic.

It will also update the view count of an XKCD comic in a database model with the comic's ID.

1. Creating and activating a virtual environment

# install the virtualenv package
pip install virtualenv

#create a virtualenv with the name venv
virtualenv venv

# activate virtualenv on windows
cd venv/Scripts
activate

# activate virtualenv on linux
./venv/bin/activate

2. Installing Django

# install django and xkcd python library
pip install django xkcd

# create a django project name xkcd_app
django-admin startproject xkcd_app

# you should get the following files created
xkcd_app
├── manage.py
└── xkcd_app
    ├── asgi.py
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Let's review and change the xkcd_app/settings.py

# /xkcd_app/xkcd_app/settings.py

# allow any host for now
ALLOWED_HOSTS = ['*']

# add our app to installed apps
INSTALLED_APPS = [
    .
    .
    'django.contrib.staticfiles',
    'xkcd_app'
]

3. Creating database models

Let's create our models. First we have to create a file named models.py on our xkcd_app folder.

# /xkcd_app/xkcd_app/models.py
from django.db import models

class XKCDComicViews(models.Model):
    comic_number = models.IntegerField(primary_key=True, unique=True)
    view_count = models.IntegerField(default=0)

Let's migrate XKCDComicViews model to our database. Django will use sqlite3 by default for local development.

# first apply default migrations
python manage.py migrate

# create our django apps migrations
python manage.py makemigrations xkcd_app

# and migrate them
python manage.py migrate xkcd_app

4. Adding models to django admin page

create a admin.py file under xkcd_app/xkcd_app

#xkcd_app/xkcd_app/admin.py
from django.contrib import admin
from .models import XKCDComicViews

admin.site.register(XKCDComicViews)

5. Creating the homepage view

Let's create views.py under our xkcd_app.

# /xkcd_app/xkcd_app/views.py
from django.shortcuts import render
from .models import XKCDComicViews
import xkcd

def get_comic_and_increase_view_count(comic_number, increase_by=1):
    # get or create this with given comic_number
    comic, _ = XKCDComicViews.objects.get_or_create(pk=comic_number)
    comic.view_count += increase_by # increase the view_count
    comic.save() # save it

def homepage(request):
    # get a random comic from xkcd lib.
    random_comic = xkcd.getRandomComic()
    # increase it's view count
    get_comic_and_increase_view_count(random_comic.number, increase_by=1)
    # create a context to render the html with.
    context = {
        number: random_comic.number,
        image_link: random_comic.getImageLink(),
        title: random_comic.getTitle(),
        alt_text: random_comic.getAltText()
    }
    # return rendered html.
    return render(request, 'xkcd_app/homepage.html', context)

6. Adding homepage view to URLs

# /xkcd_app/xkcd_app/urls.py
from django.contrib import admin
from django.urls import path
from .views import homepage
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', homepage, name='homepage')
]

7. Creating homepage.html

<!-- /xkcd_app/xkcd_app/templates/xkcd_app/homepage.html -->
<html>
<head>
<title>XKCD App</title>
<link rel=stylesheet href=https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css integrity=sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z crossorigin=anonymous>
</head>
<body>
  <div class=container>
    <div class=row style=margin-top:15px>
        <div class=col-xs-8 col-sm-10 mx-auto col-md-12>
            <div class=col text-center>
                <h2 class=header >#{{number}} | {{title}}</h2>
            <hr/>
            <img src={{image_link}} class=img-fluid alt={{alt_text}} />
            <hr/>
                <a class=btn btn-default href={% url 'homepage' %}>Another One</a>
               <hr/>
               <a rel=license href=http://creativecommons.org/licenses/by-nc/2.5/><img alt=Creative Commons License style=border:none src=http://creativecommons.org/images/public/somerights20.png></a>
               All credits for comics belongs to <a href=https://xkcd.com/license.html>XKCD.</a></p>
            </div>
        </div>
    </div>
<script src=https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js integrity=sha384-B4gt1jrGC7Jh4AgTPSdUtOBvfO8shuf57BaghqFfPlYxofvL8/KUEfYiJOMMV+rV crossorigin=anonymous></script>
</body>
</html>

8. Creating requirements.txt file

Freeze the python requirements and move the file to root directory of the project. You are expected to regularly update your requirements.txt file.

# cd into the root directory of the project and then run the command below

# be sure to activate the virtual env
pip freeze > requirements.txt

For this project the requirements.txt should look like this:

boto3
xkcd
django
psycopg2-binary
gunicorn
django-redis

9. Dockerizing our Django App

Create a Dockerfile on the root directory of the project.

# /Dockerfile
FROM python:3.8
# PYTHONUNBUFFERED variable is used for non-buffered stdout
ENV PYTHONUNBUFFERED=1
# update the packages
RUN apt update -y && apt upgrade -y

# changing our working directory to be /usr/src
WORKDIR /usr/src/

# copying and installing python requirements
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

# copying the whole django application
COPY xkcd_app/ .

# exposing our django port: 8000
EXPOSE 8000

# serving django with gunicorn on port 8000 (1 worker, timeout of 60 secs)
CMD [gunicorn, --bind, 0.0.0.0:8000, --workers, 1, --timeout, 60, xkcd_app.wsgi]

9.1 testing the Docker Image

Let's build and run our docker image.

# build the image with the latest tag
docker build -t xkcd:latest .

# run the image with port mapping 8000 to 8000
docker run -p 8000:8000 xkcd:latest

# output should look something like this.
# ... [1] [INFO] Starting gunicorn 20.1.0
# ... [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
# ... [1] [INFO] Using worker: sync
# ... [8] [INFO] Booting worker with pid: 8

If everything goes well, you can go to http://0.0.0.0:8000 or http://localhost:8000 to see XKCD app working.

Conclusion

In Part 1: Coding the XKCD Django App of this blog series, we have created a basic Django application and dockerized it. Now, we have a docker image that we want to run at scale using AWS Elastic Container Service. Before we deploy our image to ECS, please join me in Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache of this blog series where we are going to

  • create a RDS database & connect it to Django,
  • move application secrets to Parameter Store,
  • create an ElastiCache Redis server to store django sessions.

Let’s start to configure our AWS environment in Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache.