Apr 21st, 2021

Ultimate Guide to Securely Deploy Django at Scale on AWS ECS [Part 2]

#DjangoAWSDeployment

#AWSElasticContainerService

#AmazonElasticContainerRegistry

NOTE: This is the second part of a blog series.

Part 1: Coding the XKCD Django App

Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache

Part 3: AWS: Deploying XKCD App to Elastic Container Service

Overview

In Part 1 of this blog series, we’ve created and dockerized the XKCD App. In this blog post, we will continue from where we left off. We have a dockerized Django application that we want to deploy to ECS. For our database needs, we will be creating a PostgreSQL RDS database and connecting our application to it.

Then, we will move our application secrets - including the RDS connection credentials - to AWS Systems Manager Parameter Store to ensure security.

Finally, we will move our session storage to ElastiCache Redis. This change will make the XKCD application more performant under heavy load. Django uses the database as the session storage by default, so this step is optional. Please, sign in to AWS Console and we can start creating the RDS first.

Index

Part 1: Coding the XKCD Django App

Coding the XKCD Django App

  1. Creating and activating a virtual environment
  2. Installing Django
  3. Creating database models
  4. Adding models to Django admin page
  5. Creating the homepage view
  6. Adding homepage view to URLs
  7. Creating homepage.html
  8. Creating requirements.txt file
  9. Dockerizing our Django App
    1. Testing the Docker Image

Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache

Configuring AWS

  1. AWS RDS - Relational Database Service
    1. Configuring RDS Security Group
    2. Creating a Postgresql Database on RDS
    3. Updating Django settings to use the PostgreSQL Database Backend
      1. Install the PostgreSQL library
      2. Update the settings.py
  2. AWS Systems Manager Parameter Store
    1. Adding our secrets to Parameter Store
    2. Configuring Django App to use AWS Parameter Store
      1. Install AWS SDK for Python: boto3
      2. Update the settings.py file
    3. Migrating Django models to RDS instance
    4. Build & run the docker image with the aws credentials
    5. Creating Parameter Store IAM Role
    6. Creating a super user
  3. ElastiCache Redis
    1. Creating the Security Group: XKCDAppElastiCacheSecurityGroup
    2. Creating the ElastiCache Redis Instance
    3. Adding ElastiCache endpoint to Parameter Store
    4. Installing django-redis package
    5. Updating Django settings to use Redis as Session Storage

Part 3: AWS: Deploying XKCD App to Elastic Container Service

  1. Elastic Container Registry
    1. Uploading XKCD Apps Docker Image to ECR
  2. Elastic Load Balancing
    1. ELB Security Group Creation
    2. ELB Creation
      1. Step 1: Configure Load Balancer
      2. Step 2: Security Settings
      3. Step 3: Security Groups
      4. Step 4: Routing: Target Group Creation
      5. Step 5: Registering Targets to Target Group
      6. Step 6: Review
      7. Step 7: Forward traffic from port 80 to port 8000
  3. Elastic Container Service
    1. Creating an ECS Task Execution Role: XKCDAppECSTaskExecutionRole
    2. Creating ECS security group: XKCDAppECSSecurityGroup
    3. Creating Task Definition: XKCDAppTaskDefinition
      1. Adding a container to XKCDAppTaskDefinition
    4. Creating Cluster Service
      1. Step 1: Configure Service
      2. Step 2: Configure Network
      3. Step 3: Set Auto Scaling
      4. Step 4: Review
    5. Load Testing our App with Hey
    6. Creating Auto Scaling for XKCDAppClusterService
      1. Testing the Auto Scaling Policy
  4. Updating Security Groups
    1. Updating XKCDAppElasticLoadBalancerSecurityGroup
    2. Updating XKCDAppECSSecurityGroup
    3. Updating XKCDAppElastiCacheSecurityGroup
    4. Updating XKCDAppRDSSecurityGroup

 

Configuring AWS

1. AWS RDS - Relational Database Service

1.1 Configuring RDS Security Group

On AWS Console, go to Security Groups under the VPC service. Press Create Security Group button.

 Image shows RDS Security Group Creation on AWS Console

RDS Security Group Creation

 

Setting Option
Name xkcdAppRDSSecurityGroup
Description Allows access on PostgreSQL port: 5432.
VPC Select your VPC, or use the default one.
Inbound Rules Type: PostgreSQL, Source: Anywhere
Outbound Rules Type: All Trafic, Destination: Anywhere

 

1.2 Creating a Postgresql Database on RDS

On AWS Console, go to RDS and start to create a new RDS instance. RDS creation settings below uses default VPC and Free-tier. Feel free to change the configuration according to your requirements.

 

Table of Configuration

Setting Option Detail
Creation Method Standart create
Engine Option PostgreSQL
Engine Version 12.5-R1 Specify your DB engine version.
Templates Free Tier If you are building for prod or test envs, choose respectively.
DB Instance Identifier xkcdappdb
Master Username postgres
Master Password < redacted >
DB Instance Class db.t2.micro you can upgrade the instance class to your needs.
Storage Type General Purpose(SSD)
Allocated Storage 20GB
Enable Storage Autoscaling False You can enable this if you need it.
Multi-AZ Deployment False This feature not included in the free-tier. Production environments would benefit from this feature.
VPC Default VPC Ideally you should create your own VPC and choose that.
Subnet Group default
Public Access Yes We will turn this feature off later. We need public access only in the first setup to test things.
VPC Security Group Delete default and choose existing: xkcdAppRDSSecurityGroup
Availability Zone eu-west-2a Choose one AZ if you are on free-tier.
Database Authentication Password Authentication
Initial Database Name xkcdtestdb Initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance. If you fail to fill this field, you will have to manually create the database.

 

Database Engine

Image shows Choosing Database Engine panel and  PostgreSQL is selected

Choosing Database Engine: PostgreSQL

RDS Creation Template

Selecting Template panel shows three options and Standard(free tier) is selected

Selecting Template: Standard

Database Settings and Credentials

Database settings and Credentials are shown on  Selecting Template panel: Standard

Selecting Template: Standard

Database Instance Class

Image shows Database Instance Class which is t2.micro

Database Instance Class: t2.micro

Database Storage

Image shows Database Storage settings and 20GB SSD is selected

Database Storage: 20GB SSD

Connectivity

Be sure to select Public Access: Yes. We will need to access the database publicly in the first setup, but we will turn it off later.

:Image shows Connectivity panel

Connectivity panel

Database Authentication

Database Authentication: Password authentication

Database Authentication: Password authentication

Additional Configuration

NOTE: Additional Configuration: Initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance you are creating. If you fail to fill this field, you will have to manually create the database.

Image shows Additional Configuration on AWS Console

Additional Configuration

Acquiring the Database Connection Endpoint

Wait for the database creation to be complete. Go to xkcdappdb on RDS > Databases and then go to Connectivity & Security tab.

1.3 Updating Django settings to use the PostgreSQL Database Backend

1.3.1 Install the PostgreSQL library

# be sure to activate the venv environment
pip install psycopg2-binary

1.3.2 Update the settings.py

#/xkcd_app/xkcd_app/settings.py
import os

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DATABASE_NAME'),
        'USER': os.environ.get('DATABASE_USER'),
        'PASSWORD': os.environ.get('DATABASE_PASSWORD'),
        'HOST': os.environ.get('DATABASE_HOST'),
        'PORT': '5432'
    }
}

Using environment variables for application configuration is not a secure way of doing things. Integrating AWS Parameter Store to our settings.py file is something easy. Let's configure AWS Systems Manager Parameter Store for XKCD App.

2. AWS Systems Manager Parameter Store

Go to Parameter Store under AWS Systems Manager and click on Create Parameter.

Parameter Store lets you create categorized paramater names, so you can have different parameters for different apps or app environments.

2.1 Adding our secrets to Parameter Store

For DATABASE_NAME parameter, you can create a parameter as following:

Image shows AWS Parameter Store adding DATABASE_NAME as a parameter

AWS Parameter Store adding DATABASE_NAME as a parameter

 

Setting Option Detail
Name xkcdapp/test/DATABASE_NAME app and environment categorized parameter name
Description xkcd app RDS test DB identifier
Tier Standard
Type SecureString This type of parameters are going to be encrypted while sitting.
KMS key source My Current Account
KMS Key ID alias/aws/ssm Choose the default AWS managed key. This will allow ECS containers to access to parameters without further configuration.
Value xkcddbtestdb Type the database name you gave in the RDS Creation: Additional Configuration step.

 

Follow the same steps to add the remaining database connection credentials to paramater store. You may also want to add SECRET_KEY variable generated for your Django project. Django's SECRET_KEY is used for encryption jobs inside Django.

When you are finished with adding the secrets, you should see something like this on Parameter Store Console.

image shows my parameters section in Parameter Store Console

Parameter Store Console

You can check the existence of parameters with AWS CLI, if you have it set up.

aws ssm get-parameters --name "/xkcdapp/test/DATABASE_HOST" --with-decryption --region <your_region> --profile <your_aws_cli_profile>

 

2.2 Configuring Django App to use AWS Parameter Store

2.2.1 Install AWS SDK for Python: boto3

# be sure to be in the venv virtual environment
pip install boto3

2.2.2 Update the settings.py file

# xkcd_app/xkcd_app/settings.py
import boto3
ssm = boto3.client('ssm', region_name='eu-west-2')

prefix = 'xkcdapp'
env = 'test'
prefixenv = f"/{prefix}/{env}/"

SECRET_KEY = ssm.get_parameter(Name=prefixenv + "DJANGO_SECRET_KEY", WithDecryption=True)['Parameter']['Value'] 

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': ssm.get_parameter(Name=prefixenv + "DATABASE_NAME", WithDecryption=True)['Parameter']['Value'],
        'USER': ssm.get_parameter(Name=prefixenv + "DATABASE_USER", WithDecryption=True)['Parameter']['Value'],
        'PASSWORD': ssm.get_parameter(Name=prefixenv + "DATABASE_PASSWORD", WithDecryption=True)['Parameter']['Value'],
        'HOST': ssm.get_parameter(Name=prefixenv + "DATABASE_HOST", WithDecryption=True)['Parameter']['Value'],
        'PORT': '5432'
    }
}

 

2.3 Migrating Django models to RDS instance

NOTE: Be sure that your AWS Accounts credentials are set in the AWS CLI's credentials file.

Migrate our migrations to AWS RDS.

python manage.py migrate

# Operations to perform:
#   Apply all migrations: admin, auth, contenttypes, sessions, xkcd_app
# Running migrations:
#   Applying contenttypes.0001_initial... OK
#   Applying auth.0001_initial... OK
#   .
#   .
#   Applying sessions.0001_initial... OK
#   Applying xkcd_app.0001_initial... OK

 

2.4 Build & run the docker image with the aws credentials

If you want to build and test the docker image with the application secrets on the AWS Parameter Store:

  1. build the image again to apply the settings.py file changes to docker image.
  2. run the image with AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY environment variables set.

    # build the image again
    docker build -t xkcd:latest .  
    # run with AWS credentials set in the environment variables.
    docker run -p 8000:8000 -e AWS_ACCESS_KEY_ID='XXXXXXXXXXXX' -e AWS_SECRET_ACCESS_KEY='YYYYYYYYYYY' xkcd:latest
    

We won't need to send the AWS Credentials to our Docker image when we deploy on the Elastic Container Service, as the AWS automatically sets those environment variables upon instance creation.

 

2.5 Creating Parameter Store IAM Role

Go to Roles under AWS IAM and click on Create Role.

Select the System Manager use case.

Image shows that AWS IAM Role Creation System Manager use case is selected in case selection options

AWS IAM Role Creation System Manager use case

Click on Next: Permissions and then Create Policy.

 image shows AWS IAM Role Creation Inline Policy in  JSON data format

AWS IAM Role Creation Inline Policy JSON

Copy and paste the following AWS IAM Policy to editor and hit next.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSSMAccess",
            "Effect": "Allow",
            "Action": [
                "ssm:PutParameter",
                "ssm:DeleteParameter",
                "ssm:GetParameterHistory",
                "ssm:GetParametersByPath",
                "ssm:GetParameters",
                "ssm:GetParameter",
                "ssm:DescribeParameters",
                "ssm:DeleteParameters"
            ],
            "Resource": "arn:aws:ssm:*:*:parameter/*"
        }
    ]
}

Give a name and description to Parameter Store access role.

image shows the AWS IAM Role Creation Inline Policy review page

AWS IAM Role Creation Inline Policy

 

Setting Option
Name SystemsManagerParameterStoreFullAccess
Description Systems Manager Parameter Store Full Access Role.

We will be attaching this IAM Role to ECS containers we are going to run to allow them to read the parameters.

 

2.6 Creating a super user

When you are connected to your RDS instance, you can create an admin user with the following command.

python3 manage.py createsuperuser

3. ElastiCache Redis

3.1 Creating the Security Group: XKCDAppElastiCacheSecurityGroup

Image shows the creation of ElastiCache Redis Security Group

Elasti Cache Redis Security Group Creation

 

Setting Option
Name XKCDAppElastiCacheSecurityGroup
Description XKCD Apps elasti cache security group. Allows access from port: 6379
VPC Select your VPC, or use the default one.
Inbound Rules Type: CustomTCP, Port Range: 6379, Source: Anywhere
Outbound Rules Type: All Trafic, Destination: Anywhere

 

3.2 Creating the ElastiCache Redis Instance

Go to Redis under ElastiCache and press create button.

 

Table of Configuration

Setting Option
Cluster Engine Redis, Cluster Mode Disabled
Location Amazon Cloud
Name xkcdappredis
Description Session storage for XKCD App
Engine version compatibility 6.x
Port 6379
Parameter group default.redis6.x
Node type cache.t2.micro (0.5 GiB)
Number of replicas 0
Multi-AZ False
Subnet Group Create new
Subnet Group Name ecachesubnetgroup
Subnet Group Description Subnet group for XKCD apps ElastiCache Redis Storage.
Subnet Group VPC ID Default VPC ID
Subnet Group Subnets 2c, 2a, 2b
Subnet Group AZ Placement No preference
Security Group XKCDAppElastiCacheSecurityGroup
Encryption-at-Rest True
Encryption Key Default
Encryption in transit True
Access Control Option No Access

 

Engine and Location

ElastiCache: Engine and Location

ElastiCache: Engine and Location

Redis Settings

image shows redis settings for creating an ElastiCache instance

ElastiCache: Redis Settings

Advanced Redis Settings

image shows  Advanced Redis Settings for ElastiCache

ElastiCache: Advanced Redis Settings

Security

ElastiCache: Security

ElastiCache: Security

 

3.3 Adding ElastiCache endpoint to Parameter Store

Go to Parameter Store under AWS Systems Manager and add ELASTICACHE_ENDPOINT without port :6379 to parameter store.

 

Setting Option
Name xkcdapp/test/ELASTICACHE_ENDPOINT
Description xkcd app ElastiCache Redis Endpoint
Tier Standard
Type SecureString
KMS key source My Current Account
KMS Key ID alias/aws/ssm
Value xkcdappredis.xxxxx.yyyy.rrrr.cache.amazonaws.com

 

3.4 Installing Django-Redis package

# be sure to be in venv virtuall environment
pip install django-redis

 

3.5 Updating Django settings to use Redis as Session Storage

# xkcd_app/xkcd_app/settings.py
# ELASTICACHE_ENDPOINT without the port 
ELASTICACHE_ENDPOINT = ssm.get_parameter(Name=prefixenv + "ELASTICACHE_ENDPOINT", WithDecryption=True)['Parameter']['Value']
CACHE_LOCATION =  f"redis://{ELASTICACHE_ENDPOINT}/0"
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": CACHE_LOCATION,
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        }
    }
}
#Use the redis as Session storage as well.
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"

 

Conclusion

In this 2nd part of our Django blog series, we’ve created an RDS database, a Redis ElastiCache instance, and their security groups. We’ve also configured the Django application to use these services and updated our docker image. Our application secrets are securely stored on the Parameter Store. The only thing left to do is deploy our containers to Elastic Container Service.

In the next and last post, Part 3: AWS: Deploying XKCD App to Elastic Container Service, of this series, we will:

  • upload our final container image to Elastic Container Registry,
  • create a Task Definition to deploy our docker image to Elastic Container Service,
  • create a Load Balancer to distribute the traffic to multiple ECS containers,
  • set up the ECS Auto Scaling Policy,
  • configure the security groups to only accept traffic from other security groups we created.

If you liked this post, share it now!

Our Recent Posts

How to Secure Your Docker Containers: Tips and Challenges

Discover Docker technology, learn about Docker security best practices and Docker vulnerability...

Read More

Ultimate Guide to Securely Deploy Django at Scale on AWS ECS [Part 3]

Learn how to securely deploy a dockerized Django application to AWS Elastic Container Service w...

Read More

Ultimate Guide to Securely Deploy Django at Scale on AWS ECS [Part 2]

Learn how to securely deploy a dockerized Django application to AWS Elastic Container Service w...

Read More