This blog series covers the creation and deployment of the XKCD App using Django and AWS Elastic Container Service. In Part 1, we created and dockerized the XKCD App. In this post, we will continue from where we left off by deploying our dockerized Django application to ECS.

We will create a PostgreSQL RDS database for our database needs and connect our application to it. We will also ensure security by moving our application secrets, including the RDS connection credentials, to AWS Systems Manager Parameter Store. Finally, we will move our session storage to ElastiCache Redis to improve the XKCD application’s performance under heavy load.

This step is optional since Django uses the database as the session storage by default.

To get started, please sign in to the AWS Console, and let’s create the RDS.

Index

Part 1: Coding the XKCD Django App

Coding the XKCD Django App

  1. Creating and activating a virtual environment
  2. Installing Django
  3. Creating database models
  4. Adding models to the Django admin page
  5. Creating the homepage view
  6. Adding homepage view to URLs
  7. Creating homepage.html
  8. Creating requirements.txt file
  9. Dockerizing our Django App
    1. Testing the Docker Image

Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache

Configuring AWS

  1. AWS RDS – Relational Database Service
    1. Configuring RDS Security Group
    2. Creating a Postgresql Database on RDS
    3. Updating Django settings to use the PostgreSQL Database Backend
      1. Install the PostgreSQL library
      2. Update the settings.py
  2. AWS Systems Manager Parameter Store
    1. Adding our secrets to Parameter Store
    2. Configuring Django App to use AWS Parameter Store
      1. Install AWS SDK for Python: boto3
      2. Update the settings.py file
    3. Migrating Django models to RDS instance
    4. Build & run the docker image with the AWS credentials
    5. Creating Parameter Store IAM Role
    6. Creating a super user
  3. ElastiCache Redis
    1. Creating the Security Group: XKCDAppElastiCacheSecurityGroup
    2. Creating the ElastiCache Redis Instance
    3. Adding ElastiCache endpoint to Parameter Store
    4. Installing django-redis package
    5. Updating Django settings to use Redis as Session Storage

Part 3: AWS: Deploying XKCD App to Elastic Container Service

  1. Elastic Container Registry
    1. Uploading XKCD Apps Docker Image to ECR
  2. Elastic Load Balancing
    1. ELB Security Group Creation
    2. ELB Creation
      1. Step 1: Configure Load Balancer
      2. Step 2: Security Settings
      3. Step 3: Security Groups
      4. Step 4: Routing: Target Group Creation
      5. Step 5: Registering Targets to Target Group
      6. Step 6: Review
      7. Step 7: Forward traffic from port 80 to port 8000
  3. Elastic Container Service
    1. Creating an ECS Task Execution Role: XKCDAppECSTaskExecutionRole
    2. Creating ECS security group: XKCDAppECSSecurityGroup
    3. Creating Task Definition: XKCDAppTaskDefinition
      1. Adding a container to XKCDAppTaskDefinition
    4. Creating Cluster Service
      1. Step 1: Configure Service
      2. Step 2: Configure Network
      3. Step 3: Set Auto Scaling
      4. Step 4: Review
    5. Load Testing our App with Hey
    6. Creating Auto Scaling for XKCDAppClusterService
      1. Testing the Auto Scaling Policy
  4. Updating Security Groups
    1. Updating XKCDAppElasticLoadBalancerSecurityGroup
    2. Updating XKCDAppECSSecurityGroup
    3. Updating XKCDAppElastiCacheSecurityGroup
    4. Updating XKCDAppRDSSecurityGroup

Configuring AWS

1. AWS RDS – Relational Database Service

1.1 Configuring RDS Security Group

On AWS Console, go to Security Groups under the VPC service. Press Create Security Group button.

Configuring RDS Security Group

RDS Security Group Creation

Setting Option
Name xkcdAppRDSSecurityGroup
Description Allows access on PostgreSQL port: 5432.
VPC Select your VPC, or use the default one.
Inbound Rules Type: PostgreSQL, Source: Anywhere
Outbound Rules Type: All Trafic, Destination: Anywhere

1.2 Creating a Postgresql Database on RDS

On AWS Console, go to RDS and start to create a new RDS instance. RDS creation settings below use default VPC and Free-tier. Feel free to change the configuration according to your requirements.

Table of Configuration

Setting Option Detail
Creation Method Standard create
Engine Option PostgreSQL
Engine Version 12.5-R1 Specify your DB engine version.
Templates Free Tier If you are building for prod or test envs, choose respectively.
DB Instance Identifier xkcdappdb
Master Username postgres
Master Password < redacted >
DB Instance Class db.t2.micro you can upgrade the instance class to your needs.
Storage Type General Purpose(SSD)
Allocated Storage 20GB
Enable Storage Autoscaling False You can enable this if you need it.
Multi-AZ Deployment False This feature is not included in the free-tier. Production environments would benefit from this feature.
VPC Default VPC Ideally, you should create your own VPC and choose that.
Subnet Group default
Public Access Yes We will turn this feature off later. We need public access only in the first setup to test things.
VPC Security Group Delete default and choose existing: xkcdAppRDSSecurityGroup
Availability Zone eu-west-2a Choose one AZ if you are on free-tier.
Database Authentication Password Authentication
Initial Database Name xkcd_test_db The initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance. If you fail to fill this field, you will have to manually create the database.

Database Engine

Database Engine

Choosing Database Engine: PostgreSQL

RDS Creation Template

RDS Creation Template

Selecting Template: Standard

Database Settings and Credentials

Database Settings and Credentials

Selecting Template: Standard

Database Instance Class

Database Instance Class

Database Instance Class: t2.micro

Database Storage

Database Storage

Database Storage: 20GB SSD

Connectivity

Be sure to select Public Access: Yes. We will need to access the database publicly in the first setup, but we will turn it off later.

Connectivity

Connectivity panel

Database Authentication

Database Authentication

Database Authentication: Password authentication

Additional Configuration

NOTE: Additional Configuration: Initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance you are creating. If you fail to fill this field, you will have to manually create the database.

Additional Configuration

Additional Configuration

Acquiring the Database Connection Endpoint

Wait for the database creation to be complete. Go to xkcdappdb on RDS > Databases and then go to Connectivity & Security tab.

1.3 Updating Django settings to use the PostgreSQL Database Backend

1.3.1 Install the PostgreSQL library

# be sure to activate the venv environment
pip install psycopg2-binary

1.3.2 Update the settings.py

#/xkcd_app/xkcd_app/settings.py
import os

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DATABASE_NAME'),
        'USER': os.environ.get('DATABASE_USER'),
        'PASSWORD': os.environ.get('DATABASE_PASSWORD'),
        'HOST': os.environ.get('DATABASE_HOST'),
        'PORT': '5432'
    }
}

Using environment variables for application configuration is not a secure way of doing things. Integrating AWS Parameter Store to our settings.py file is something easy. Let’s configure AWS Systems Manager Parameter Store for XKCD App.

2. AWS Systems Manager Parameter Store

Go to Parameter Store under AWS Systems Manager and click on Create Parameter.

Parameter Store lets you create categorized parameter names, so you can have different parameters for different apps or app environments.

2.1 Adding our secrets to Parameter Store

For the DATABASE_NAME parameter, you can create a parameter as following:

Adding our secrets to Parameter Store

AWS Parameter Store adding DATABASE_NAME as a parameter

Setting Option Detail
Name xkcdapp/test/DATABASE_NAME app and environment categorized parameter name
Description xkcd app RDS test DB identifier
Tier Standard
Type SecureString This type of parameters are going to be encrypted while sitting.
KMS key source My Current Account
KMS Key ID alias/aws/ssm Choose the default AWS managed key. This will allow ECS containers to access parameters without further configuration.
Value xkcddb_test_db Type the database name you gave in the RDS Creation: Additional Configuration step.

Follow the same steps to add the remaining database connection credentials to the parameter store. You may also want to add SECRET_KEY a variable generated for your Django project. Django’s SECRET_KEY is used for encryption jobs inside Django.

When you are finished with adding the secrets, you should see something like this on Parameter Store Console.

Adding our secrets to Parameter Store

Parameter Store Console

You can check the existence of parameters with AWS CLI if you have it set up.

aws ssm get-parameters --name /xkcdapp/test/DATABASE_HOST --with-decryption --region <your_region> --profile <your_aws_cli_profile>

2.2 Configuring Django App to Use AWS Parameter Store

2.2.1 Install AWS SDK for Python: boto3

# be sure to be in the venv virtual environment
pip install boto3

2.2.2 Update the settings.py file

# xkcd_app/xkcd_app/settings.py
import boto3
ssm = boto3.client('ssm', region_name='eu-west-2')

prefix = 'xkcdapp'
env = 'test'
prefixenv = f/{prefix}/{env}/

SECRET_KEY = ssm.get_parameter(Name=prefixenv + DJANGO_SECRET_KEY, WithDecryption=True)['Parameter']['Value']

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': ssm.get_parameter(Name=prefixenv + DATABASE_NAME, WithDecryption=True)['Parameter']['Value'],
        'USER': ssm.get_parameter(Name=prefixenv + DATABASE_USER, WithDecryption=True)['Parameter']['Value'],
        'PASSWORD': ssm.get_parameter(Name=prefixenv + DATABASE_PASSWORD, WithDecryption=True)['Parameter']['Value'],
        'HOST': ssm.get_parameter(Name=prefixenv + DATABASE_HOST, WithDecryption=True)['Parameter']['Value'],
        'PORT': '5432'
    }
}

2.3 Migrating Django models to RDS instance

NOTE: Be sure that your AWS Accounts credentials are set in the AWS CLI’s credentials file.

Migrate our migrations to AWS RDS.

python manage.py migrate

# Operations to perform:
#   Apply all migrations: admin, auth, contenttypes, sessions, xkcd_app
# Running migrations:
#   Applying contenttypes.0001_initial... OK
#   Applying auth.0001_initial... OK
#   .
#   .
#   Applying sessions.0001_initial... OK
#   Applying xkcd_app.0001_initial... OK

2.4 Build & run the docker image with the AWS credentials

If you want to build and test the docker image with the application secrets on the AWS Parameter Store:

  1. build the image again to apply the settings.py file changes to the Docker image.
  2. run the image with AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEY environment variables set.
# build the image again
docker build -t xkcd:latest .
# run with AWS credentials set in the environment variables.
docker run -p 8000:8000 -e AWS_ACCESS_KEY_ID='XXXXXXXXXXXX' -e AWS_SECRET_ACCESS_KEY='YYYYYYYYYYY' xkcd:latest

We won’t need to send the AWS Credentials to our Docker image when we deploy on the Elastic Container Service, as AWS automatically sets those environment variables upon instance creation.

2.5 Creating Parameter Store IAM Role

Go to Roles under AWS IAM and click on Create Role.

Select the System Manager use case.

Creating Parameter Store IAM Role

AWS IAM Role Creation System Manager use case

Click on Next: Permissions and then Create Policy.

Creating Parameter Store IAM Role

AWS IAM Role Creation Inline Policy JSON

Copy and paste the following AWS IAM Policy to the editor and hit next.

{
  Version: 2012-10-17,
  Statement: [
    {
      Sid: AllowSSMAccess,
      Effect: Allow,
      Action: [
        ssm:PutParameter,
        ssm:DeleteParameter,
        ssm:GetParameterHistory,
        ssm:GetParametersByPath,
        ssm:GetParameters,
        ssm:GetParameter,
        ssm:DescribeParameters,
        ssm:DeleteParameters
      ],
      Resource: arn:aws:ssm:*:*:parameter/*
    }
  ]
}

Give a name and description to the Parameter Store access role.

Creating Parameter Store IAM Role

AWS IAM Role Creation Inline Policy

Setting Option
Name SystemsManagerParameterStoreFullAccess
Description Systems Manager Parameter Store Full Access Role.

We will be attaching this IAM Role to ECS containers we are going to run to allow them to read the parameters.

2.6 Creating a super user

When you are connected to your RDS instance, you can create an admin user with the following command.

python3 manage.py createsuperuser

This article may interest you: AWS IAM for Red and Blue Teams

3. ElastiCache Redis

3.1 Creating the Security Group: XKCDAppElastiCacheSecurityGroup

Creating the Security Group

ElastiCache Redis Security Group Creation

Setting Option
Name XKCDAppElastiCacheSecurityGroup
Description XKCD Apps elasti cache security group. Allows access from port: 6379
VPC Select your VPC, or use the default one.
Inbound Rules Type: CustomTCP, Port Range: 6379, Source: Anywhere
Outbound Rules Type: All Trafic, Destination: Anywhere

3.2 Creating the ElastiCache Redis Instance

Go to Redis under ElastiCache and press the “create” button.

Table of Configuration

Setting Option
Cluster Engine Redis, Cluster-Mode Disabled
Location Amazon Cloud
Name xkcdappredis
Description Session storage for XKCD App
Engine version compatibility 6.x
Port 6379
Parameter group default.redis6.x
Node type cache.t2.micro (0.5 GiB)
Number of replicas 0
Multi-AZ False
Subnet Group Create new
Subnet Group Name ecachesubnetgroup
Subnet Group Description Subnet group for XKCD apps ElastiCache Redis Storage.
Subnet Group VPC ID Default VPC ID
Subnet Group Subnets 2c, 2a, 2b
Subnet Group AZ Placement No preference
Security Group XKCDAppElastiCacheSecurityGroup
Encryption-at-Rest True
Encryption Key Default
Encryption in transit True
Access Control Option No Access

Engine and Location

Engine and Location

ElastiCache: Engine and Location

Redis Settings

Redis Settings

ElastiCache: Redis Settings

Advanced Redis Settings

Advanced Redis Settings

ElastiCache: Advanced Redis Settings

Security

Security

ElastiCache: Security

3.3 Adding ElastiCache endpoint to Parameter Store

Go to Parameter Store under AWS Systems Manager and add ELASTICACHE_ENDPOINT without port :6379 to parameter store.

Setting Option
Name xkcdapp/test/ELASTICACHE_ENDPOINT
Description xkcd app ElastiCache Redis Endpoint
Tier Standard
Type SecureString
KMS key source My Current Account
KMS Key ID alias/aws/ssm
Value xkcdappredis.xxxxx.yyyy.rrrr.cache.amazonaws.com

3.4 Installing the Django-Redis package

# be sure to be in venv virtuall environment
pip install django-redis

3.5 Updating Django settings to use Redis as Session Storage

# xkcd_app/xkcd_app/settings.py
# ELASTICACHE_ENDPOINT without the port
ELASTICACHE_ENDPOINT = ssm.get_parameter(Name=prefixenv + ELASTICACHE_ENDPOINT, WithDecryption=True)['Parameter']['Value']
CACHE_LOCATION =  fredis://{ELASTICACHE_ENDPOINT}/0
CACHES = {
    default: {
        BACKEND: django_redis.cache.RedisCache,
        LOCATION: CACHE_LOCATION,
        OPTIONS: {
            CLIENT_CLASS: django_redis.client.DefaultClient,
        }
    }
}
#Use the redis as Session storage as well.
SESSION_ENGINE = django.contrib.sessions.backends.cache
SESSION_CACHE_ALIAS = default

Conclusion

In this second part of our Django blog series, we have created an RDS database and a Redis ElastiCache instance, along with their respective security groups. We have also configured the Django application to use these services and updated our Docker image. Our application secrets are securely stored in the Parameter Store. The only remaining step is to deploy our containers to Elastic Container Service.

In the next and final post of this series, Part 3: AWS: Deploying XKCD App to Elastic Container Service, we will:

  • Upload our final container image to Elastic Container Registry.
  • Create a Task Definition to deploy our docker image to Elastic Container Service.
  • Create a Load Balancer to distribute traffic to multiple ECS containers.
  • Set up the ECS Auto Scaling Policy.
  • Configure the security groups to only accept traffic from other security groups we created.

To stay secure, check out our DevSecOps and Cloud Security services!