This blog series covers the creation and deployment of the XKCD App using Django and AWS Elastic Container Service. In Part 1, we created and dockerized the XKCD App. In this post, we will continue from where we left off by deploying our dockerized Django application to ECS.
We will create a PostgreSQL RDS database for our database needs and connect our application to it. We will also ensure security by moving our application secrets, including the RDS connection credentials, to AWS Systems Manager Parameter Store. Finally, we will move our session storage to ElastiCache Redis to improve the XKCD application’s performance under heavy load.
This step is optional since Django uses the database as the session storage by default.
To get started, please sign in to the AWS Console, and let’s create the RDS.
Index
Part 1: Coding the XKCD Django App
Coding the XKCD Django App
- Creating and activating a virtual environment
- Installing Django
- Creating database models
- Adding models to the Django admin page
- Creating the homepage view
- Adding homepage view to URLs
- Creating homepage.html
- Creating requirements.txt file
- Dockerizing our Django App
- Testing the Docker Image
Part 2: AWS: Configuring RDS, Parameter Store, ElastiCache
Configuring AWS
- AWS RDS – Relational Database Service
- Configuring RDS Security Group
- Creating a Postgresql Database on RDS
- Updating Django settings to use the PostgreSQL Database Backend
- Install the PostgreSQL library
- Update the settings.py
- AWS Systems Manager Parameter Store
- Adding our secrets to Parameter Store
- Configuring Django App to use AWS Parameter Store
- Install AWS SDK for Python: boto3
- Update the settings.py file
- Migrating Django models to RDS instance
- Build & run the docker image with the AWS credentials
- Creating Parameter Store IAM Role
- Creating a super user
- ElastiCache Redis
- Creating the Security Group: XKCDAppElastiCacheSecurityGroup
- Creating the ElastiCache Redis Instance
- Adding ElastiCache endpoint to Parameter Store
- Installing django-redis package
- Updating Django settings to use Redis as Session Storage
Part 3: AWS: Deploying XKCD App to Elastic Container Service
- Elastic Container Registry
- Uploading XKCD Apps Docker Image to ECR
- Elastic Load Balancing
- ELB Security Group Creation
- ELB Creation
- Step 1: Configure Load Balancer
- Step 2: Security Settings
- Step 3: Security Groups
- Step 4: Routing: Target Group Creation
- Step 5: Registering Targets to Target Group
- Step 6: Review
- Step 7: Forward traffic from port 80 to port 8000
- Elastic Container Service
- Creating an ECS Task Execution Role: XKCDAppECSTaskExecutionRole
- Creating ECS security group: XKCDAppECSSecurityGroup
- Creating Task Definition: XKCDAppTaskDefinition
- Adding a container to XKCDAppTaskDefinition
- Creating Cluster Service
- Step 1: Configure Service
- Step 2: Configure Network
- Step 3: Set Auto Scaling
- Step 4: Review
- Load Testing our App with Hey
- Creating Auto Scaling for XKCDAppClusterService
- Testing the Auto Scaling Policy
- Updating Security Groups
- Updating XKCDAppElasticLoadBalancerSecurityGroup
- Updating XKCDAppECSSecurityGroup
- Updating XKCDAppElastiCacheSecurityGroup
- Updating XKCDAppRDSSecurityGroup
Configuring AWS
1. AWS RDS – Relational Database Service
1.1 Configuring RDS Security Group
On AWS Console, go to Security Groups
under the VPC
service. Press Create Security Group button.
RDS Security Group Creation
Setting | Option |
---|---|
Name | xkcdAppRDSSecurityGroup |
Description | Allows access on PostgreSQL port: 5432. |
VPC | Select your VPC, or use the default one. |
Inbound Rules | Type: PostgreSQL , Source: Anywhere |
Outbound Rules | Type: All Trafic , Destination: Anywhere |
1.2 Creating a Postgresql Database on RDS
On AWS Console, go to RDS
and start to create a new RDS instance. RDS creation settings below use default VPC and Free-tier. Feel free to change the configuration according to your requirements.
Table of Configuration
Setting | Option | Detail |
---|---|---|
Creation Method | Standard create | |
Engine Option | PostgreSQL | |
Engine Version | 12.5-R1 | Specify your DB engine version. |
Templates | Free Tier | If you are building for prod or test envs, choose respectively. |
DB Instance Identifier | xkcdappdb | |
Master Username | postgres | |
Master Password | < redacted > | |
DB Instance Class | db.t2.micro | you can upgrade the instance class to your needs. |
Storage Type | General Purpose(SSD) | |
Allocated Storage | 20GB | |
Enable Storage Autoscaling | False | You can enable this if you need it. |
Multi-AZ Deployment | False | This feature is not included in the free-tier. Production environments would benefit from this feature. |
VPC | Default VPC | Ideally, you should create your own VPC and choose that. |
Subnet Group | default | |
Public Access | Yes | We will turn this feature off later. We need public access only in the first setup to test things. |
VPC Security Group | Delete default and choose existing: xkcdAppRDSSecurityGroup |
|
Availability Zone | eu-west-2a | Choose one AZ if you are on free-tier. |
Database Authentication | Password Authentication | |
Initial Database Name | xkcd_test_db | The initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance. If you fail to fill this field, you will have to manually create the database. |
Database Engine
Choosing Database Engine: PostgreSQL
RDS Creation Template
Selecting Template: Standard
Database Settings and Credentials
Selecting Template: Standard
Database Instance Class
Database Instance Class: t2.micro
Database Storage
Database Storage: 20GB SSD
Connectivity
Be sure to select Public Access: Yes. We will need to access the database publicly in the first setup, but we will turn it off later.
Connectivity panel
Database Authentication
Database Authentication: Password authentication
Additional Configuration
NOTE: Additional Configuration: Initial Database Name field has to be filled or the AWS will not create a database inside the RDS instance you are creating. If you fail to fill this field, you will have to manually create the database.
Additional Configuration
Acquiring the Database Connection Endpoint
Wait for the database creation to be complete. Go to xkcdappdb
on RDS > Databases
and then go to Connectivity & Security
tab.
1.3 Updating Django settings to use the PostgreSQL Database Backend
1.3.1 Install the PostgreSQL library
# be sure to activate the venv environment
pip install psycopg2-binary
1.3.2 Update the settings.py
#/xkcd_app/xkcd_app/settings.py
import os
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ.get('DATABASE_NAME'),
'USER': os.environ.get('DATABASE_USER'),
'PASSWORD': os.environ.get('DATABASE_PASSWORD'),
'HOST': os.environ.get('DATABASE_HOST'),
'PORT': '5432'
}
}
Using environment variables for application configuration is not a secure way of doing things. Integrating AWS Parameter Store to our settings.py
file is something easy. Let’s configure AWS Systems Manager Parameter Store for XKCD App.
2. AWS Systems Manager Parameter Store
Go to Parameter Store
under AWS Systems Manager
and click on Create Parameter.
Parameter Store lets you create categorized parameter names, so you can have different parameters for different apps or app environments.
2.1 Adding our secrets to Parameter Store
For the DATABASE_NAME parameter, you can create a parameter as following:
AWS Parameter Store adding DATABASE_NAME as a parameter
Setting | Option | Detail |
---|---|---|
Name | xkcdapp/test/DATABASE_NAME | app and environment categorized parameter name |
Description | xkcd app RDS test DB identifier | |
Tier | Standard | |
Type | SecureString | This type of parameters are going to be encrypted while sitting. |
KMS key source | My Current Account | |
KMS Key ID | alias/aws/ssm | Choose the default AWS managed key. This will allow ECS containers to access parameters without further configuration. |
Value | xkcddb_test_db | Type the database name you gave in the RDS Creation: Additional Configuration step. |
Follow the same steps to add the remaining database connection credentials to the parameter store. You may also want to add SECRET_KEY
a variable generated for your Django project. Django’s SECRET_KEY
is used for encryption jobs inside Django.
When you are finished with adding the secrets, you should see something like this on Parameter Store Console.
Parameter Store Console
You can check the existence of parameters with AWS CLI if you have it set up.
aws ssm get-parameters --name /xkcdapp/test/DATABASE_HOST --with-decryption --region <your_region> --profile <your_aws_cli_profile>
2.2 Configuring Django App to Use AWS Parameter Store
2.2.1 Install AWS SDK for Python: boto3
# be sure to be in the venv virtual environment
pip install boto3
2.2.2 Update the settings.py file
# xkcd_app/xkcd_app/settings.py
import boto3
ssm = boto3.client('ssm', region_name='eu-west-2')
prefix = 'xkcdapp'
env = 'test'
prefixenv = f/{prefix}/{env}/
SECRET_KEY = ssm.get_parameter(Name=prefixenv + DJANGO_SECRET_KEY, WithDecryption=True)['Parameter']['Value']
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': ssm.get_parameter(Name=prefixenv + DATABASE_NAME, WithDecryption=True)['Parameter']['Value'],
'USER': ssm.get_parameter(Name=prefixenv + DATABASE_USER, WithDecryption=True)['Parameter']['Value'],
'PASSWORD': ssm.get_parameter(Name=prefixenv + DATABASE_PASSWORD, WithDecryption=True)['Parameter']['Value'],
'HOST': ssm.get_parameter(Name=prefixenv + DATABASE_HOST, WithDecryption=True)['Parameter']['Value'],
'PORT': '5432'
}
}
2.3 Migrating Django models to RDS instance
NOTE: Be sure that your AWS Accounts credentials are set in the AWS CLI’s credentials
file.
Migrate our migrations to AWS RDS.
python manage.py migrate
# Operations to perform:
# Apply all migrations: admin, auth, contenttypes, sessions, xkcd_app
# Running migrations:
# Applying contenttypes.0001_initial... OK
# Applying auth.0001_initial... OK
# .
# .
# Applying sessions.0001_initial... OK
# Applying xkcd_app.0001_initial... OK
2.4 Build & run the docker image with the AWS credentials
If you want to build and test the docker image with the application secrets on the AWS Parameter Store:
- build the image again to apply the
settings.py
file changes to the Docker image. - run the image with
AWS_ACCESS_KEY_ID
,AWS_SECRET_ACCESS_KEY
environment variables set.
# build the image again
docker build -t xkcd:latest .
# run with AWS credentials set in the environment variables.
docker run -p 8000:8000 -e AWS_ACCESS_KEY_ID='XXXXXXXXXXXX' -e AWS_SECRET_ACCESS_KEY='YYYYYYYYYYY' xkcd:latest
We won’t need to send the AWS Credentials to our Docker image when we deploy on the Elastic Container Service, as AWS automatically sets those environment variables upon instance creation.
2.5 Creating Parameter Store IAM Role
Go to Roles
under AWS IAM
and click on Create Role.
Select the System Manager use case.
AWS IAM Role Creation System Manager use case
Click on Next: Permissions and then Create Policy
.
AWS IAM Role Creation Inline Policy JSON
Copy and paste the following AWS IAM Policy to the editor and hit next.
{
Version: 2012-10-17,
Statement: [
{
Sid: AllowSSMAccess,
Effect: Allow,
Action: [
ssm:PutParameter,
ssm:DeleteParameter,
ssm:GetParameterHistory,
ssm:GetParametersByPath,
ssm:GetParameters,
ssm:GetParameter,
ssm:DescribeParameters,
ssm:DeleteParameters
],
Resource: arn:aws:ssm:*:*:parameter/*
}
]
}
Give a name and description to the Parameter Store access role.
AWS IAM Role Creation Inline Policy
Setting | Option |
---|---|
Name | SystemsManagerParameterStoreFullAccess |
Description | Systems Manager Parameter Store Full Access Role. |
We will be attaching this IAM Role to ECS containers we are going to run to allow them to read the parameters.
2.6 Creating a super user
When you are connected to your RDS instance, you can create an admin user with the following command.
python3 manage.py createsuperuser
This article may interest you: AWS IAM for Red and Blue Teams
3. ElastiCache Redis
3.1 Creating the Security Group: XKCDAppElastiCacheSecurityGroup
ElastiCache Redis Security Group Creation
Setting | Option |
---|---|
Name | XKCDAppElastiCacheSecurityGroup |
Description | XKCD Apps elasti cache security group. Allows access from port: 6379 |
VPC | Select your VPC, or use the default one. |
Inbound Rules | Type: CustomTCP , Port Range: 6379 , Source: Anywhere |
Outbound Rules | Type: All Trafic , Destination: Anywhere |
3.2 Creating the ElastiCache Redis Instance
Go to Redis
under ElastiCache
and press the “create” button.
Table of Configuration
Setting | Option |
---|---|
Cluster Engine | Redis, Cluster-Mode Disabled |
Location | Amazon Cloud |
Name | xkcdappredis |
Description | Session storage for XKCD App |
Engine version compatibility | 6.x |
Port | 6379 |
Parameter group | default.redis6.x |
Node type | cache.t2.micro (0.5 GiB) |
Number of replicas | 0 |
Multi-AZ | False |
Subnet Group | Create new |
Subnet Group Name | ecachesubnetgroup |
Subnet Group Description | Subnet group for XKCD apps ElastiCache Redis Storage. |
Subnet Group VPC ID | Default VPC ID |
Subnet Group Subnets | 2c, 2a, 2b |
Subnet Group AZ Placement | No preference |
Security Group | XKCDAppElastiCacheSecurityGroup |
Encryption-at-Rest | True |
Encryption Key | Default |
Encryption in transit | True |
Access Control Option | No Access |
Engine and Location
ElastiCache: Engine and Location
Redis Settings
ElastiCache: Redis Settings
Advanced Redis Settings
ElastiCache: Advanced Redis Settings
Security
ElastiCache: Security
3.3 Adding ElastiCache endpoint to Parameter Store
Go to Parameter Store
under AWS Systems Manager
and add ELASTICACHE_ENDPOINT
without port :6379
to parameter store.
Setting | Option |
---|---|
Name | xkcdapp/test/ELASTICACHE_ENDPOINT |
Description | xkcd app ElastiCache Redis Endpoint |
Tier | Standard |
Type | SecureString |
KMS key source | My Current Account |
KMS Key ID | alias/aws/ssm |
Value | xkcdappredis.xxxxx.yyyy.rrrr.cache.amazonaws.com |
3.4 Installing the Django-Redis package
# be sure to be in venv virtuall environment
pip install django-redis
3.5 Updating Django settings to use Redis as Session Storage
# xkcd_app/xkcd_app/settings.py
# ELASTICACHE_ENDPOINT without the port
ELASTICACHE_ENDPOINT = ssm.get_parameter(Name=prefixenv + ELASTICACHE_ENDPOINT, WithDecryption=True)['Parameter']['Value']
CACHE_LOCATION = fredis://{ELASTICACHE_ENDPOINT}/0
CACHES = {
default: {
BACKEND: django_redis.cache.RedisCache,
LOCATION: CACHE_LOCATION,
OPTIONS: {
CLIENT_CLASS: django_redis.client.DefaultClient,
}
}
}
#Use the redis as Session storage as well.
SESSION_ENGINE = django.contrib.sessions.backends.cache
SESSION_CACHE_ALIAS = default
Conclusion
In this second part of our Django blog series, we have created an RDS database and a Redis ElastiCache instance, along with their respective security groups. We have also configured the Django application to use these services and updated our Docker image. Our application secrets are securely stored in the Parameter Store. The only remaining step is to deploy our containers to Elastic Container Service.
In the next and final post of this series, Part 3: AWS: Deploying XKCD App to Elastic Container Service, we will:
- Upload our final container image to Elastic Container Registry.
- Create a Task Definition to deploy our docker image to Elastic Container Service.
- Create a Load Balancer to distribute traffic to multiple ECS containers.
- Set up the ECS Auto Scaling Policy.
- Configure the security groups to only accept traffic from other security groups we created.
To stay secure, check out our DevSecOps and Cloud Security services!