11 January 2019

Provisioning Flask Apps | Traditional Server Deployment

by Sasha Elaine Fox

Provisioning Flask Apps

Overview

Python Flask is one of two major frameworks for making Python web apps (the other being Djanjo). It allows web apps, like back-end APIs, to be constructed with relative ease.

Once we write a shiny new Flask app an obvious question comes to mind—how are we supposed to deploy it?

Flask itself is only a web framework, not a self-contained server + web application framework. In order to deploy Flask apps we will need to use an HTTP server (a la nginx, Apache2, etc.) or a serverless architecture such as AWS Lambda. We could also completely circumvent using an HTTP server at together by going with a service like AWS API Gateway.

We’ll cover serverless deployment with Lambda + API gateway in another article, but for now let’s look traditional server deployment as it applies to Flask apps.

Traditional Server Deployment

If our app is relatively large, is responsible for many things, or has special hardware requirements, it might make sense to deploy it on a single, powerful server instance.

Since we’re going with tradition server deployment we’ll need an HTTP server. By far the most used server is Apache2. It’s old and somewhat arcane, but tractable enough once some effort has been put in.

We’ll also need a place to host our server. We could go with Google Compute Engine, Microsoft Azure, or AWS. We could even go with local server deployment, if we were feeling particularly feisty. For no particular reason we’ll focus on using AWS for this guide. However, the steps outlined here should be adaptable for other cloud service providers/local deployment fairly easily.

An Example Provisioning Work-Flow

First we’ll walk through the steps of deploying a Flask app by hand then we’ll discuss automated provisioning. We’ll assume some basic knowledge with AWS EC2, such as how to set-up security groups and launch instances.

This example work-flow makes some fairly arbitrary choices, which are outlined below:

Ubuntu 16.04 for Host System.
App is implement in Python 3 and configured as a SetupTools package.
App is hosted in a Git repo somewhere.

Basic AWS set-up

Start AWS EC2 instance

From the EC2 Management Console choose Launch Instance. At the Choose AMI screen choose Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-835b4efa or similar. Choose whatever instance type is appropriate and launch it. You will be prompted to create a new key pair for the AWS instance – create one and copy the *.pem file to local machine. This is the key-pair that will be used to login to the default ubuntu user.

Set up an AWS security group

Set up an AWS security group with with inbound and outbound permission for SSH ( port 22) and HTTP(S) ( ports 80 and 443). Assign the instance created in the previous step to this security group.

Setup initial Admin User

#Log into EC2 instance as default user `ubuntu`.
sudo apt-get update
sudo apt-get upgrade
sudo adduser $USERNAME #Create a new user to replace default `ubuntu` user

#Type password and credentials for new user

sudo adduser $USERNAME sudo #Give new user sudo privileges
sudo su $USERNAME #Login into new user so that subsequent privileges will be set correctly.
mkdir .ssh
chmod 700 ./.ssh
touch .ssh/authorized_keys
chmod 600 .ssh/authorized_keys

Copy public key for new user into .ssh/authorized_keys

The new user, $USERNAME should now be able to login using ssh. At this point it is a good idea to remove the default user ubuntu with userdel -r ubuntu.

Note that all commands later in the tutorial are assumed to be performed by $USERNAME.

Install Dependencies

We need to install core components, like the Python 3 interpreter, GCC, and Apache2 itself before we do anything else.

# Install basic build requirements.
sudo apt-get update
sudo apt-get upgrade -y

sudo apt-get install -y python3 python3-dev
sudo apt-get install -y python3-pip
sudo python3 -m pip install --upgrade pip
sudo apt-get install -y gcc build-essential

# Install HTTP Server
sudo apt-get install -y apache2 apache2-dev
# Install the python 3 version of mod-WSGI for apache2
sudo apt-get install libapache2-mod-wsgi-py3

App Structure

Let’s assume we have our Flask app hosted at https://github.com/demo/example_app.git and it is structured as follows.

├── setup.py (SetupTools script)
├── app.py (Flask entry point)
├───example_app
│   ├── __init__.py
│   ├── __main__.py
│   ├── (other app files)
├───apache2_files
│   ├── app.conf (apache2 configuration file)
│   ├── app.wsgi (WSGI file, required for apache2)

The files contained in apache2_files are what allow Flask to work with apache.

The app.conf file contains configuration information used by apache. These config files are obtuse and not fun to write, especially when things like WSGI is involved. Below is a conf file that should work for most flask apps.

<VirtualHost *:80>
  DocumentRoot /var/www/example_app

  <Directory /var/www/>
    Options Indexes FollowSymLinks MultiViews
    Require all granted
  </Directory>

	WSGIDaemonProcess app user=www-data group=www-data processes=1 threads=8
  WSGIScriptAlias / /var/www/wsgi_scripts/example_app.wsgi process-group=app application-group=%{GLOBAL}

  <Directory /var/www/example_app>
    WSGIProcessGroup %{GLOBAL}
    WSGIApplicationGroup %{GLOBAL}
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Require all granted
  </Directory>

  <Directory /var/www/wsgi_scripts>
  	AllowOverride None
  	Require all granted
  </Directory>

  CustomLog /var/www/example_app/logs/access.log combined
  ErrorLog /var/www/example_app/logs/error.log

</VirtualHost>

For more information about writing conf files for use with WSGI apps, visit the modwsgi documentation;

The app.wsgi file is what apache will actually run in order to access out app. These files are usually short and do nothing more than make sure out Flask app is visible.

A WSGI file that should work for most Flask apps is shown below.

import sys
import logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0,'/var/www/example_app')

from example_app import app as application

Now that we know more-or-less what our app looks like, we can actually install everything.

Clone and Install the App

# Make directory for the App and set (temporary) permissions
# We mainly set these permissions now so that we do not have to run git with sudo
sudo mkdir /var/www/example_app
sudo chown $(whoami) /var/www/example_app
sudo chgrp $(whoami) /var/www/example_app

# clone app from GitHub into app directory
git clone -b dev https://github.com/demo/example_app.git /var/www/example_app/
# Install the rest of the requirements and the app itself
sudo python3 -m pip install /var/www/example_app/

Set-Up Apache2

Now we need to move the apache config file and WSGI script to the correct locations on the system and set permissions.

# Make directories for WSGI scripts
sudo mkdir /var/www/wsgi_scripts

# Copy WSGI script for the app
sudo cp /var/www/app/apache2_files/app.wsgi /var/www/wsgi_scripts

# Make directory for apache2 logs
mkdir /var/www/app/logs

# Set permissions for the app directory
sudo chown -R www-data  /var/www
sudo chgrp -R www-data  /var/www
sudo chmod -R 755  /var/www
sudo chmod g+s  /var/www

# Set permissions for WSGI directory
sudo chmod -R 755 /var/www/wsgi_scripts

# Add app .conf file to apache2's sites-available
sudo cp /var/www/app/apache2_files/app.conf /etc/apache2/sites-available
# Disable default site
sudo a2dissite 000-default
# Enable Librarian QAS site
sudo a2ensite app
# Reload apache2
sudo service apache2 restart

Our app should now be set-up and ready to go!

( Optional ) Setup SSL certificates through Let’s Encrypt

If we want to enable SSL encryption (which we certainly do for production environments) then we’ll need to get an SSL certificate for our app. While there are a few different certification authorities we can work with, the wonderful Let’s Encrypt project provides a free and easy to set-up certs.

To use Let’s Encrypt certs all we need to do is install the certbot utility and set it to renew certificates via a chron job.

sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot --apache -d $SITE_URL

#Select `example_app.conf` when prompted
sudo crontab -e

#Add the following line to crontab
15 3 * * * /usr/bin/certbot renew --quiet
#to enable automatic renewal of certificates.

Putting Everything Together

Now that we’ve walked through all the steps needed to get a Flask app up and running by hand, let’s assemble a provisioning script we can use to use to provision a bare VM.

We’ll still need to initialize the AWS EC 2 instance by hand, but we should be able to login and run a single script (or have a provisioning tool do the same) and have our Flask app up and running!

(Note the following script is designed to work with Ubuntu 16.04 and requires sudo privileges.)

Provisioning Script

#!/bin/bash

# app_setup.sh

# End-to-end set-up script template

# Install basic build requirements.
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y python3 python3-dev
sudo apt-get install -y python3-pip
sudo python3 -m pip install --upgrade pip
sudo apt-get install -y gcc build-essential

# Install HTTP Server
sudo apt-get install -y apache2 apache2-dev
# Install the python 3 version of mod-WSGI for apache2
sudo apt-get install libapache2-mod-wsgi-py3

# Make directory for the QAS and set (temporary) permissions
# We mainly set these permissions now so that we do not have to run git with sudo
sudo mkdir /var/www/example_app
sudo chown $(whoami) /var/www/example_app
sudo chgrp $(whoami) /var/www/example_app

# clone QAS app into example_app directory
git clone -b dev https://github.com/bejphil/example_app.git /var/www/example_app/
# Install the rest of the requirements and the QAS itself
sudo python3 -m pip install /var/www/example_app/

# Make directories for WSGI scripts and the QAS
sudo mkdir /var/www/wsgi_scripts

# Copy WSGI script for the QAS
sudo cp /var/www/example_app/apache2_files/example_app.wsgi /var/www/wsgi_scripts

# Make directory for apache2 logs
mkdir /var/www/example_app/logs

# Set permissions for QAS directory
sudo chown -R www-data  /var/www
sudo chgrp -R www-data  /var/www
sudo chmod -R 755  /var/www
sudo chmod g+s  /var/www

# Set permissions for WSGI directory
sudo chmod -R 755 /var/www/wsgi_scripts

# Add QAS .conf file to apache2's sites-available
sudo cp /var/www/example_app/apache2_files/app.conf /etc/apache2/sites-available
# Disable default site
sudo a2dissite 000-default
# Enable Librarian QAS site
sudo a2ensite app
# Reload apache2
sudo service apache2 restart

# Set up SSL certs
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot --apache -d $SITE_URL

# Set up certs to renew automatically
sudo (crontab -l ; echo "15 3 * * * /usr/bin/certbot renew --quiet") | crontab

tags: python - flask - aws - ec2 - apache2

The Foxfire Blog

Notes from my past self.