Configuring DynamoDB tables per deployment stage using Serverless Framework

2019-05-30

tags: dynamodb, lambda, env-vars

Context

During serverless application development, you’d like to separate the DynamoDB tables and data that is used for production and for lower environments like development. For sure you don’t want to mess them up!

There are a few ways how to deal with it - the most secure would be to create separate AWS accounts within one organization and use it accordingly for dev and prod deployment stages.

There is also a simpler approach that might be more feasible for smaller projects or if you want to do a quick start. It depends on selecting a naming convention and create DynamoDB tables using this convention. It might be as simple as:

[name-of-serverless-service]-[table-name]-[stage]

In case of a table named fuel, 3 stages (local, dev and prod) of the serverless service named hq-serverless you’d end up with 3 DynamoDB tables named:

• hq-serverless-fuel-local
• hq-serverless-fuel-dev
• hq-serverless-fuel-prod

How to do it in Serverless Framework?

Below you can find an excerpt from Serverless Framework serverless.yaml I’m using (not relevant sections were removed for brevity):

service: hq-serverless

provider:
  name: aws
  (...)
  stage: ${opt:stage, 'local'}
  environment:
    TABLE_FUEL: "${self:custom.tableFuel}"

resources:
  Resources:
    HQFuel:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: "${self:custom.tableFuel}"
        (...)
custom:
  tableFuel: "${self:service}-fuel-${self:provider.stage}"

Mind that the actual logic for the table name creation is defined in custom.tableFuel variable, i.e.: "${self:service}-fuel-${self:provider.stage}".

The provider.stage itself is taken from sls --stage parameter value or (if not defined) it defaults to local.

Another solution would be to:

• omit the custom.tableFuel variable,
• define the table naming convention in environment.TABLE_FUEL and
• refer to it from HQFuel.Properties.TableName using ${env.TABLE_FUEL}

However, this doesn’t seem to work in serverless (1.43.0) generated CloudFormation file. It looks like it uses ${env.TABLE_FUEL} instead of the resolved value (hq-serverless-fuel-dev).

CloudFormation can’t cope with that and just fails, so I ended up with configuration as presented.

How to access table name from lambda source code?

Why do we need the environment.TABLE_FUEL? Well, it’s useful to be able to reference the table name from your code. You can use it to invoke queries or from integration tests to populate it with some test data.

In case of Python Lambda code, reading it it’s just a matter of reading environmental property:

    @staticmethod
    def table_name():
        return os.environ["TABLE_FUEL"]

That being said - for larger projects I would not recommend this idea and go with physical separation of production and pre-production environments by creating separate AWS accounts.