Piotr Nowicki's Blog

AWS Lambda Local Invocation Using SAM Local and Localstack

2018-11-16

tags:

The biggest struggle I had in my mind when thinking about serverless and AWS itself was - how the hell to properly test it and create development environment? There is no way to hit production without properly and in easy way testing all parts of AWS services.

I could identify 2 biggest players in this area:

delivered by AWS directly - SAM Local (abbreviation of Serverless Application Module) with its local option,
localstack with its large set of docker images for AWS services.

As an example - you can run local lambda and API Gateway with one command: sam local start-api. This will take the template.yml CloudFormation file from your current directory and create the resources for you.

You can then easily access API Gateway by default entering localhost:3000/YOUR_ENDPOINT and that’s it.

Now, sam local is a bit limited as it supports Lambdas and API Gateway but lack support e.g. for other services from AWS your Lambda might be using.

In my case it was Python 3.6 Lambda that required DynamoDB access, so I was looking for Local DynamoDB I could connect to. There is a AWS delivered Local DynamoDB but I was more interested in a wider variety of AWS services running locally. Reason behind is that right now I’m using DynamoDB only but soon I’ll be using SQS or S3.

Meet localstack. You can easily start the whole stack of AWS services with one command: localstack start.

Now the question is - how to connect from your Lambda, executed using sam local to DynamoDB started by localstack.

You might suspect that accessing localhost:4569 (where 4569 is the DynamoDB port) is not an option. That’s because sam local is starting the Docker container for your lambda. Hence, accessing localhost:4569 will not transit you out of the Docker box. You might investigate what is your Docker IP address using ifconfig, e.g.:

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:a9ff:fe18:7ebf  prefixlen 64  scopeid 0x20<link>
        ether 02:42:a9:18:7e:bf  txqueuelen 0  (Ethernet)
        RX packets 167  bytes 16155 (16.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 196  bytes 41782 (41.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

And using this IP address as DynamoDB Endpoint (Python code below):

dynamodb = boto3.resource('dynamodb', endpoint_url='http://172.17.0.1:4569', region_name='eu-central-1')

Docker networks

(not tested by me!) Another approach to the same problem might be to:

create a docker network,
start localstack in docker mode using this network (localstack start --docker)
start SAM local Api Gateway using the same network

I have not verified if this will work but instead bumped into following Pull Request that is yet not merged, so it might be a showstopper.

Identifying if Lambda is called locally or not

If you’d like to identify if the Lambda is being invoked manually or from production AWS environment, you can easily do so with AWS SAM Local by checking the environment variable named AWS_SAM_LOCAL is set to true.