Shift from Containers to Serverless Computing using AWS Lambda - Part 2

In a Shift from Containers to Serverless Computing using AWS Lambda post we created a SNS topic, Lambda, SNS Topic Subscription and IAM Role with required policies using CloudFormation Template. We’ll continue on the same theme and switch to more advanced Lambda which will persist SNS event message into DynamoDB Table. We’ll again use CloudFormation Template to create complete stack including DynamoDB table. We’ll also take a closer looks at anatomy of CloudFormation template.

So Let’s get started. We’ll use Resources section of CloudFormation Template to create following required AWS resources -

SNS Topic

Following Topic section will create SNS Topic. DisplayName can be used for identifying SNS topic. We’ll intentionally leave out TopicName. It’s mandatory to have unique name across all stacks in AWS region hence better let AWS CloudFormation generate one.

1  Topic:
2    Type: 'AWS::SNS::Topic'
3    DisplayName: 'DynamodDB-Lambda-SNS-Topic'

DynamoDB Table

Following DynamoDB Table section will create DynamoDB Table. Being a NoSQL database, we don’t need to declare all the attributes in AttributeDefinitions section of of DynamoDB table. Only attributes that are used in KeySchema should be declared in AttributeDefinitions. KeySchema contains attributes that are Primary key/Partition key(with KeyType=HASH) and Index(with KeyType=RANGE). For demo sake it’s good enough to configure desired minimum number of consistent read and write per second to 1 in ProvisionedThroughput section. We’ll name this table as Account using TableName attribute.

 1  DynamoDBTable:
 2    Type: 'AWS::DynamoDB::Table'
 3    Properties:
 4      AttributeDefinitions:
 5        - AttributeName: 'id'
 6          AttributeType: 'S'
 7      KeySchema:
 8        - AttributeName: 'id'
 9          KeyType: 'HASH'
10      ProvisionedThroughput:
11        ReadCapacityUnits: 1
12        WriteCapacityUnits: 1
13      TableName: Account

Lambda Function

In previous blog we had used inline code and Node.js 6.10 runtime. AWS started supporting Node.js 8.10 runtime starting Apr 2018. This is very exciting as Node.js 8.10 supports async/await besides other features. In async function you can await for Promise or handle it’s rejection. It makes handling non blocking calls lot more cleaner, simpler and readable as if it is a synchronous call. More on this may be in future blog. We’ll switch to Node.js 8.10 runtime using Runtime and external handler function from index.js using Handler attribute. This Lambda function also needs execution role. For that we’ll need ARN(Amazon Resource Name) of LambdaExecutionRole. However that resource is not yet created. Hence we’ll use DependsOn attribute. It means that this Lambda Resource creation follows LambdaExecutionRole resource creation. Sequencing being sorted out we still need ARN of LambdaExecutionRole resource. We’ll use GetAtt function to get ARN attribute of LambdaExecutionRole.

 1    Type: 'AWS::Lambda::Function'
 2    Properties:
 3      Handler: index.handler
 4      Runtime: nodejs8.10
 5      Code: .
 6      Role: !GetAtt
 7        - LambdaExecutionRole
 8        - Arn
 9      Timeout: '30'
10    DependsOn:
11      - LambdaExecutionRole

SNS Topic Subscription

Now that we had created SNS topic and Lambda, we’ll subscribe Lambda to SNS topic using SNS Subscription. For that we need to specify Lambda Function ARN as a Endpoint and lambda as a Protocol. We also need to specify topic ARN. This will instruct SNS to deliver JSON encoded message to Lambda ARN. You must have noticed that we used GetAtt to get Lambda ARN and Ref to get SNS Topic ARN. Ref is another handy function which Returns SNS topic’s ARN however for Lambda it returns Name. Hence we can not use it for Lambda Function ARN. We also used DependsOn to manage the sequencing of resource creation.

 1  Subscription:
 2    Type: 'AWS::SNS::Subscription'
 3    Properties:
 4      Endpoint: !GetAtt
 5        - Function
 6        - Arn
 7      Protocol: lambda
 8      TopicArn: !Ref Topic
 9    DependsOn:
10      - Function

Lambda Invoke Permission for SNS Topic

Although we created Subscription, SNS won’t be able to trigger Lambda yet. We need to Grant lambda:InvokeFunction(Action) on Lambda ARN(FunctionName) to SNS service(Principal) and SNS Topic(sourceArn)

1  LambdaInvokePermission:
2    Type: 'AWS::Lambda::Permission'
3    Properties:
4      Action: 'lambda:InvokeFunction'
5      Principal: sns.amazonaws.com
6      FunctionName: !GetAtt
7        - Function
8        - Arn
9      SourceArn: !Ref Topic

Lambda Execution IAM Role

Finally we need create Lambda execution Role. We need to associate policies to this Role so that Lambda can - - Put log events in CloudWatch Logs - Subscribe and receive notifications from SNS - Write to DynamoDB Table

 1  LambdaExecutionRole:
 2    Type: 'AWS::IAM::Role'
 3    Properties:
 4      Policies:
 5        - PolicyName: LambdaPolicy
 6          PolicyDocument:
 7            Version: 2012-10-17
 8            Statement:
 9              - Action:
10                  - 'logs:CreateLogGroup'
11                  - 'logs:CreateLogStream'
12                  - 'logs:PutLogEvents'
13                Resource:
14                  - 'arn:aws:logs:*:*:*'
15                Effect: Allow
16              - Action:
17                  - 'sns:Subscribe'
18                  - 'sns:Receive'
19                  - 'sns:Unsubscribe'
20                Resource: !Ref Topic
21                Effect: Allow
22              - Action:
23                  - 'dynamodb:PutItem'
24                Resource: !GetAtt
25                  - DynamoDBTable
26                  - Arn
27                Effect: Allow
28      AssumeRolePolicyDocument:
29        Version: 2012-10-17
30        Statement:
31          - Action:
32              - 'sts:AssumeRole'
33            Effect: Allow
34            Principal:
35              Service:
36                - lambda.amazonaws.com

Putting it all together will lead to following CloudFormation template -

 1AWSTemplateFormatVersion: 2010-09-09
 2Resources:
 3  Topic:
 4    Type: 'AWS::SNS::Topic'
 5  DynamoDBTable:
 6    Type: 'AWS::DynamoDB::Table'
 7    Properties:
 8      AttributeDefinitions:
 9        - AttributeName: 'id'
10          AttributeType: 'S'
11      KeySchema:
12        - AttributeName: 'id'
13          KeyType: 'HASH'
14      ProvisionedThroughput:
15        ReadCapacityUnits: 1
16        WriteCapacityUnits: 1
17      TableName: Account
18  Function:
19    Type: 'AWS::Lambda::Function'
20    Properties:
21      Handler: index.handler
22      Runtime: nodejs8.10
23      Code: .
24      Role: !GetAtt
25        - LambdaExecutionRole
26        - Arn
27      Timeout: '30'
28    DependsOn:
29      - LambdaExecutionRole
30  Subscription:
31    Type: 'AWS::SNS::Subscription'
32    Properties:
33      Endpoint: !GetAtt
34        - Function
35        - Arn
36      Protocol: lambda
37      TopicArn: !Ref Topic
38    DependsOn:
39      - Function
40  LambdaInvokePermission:
41    Type: 'AWS::Lambda::Permission'
42    Properties:
43      Action: 'lambda:InvokeFunction'
44      Principal: sns.amazonaws.com
45      FunctionName: !GetAtt
46        - Function
47        - Arn
48      SourceArn: !Ref Topic
49  LambdaExecutionRole:
50    Type: 'AWS::IAM::Role'
51    Properties:
52      Policies:
53        - PolicyName: LambdaPolicy
54          PolicyDocument:
55            Version: 2012-10-17
56            Statement:
57              - Action:
58                  - 'logs:CreateLogGroup'
59                  - 'logs:CreateLogStream'
60                  - 'logs:PutLogEvents'
61                Resource:
62                  - 'arn:aws:logs:*:*:*'
63                Effect: Allow
64              - Action:
65                  - 'sns:Subscribe'
66                  - 'sns:Receive'
67                  - 'sns:Unsubscribe'
68                Resource: !Ref Topic
69                Effect: Allow
70              - Action:
71                  - 'dynamodb:PutItem'
72                Resource: !GetAtt
73                  - DynamoDBTable
74                  - Arn
75                Effect: Allow
76      AssumeRolePolicyDocument:
77        Version: 2012-10-17
78        Statement:
79          - Action:
80              - 'sts:AssumeRole'
81            Effect: Allow
82            Principal:
83              Service:
84                - lambda.amazonaws.com

AWS also introduced new types of Handler function with Node.js 8.10 runtime. Following handler function from index.js will extract message from SNS event notification and persist it in DynamodDB Account Table.

 1// Create DynamoDB document client
 2const dynamoDb = new AWS.DynamoDB.DocumentClient();
 3
 4exports.handler = async (event) => {
 5    try {
 6        // Extract message
 7        const message = JSON.parse(event.Records[0].Sns.Message);
 8
 9        // Prepare account item
10        const accountItem = {
11            TableName: "Account",
12            Item: {
13                id: uuid.v1(),
14                name: message.name,
15                iban: message.iban
16            }
17        };
18
19        // insert account in DynamoDB
20        await dynamoDb.put(accountItem).promise();
21
22        return "Registered new account with id " + accountItem.Item.id;
23    } catch (err) {
24        console.log(err);
25        throw err;
26    }
27}
28

Now that everything is sorted out, we need to package it before creating CloudFormation Stack. Package command takes care of uploading local artifacts to s3 bucket and replacing there references with s3 location. Following AWS CLI command will take care of packaging -

$ aws cloudformation package --template-file <your-temaplate.yaml> --s3-bucket <your-s3-bucket-name> --output-template-file packaged-template.yaml


 Successfully packaged artifacts and wrote output template to file packaged-template.yaml.
 Execute the following command to deploy the packaged template
 aws cloudformation deploy --template-file /template-path/packaged-template.yaml --stack-name

Now we can create Stack using following command -

$ aws cloudformation deploy --template-file packaged-template.yaml --stack-name <your-stack-name> --capabilities CAPABILITY_IAM

  Waiting for changeset to be created..
  Waiting for stack create/update to complete
  Successfully created/updated stack - your-stack-name

Once stack is created we can list stack resources using following command -

$ aws cloudformation list-stack-resources --stack-name <your-stack-name>

You will see list of resources. Please copy SNS topic(AWS::SNS::Topic) ARN (PhysicalResourceId). We’ll need it to publish message to SNS topic.

Now that everything is in place, let’s publish a message to SNS topic using following command -

aws sns publish --topic-arn <your-sns-topic-arn> --message '{"name": "Foo", "iban": "NL38FRBK0292964727"}'

This will result in invoking Lambda which will in turn write SNS message into DynamoDB table. You can verify this using CloudWatch logs and DynamoDB.

Source code is available on GitHub

That’s it. In next blog we’ll do something more advanced and meaningful using Lambda.

comments powered by Disqus