Building an HTTP API with AWS Serverless Application Model (AWS SAM)
What is AWS SAM? Amazon Web Services SAM is a framework for building serverless applications on the AWS platform. SAM helps you to quickly get started building and deploying serverless code for AWS and managing the experience.
What is a serverless application?
Serverless applications are a method of providing applications on a usage only basis, often in a very highly scalable way. They allow us to forget about concepts like servers and infrastructure and just focus on what our application is trying to achieve.
Serverless is another tool in the Cloud Computing arsenal, providing a quick and easy way to deploy code and applications. Building on concepts like Platform-as-a-Service (PaaS), most serverless environments provide Function-as-a-Service (FaaS), and a suite of tools to run and deploy your application in the Cloud without infrastructure management.
In AWS Serverless applications can be constructed with a combination of AWS Lambda and AWS API Gateway (if you wish to provide HTTP access to your Lambda Function). AWS Lambda allows us to write and deploy code without having to setup any infrastructure, such as servers, to manage that code. Every time we call the function, the code will automatically execute within the AWS runtime environment. The runtime will scale up and down as needed to run as many instances of the function as we require.
Although serverless applications are not always the best architecture they certainly have their use cases.
Building a Serverless Application on AWS with SAM
SAM builds upon traditional serverless applications helping with development, testing, and deployment of serverless applications within AWS. Although we're going to cover AWS SAM in this article, it's also worth checking out the Serverless Framework (and the Serverless Comparison for more information and comparisons).
Although you can roll your own Lambda functions in AWS without SAM, and you might later end up using only part of the toolkit, it's worth giving SAM a try if you're writing an app for AWS and want to try building a serverless one.
Essentially, there are two key components to the SAM framework:
- The SAM Command Line Interface (CLI) and tools, which helps you manage your application; and
- The Cloudformation extensions that make it easier to manage and deploy your application.
In case something isn't clear we've also addded the code to Github.
Getting Started with the CLI
In order to get started with the CLI you'll just need to install the CLI. AWS offers options for Linux, Windows and macOS.
We're on macOS so we use homebrew (we also highly recommend having Docker installed):
brew tap aws/tap
brew install aws-sam-cli
We're going to use JavaScript (Node.js) for these examples. However, AWS Lambda runtimes include: Node, Python, Ruby, Java, Go, .Net, or most other languages via a custom runtime.
We're also going to choose the Zip deployment method and use a quickstart template. Although we could use the hello world example, it's probably easier to start from scratch and add a few parts ourselves.
sam init
The full output of the init command is as follows with the options we chose:
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template Location
Choice: 1
What package type would you like to use?
1 - Zip (artifact is a zip uploaded to S3)
2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1
Which runtime would you like to use?
1 - nodejs14.x
2 - python3.8
3 - ruby2.7
4 - go1.x
5 - java11
6 - dotnetcore3.1
7 - nodejs12.x
8 - nodejs10.x
9 - python3.7
10 - python3.6
11 - python2.7
12 - ruby2.5
13 - java8.al2
14 - java8
15 - dotnetcore2.1
Runtime: 1
Project name [sam-app]:
Cloning from https://github.com/aws/aws-sam-cli-app-templates
AWS quick start application templates:
1 - Hello World Example
2 - Step Functions Sample App (Stock Trader)
3 - Quick Start: From Scratch
4 - Quick Start: Scheduled Events
5 - Quick Start: S3
6 - Quick Start: SNS
7 - Quick Start: SQS
8 - Quick Start: Web Backend
Template selection: 3
-----------------------
Generating application:
-----------------------
Name: sam-app
Runtime: nodejs14.x
Dependency Manager: npm
Application Template: quick-start-from-scratch
Output Directory: .
Next steps can be found in the README file at ./sam-app/README.md
The Serverless App
Now that we've generated the SAM app we can see the core structure.
We have a src/handler's folder with a lambda function, some tests, and a
template.yml file.
If we look at the handler, we can see a simple JavaScript function.
exports.helloFromLambdaHandler = async () => {
// If you change this message, you will need to change hello-from-lambda.test.js
const message = 'Hello from Lambda!';
// All log statements are written to CloudWatch
console.info(`${message}`);
return message;
}
Building the App
Now we can start to see the real benefit of the SAM CLI. We can build this function locally using:
sam build
If you want to use a Docker container to build your app, without having to have
node or NPM installed (or even if you have a different version than the one
your lambda function uses), you can pass --use-container to the builder.
This will build download a NodeJS image from the ECR public registry and build our SAM application inside this. The image is derived from Amazon Linux, the same linux version used inside AWS Lambda itself.
sam build --use-container
The build function also makes use of the template.yml file to understand the
structure of our application.
The build command creates a .aws-sam/build folder containing our function
code and a formatted version of the template.yaml file. It also contains a
file called build.toml.
The build file is in TOML (Tom's Obvious, Minimal Language) format and just contains some information about the build.
Running our Lambda Function
We can now invoke the Lambda function locally, again showing the power of using SAM versus just creating a standard lambda function.
sam local invoke
In order to use SAM local invoke we again need Docker installed.
This will pull the amazon/aws-sam-cli-emulation-image-nodejs14.x image down
from Docker and will execute our Lambda function within the image.
Running the command above we will see the following output:
START RequestId: d850c293-8091-44bc-8deb-f919e8003533 Version: $LATEST
2021-08-04T10:19:43.690Z d850c293-8091-44bc-8deb-f919e8003533 INFO Hello from Lambda!
END RequestId: d850c293-8091-44bc-8deb-f919e8003533
REPORT RequestId: d850c293-8091-44bc-8deb-f919e8003533 Init Duration: 1.02 ms Duration: 753.73 ms Billed Duration: 800 ms Memory Size: 128 MB Max Memory Used: 128 MB
"Hello from Lambda!"
Awesome! We've now managed to execute our Lambda function and we saw the "Hello from Lambda" output.
Adding an event
Lambda functions aren't hugely useful on their own. Generally, they run in response to an event. So let's transform this function to take some input and change it's response based on the input:
exports.helloFromLambdaHandler = async (event, context) => {
const name = event.name || 'World';
const message = `Hello ${name}`;
console.info(`${message}`);
return message;
};
Now we're expecting an event and if there's no event.name we'll simply say
"Hello World".
Let's also create a simple JSON file (called /events/name.json) containing
a simple event:
{
"name": "Steve"
}
Okay, let's build this and go ahead and run it again. This time, we will pass the
events/name.json JSON file as an event into the Lambda function.
Since sending events is a pretty common use case for SAM, we can execute the CLI
with the ``--event parameter`.
sam build --use-container && sam local invoke --event ./events/name.json
Now we get a different output:
"Hello Steve"
Responding to a web request
Although we could deploy our function now, it's still not hugely useful. Let's take advantage of one of the really impressive parts of the AWS Serverless Stack. We'll extend the function to handle a web request using the AWS API Gateway Lambda Proxy system. With this setup, API Gateway acts as a proxy to our Lambda function, invokes the function, and then returns the result of the Lambda function.
exports.helloFromLambdaHandler = async (event, context) => {
console.info(`${JSON.stringify(event)}`);
return {
"statusCode": 201,
"body": JSON.stringify({
message: "Hello World"
})
};
};
This handler will accept any request, print the contents of the event, and then return an object in the expected format of the API Gateway. Running the function will look like this:
START RequestId: 446ce558-c793-415f-9d68-1f16ae93a1c5 Version: $LATEST
2021-08-04T12:24:30.145Z 446ce558-c793-415f-9d68-1f16ae93a1c5 INFO {"name":"Steve"}
END RequestId: 446ce558-c793-415f-9d68-1f16ae93a1c5
REPORT RequestId: 446ce558-c793-415f-9d68-1f16ae93a1c5 Init Duration: 1.16 ms Duration: 971.35 ms Billed Duration: 1000 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":201,"body":"{\"message\":\"Hello World\"}"}
This function is great, but how do we call it over HTTP? Well, this is where API Gateway comes in. API Gateway will allow us to proxy requests sent over HTTP to our Lambda function and return the response over HTTP too.
There's even a command to simulate this locally too. However, before we can
simulate an HTTP Server locally we need to tell our CloudFormation template
(template.yml) that we intend to use API Gateway with the Lambda function.
Modifying our CloudFormation template
AWS CloudFormation is another AWS product that allows us to describe in JSON or YAML files a group of resources that we want to use on AWS. We can then deploy that group of resources as, what CloudFormation calls, a stack. CloudFormation is an AWS specific solution to this problem, similar to Terraform if you've come across that before.
AWS SAM adds an additional feature to CloudFormation, called the AWS::Serverless Transform. The serverless transform allows us to describe a serverless application in a really simple way. When the CloudFormation template is deployed the transform essentially just translates to create a more complicated set of resources that get deployed.
If we look at the content of the sample app, excluding the comments (which is how it appears in the build directory), then we see the following:
AWSTemplateFormatVersion: 2010-09-09
Description: sam-app
Transform:
- AWS::Serverless-2016-10-31
Resources:
helloFromLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/hello-from-lambda.helloFromLambdaHandler
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
Description: A Lambda function that returns a static string.
Policies:
- AWSLambdaBasicExecutionRole
CodeUri: helloFromLambdaFunction
The Serverless Transform
The serverless function (called hellFromLambdaFunction) is pretty easy to
understand. We're specifying the handler as our
hello-from-lambda.helloFromLambdaHander, allocating 128MB of RAM for the
function, and specifying a timeout of 100 seconds.
We also need permission to execute the lambda function, which CloudFormation
will configure.
Full details are in the AWS::Serverless::Function Docs.
As mentioned above, this resource is basically just a transform around the CloudFormation template. If you're interested in what the real template looks like, then you can run:
sam validate --debug
This command translates the Serverless components and shows how they will be executed in regular CloudFormation. This is especially useful if you want to allow for more complicated permissions or setups later.
AWSTemplateFormatVersion: 2010-09-09
Description: sam-app
Resources:
helloFromLambdaFunction:
Properties:
Code:
S3Bucket: bucket
S3Key: value
Description: A Lambda function that returns a static string.
Handler: src/handlers/hello-from-lambda.helloFromLambdaHandler
MemorySize: 128
Role:
Fn::GetAtt:
- helloFromLambdaFunctionRole
- Arn
Runtime: nodejs14.x
Tags:
- Key: lambda:createdBy
Value: SAM
Timeout: 100
Type: AWS::Lambda::Function
helloFromLambdaFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Tags:
- Key: lambda:createdBy
Value: SAM
Type: AWS::IAM::Role
For example, this is the expanded version of the above template. You can see how
the simple Policies/AWSLambdaBasicExecutionRole is expanded to add a new
resource called helloFromLambdaFunctionRole, this role attaches a full policy
document creating an IAM role.
When the API Gateway is added a similar transform will take place to add
a resource called ServerlessRestApi describing the gateway.
Expanding the CloudFormation template to add HTTP
This is where the power of the Serverless Transform really comes into play.
We're going to add the following event to our Lambda function:
@@ -30,3 +30,9 @@ Resources:
Policies:
# Give Lambda basic execution Permission to the helloFromLambda
- AWSLambdaBasicExecutionRole
+ Events:
+ HelloWorld:
+ Type: Api
+ Properties:
+ Path: /hello
+ Method: get
AWS allows you to add a few different event sources, but here we're adding Api
and we're setting a GET path to /hello.
Our full template now looks like the following:
AWSTemplateFormatVersion: 2010-09-09
Description: sam-app
Transform:
- AWS::Serverless-2016-10-31
Resources:
helloFromLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/hello-from-lambda.helloFromLambdaHandler
Runtime: nodejs14.x
MemorySize: 128
Timeout: 5
Description: A Lambda function that returns a static string.
Policies:
- AWSLambdaBasicExecutionRole
Events:
HelloWorld:
Type: Api
Properties:
Path: /hello
Method: get
CodeUri: helloFromLambdaFunction
This transforms to a much more complex CloudFormation template that adds an
AWS::ApiGateway::RestApi called ServerlessRestApi and all of the permissions
we need to run this.
We've also dropped the timeout to 5 seconds, as this is serverless after all.
Okay, let's go ahead and run this now, remembering to build the application again:
sam build --use-container && sam local start-api
By default, this will run the server on http://127.0.0.1:3000/. You might get an
authentication error calling / but that's to be expected as we don't have
anything running here.
If you head to http://127.0.0.1:3000/hello we'll see the Lambda function execute.
In the browser we'll see:
{
"message": "Hello World"
}
In the console we'll also see the event output to the logs. Let's call
http://127.0.0.1:3000/hello?name=steve and we can see an entry in our logs called
queryStringParameters.
So once more, we can modify our handler code to the following:
exports.helloFromLambdaHandler = async (event, context) => {
const name = event.queryStringParameters?.name || "world";
return {
"statusCode": 201,
"body": JSON.stringify({
message: `Hello ${name}`
})
};
};
Perfect. We're now seeing {"message":"Hello steve"} when we call our API.
All that's left now is to deploy this.
Deploying a SAM application
Deploying applications with AWS SAM is really just a case of executing the CloudFormation template to create a CloudFormation stack with our code.
As before, we can use the SAM CLI to help with this process. Assuming you have an
AWS credentials file setup already (you can use aws configure if you have
the AWS CLI installed), we can now run:
sam deploy --guided
This will walk you through the options to deploy the SAM application. We recommend 'Confirm changes before deploy' being set to 'Yes' so that you can see the resources that will be deployed and the changeset.
We can then see AWS deploy all of the resources, and the IAM Role, Lambda Function, API Gateway etc being provisioned.
Once this is complete, our app is up and running in AWS! But where is it?!
Well, let's add one more thing to our template output:
Outputs:
HelloWorldURL:
Description: "API endpoint URL for our Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
The !Sub function allows us to substitute the output from resources in our
CloudFormation template. By Default the API Gateway return value is it's ID.
So for the URL we take the ServerlessRestApi ID (the name of the API Gateway
resource once the template is expanded), the AWS Region our stack runs in and we
combine it with the Production stage of our API Gateway.
We then ask CloudFormation to output this URL as an output from our stack. With SAM this is printed after we deploy, saving us having to find the resource in our AWS account.
Great! We've just deployed our HTTP API. You've now got an application that massively scales without you having to be involved at all. The serverless ecosystem also includes the ability to create a SimpleTable to quickly provision a DynamoDB table, and of course you can hook up any other CloudFormation resource with only a little more work.
As we mentioned before, serverless isn't always the best choice, but having it in your toolbox can be a great plus.
Questions?
We've also added a few quick hints here for things that we wish we knew when we started using SAM.
How can I see the full Cloudformation template for a Serverless Application / Function?
You can use the sam validate --debug command to see how the AWS::Serverless
transform will translate the CloudFormation template. This lets you see the fully
expanded list of resources and names.
How can I prevent my build from being so large?
In the case of a JavaScript application, your NPM package file is respected.
You can use a .npmignore file or add a files section to your package.json
file.
How do I delete a deployment?
SAM Deployments are just CloudFormation stacks. At the time of writing, there's no SAM CLI command to delete the stack. However, you can use the AWS CLI or Console to delete your SAM application deployment.
Where does my code get uploaded?
AWS SAM creates an S3 bucket to upload your code to before it runs the CloudFormation template for your application. The bucket itself is created by another AWS CloudFormation stack.
The full URL to the code is included in the template that you'll see on the CloudFormation stack template panel.
You can visit: https://console.aws.amazon.com/cloudformation to see the stacks.
I get a build failed message when I try to build my container
When AWS builds the lambda function it has a set of steps called a workflow that it executes step by step. For the Node workflow this includes running a couple of NPM commands. If the correct version of Node isn't installed we'll see an error message.
Build Failed
Error: NodejsNpmBuilder:Resolver - Path resolution for runtime: nodejs14.x of binary: npm was not successful
To get around this we can use docker to build our application instead with the
--use-container flag:
sam build --use-container
If you have any questions, especially relating to using Serverless applications with our platform, feel free to contact us.
