dev-resources.site

for different kinds of informations.

Build a highly scalable Serverless CRUD Microservice with AWS Lambda and the Serverless Framework

Published at

12/29/2024

Categories

4 categories in total

Author

13 person written this

ideepaksharma

open

Build a highly scalable Serverless CRUD Microservice with AWS Lambda and the Serverless Framework

In today’s cloud-native world, serverless architecture has become increasingly popular for building scalable and cost-effective applications. This blog post will guide you through creating a production-ready serverless CRUD (Create, Read, Update, Delete) microservice using AWS Lambda, DynamoDB, and the Serverless Framework.

Agenda:
Create a fully functional REST API that can:

Create new items
Retrieve items (both individual and list)
Update existing items
Delete items

Prerequisites:

Node.js installed on your machine
AWS account with appropriate permissions
Basic understanding of JavaScript/Node.js
AWS CLI installed and configured
Serverless Framework CLI installed

Architecture:
Below is the high level architectural diagram of the implementation.

AWS Lambda functions for each CRUD operation
DynamoDB for data storage
API Gateway for HTTP endpoints
Proper IAM roles and permissions
Cloudwatch logs to access the logs

Project Setup:
First, let’s create our project structure and install necessary dependencies:

mkdir serverless-crud-ms
cd serverless-crud-ms
npm init -y
npm install aws-sdk uuid

Once the above dependencies are installed, we proceed with the project structure creation which should look like below:

.
├── serverless.yml
├── handlers
│   ├── create.js
│   ├── get.js
│   ├── update.js
│   └── delete.js
└── package.json

Project Structure:

serverless.yml: Configuration for functions, resources, and permissions. It is the backbone of the application, defining our infrastructure as code.
handlers/: Contains Lambda function handlers
package.json: Project dependencies

Implementing CRUD Operations:
For each database operation a respective handler (.js) file is created under the handler directory. We will go through each one of them in corresponding sections. The names of the files are create.js, get.js, update.js, delete.js

Create Operation
To start with, a new file “create.js” is created for the POST operation, where a new item is inserted in the dynamo db table. The api uses the aws sdk client kit to insert records in dynamodb table. The operation performs the below:

Input validation
Error handling
CORS support
Unique ID generation
Timestamps for created/updated items Below is the code snippet for reference:

const AWS = require('aws-sdk');
const { v4: uuidv4 } = require('uuid');

const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.create = async (event) => {
    try {
        const timestamp = new Date().getTime();
        const data = JSON.parse(event.body);

        if (!data.name || !data.description) {
            return {
                statusCode: 400,
                headers: {
                    'Content-Type': 'application/json',
                    'Access-Control-Allow-Origin': '*'
                },
                body: JSON.stringify({
                    message: 'Missing required fields'
                })
            };
        }

        const params = {
            TableName: process.env.DYNAMODB_TABLE,
            Item: {
                id: uuidv4(),
                name: data.name,
                description: data.description,
                createdAt: timestamp,
                updatedAt: timestamp
            }
        };

        await dynamoDb.put(params).promise();

        return {
            statusCode: 201,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify(params.Item)
        };
    } catch (error) {
        console.error(error);
        return {
            statusCode: error.statusCode || 500,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify({
                message: error.message || 'Internal server error'
            })
        };
    }
};

Read Operation
To start with, a new file “get.js” is created for the GET operation, which fetches all the item records or a specific item record from the dynamo db table.

Below is the code snippet for reference:

const AWS = require('aws-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.getAll = async (event) => {
    try {
        const params = {
            TableName: process.env.DYNAMODB_TABLE
        };

        const result = await dynamoDb.scan(params).promise();

        return {
            statusCode: 200,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify(result.Items)
        };
    } catch (error) {
        console.error(error);
        return {
            statusCode: error.statusCode || 500,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify({
                message: error.message || 'Internal server error'
            })
        };
    }
};

module.exports.getOne = async (event) => {
    try {
        const params = {
            TableName: process.env.DYNAMODB_TABLE,
            Key: {
                id: event.pathParameters.id
            }
        };

        const result = await dynamoDb.get(params).promise();

        if (!result.Item) {
            return {
                statusCode: 404,
                headers: {
                    'Content-Type': 'application/json',
                    'Access-Control-Allow-Origin': '*'
                },
                body: JSON.stringify({
                    message: 'Item not found'
                })
            };
        }

        return {
            statusCode: 200,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify(result.Item)
        };
    } catch (error) {
        console.error(error);
        return {
            statusCode: error.statusCode || 500,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify({
                message: error.message || 'Internal server error'
            })
        };
    }
};

Update Operation
To start with, a new file “update.js” is created for the PUT operation, which updates an existing item record in the dynamo db table.

Below is the code snippet for reference:

const AWS = require('aws-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.update = async (event) => {
    try {
        const timestamp = new Date().getTime();
        const data = JSON.parse(event.body);

        if (!data.name || !data.description) {
            return {
                statusCode: 400,
                headers: {
                    'Content-Type': 'application/json',
                    'Access-Control-Allow-Origin': '*'
                },
                body: JSON.stringify({
                    message: 'Missing required fields'
                })
            };
        }

        const params = {
            TableName: process.env.DYNAMODB_TABLE,
            Key: {
                id: event.pathParameters.id
            },
            ExpressionAttributeNames: {
                '#item_name': 'name'
            },
            ExpressionAttributeValues: {
                ':name': data.name,
                ':description': data.description,
                ':updatedAt': timestamp
            },
            UpdateExpression: 'SET #item_name = :name, description = :description, updatedAt = :updatedAt',
            ReturnValues: 'ALL_NEW'
        };

        const result = await dynamoDb.update(params).promise();

        return {
            statusCode: 200,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify(result.Attributes)
        };
    } catch (error) {
        console.error(error);
        return {
            statusCode: error.statusCode || 500,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify({
                message: error.message || 'Internal server error'
            })
        };
    }
};

Delete Operation
To start with, a new file “delete.js” is created for the DEL operation, which deletes a specific item record from the dynamo db table whose id is provided as input.

Below is the code snippet for reference:

const AWS = require('aws-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.delete = async (event) => {
    try {
        const params = {
            TableName: process.env.DYNAMODB_TABLE,
            Key: {
                id: event.pathParameters.id
            }
        };

        await dynamoDb.delete(params).promise();

        return {
            statusCode: 204,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: ''
        };
    } catch (error) {
        console.error(error);
        return {
            statusCode: error.statusCode || 500,
            headers: {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            body: JSON.stringify({
                message: error.message || 'Internal server error'
            })
        };
    }
};

serverless.yml
The serverless.yml file contains the configuration of runtime, handler functions, aws region, and dynamoDB resources. Below is how my file looks like:

service: serverless-crud-ms

provider:
  name: aws
  runtime: nodejs18.x
  stage: ${opt:stage, 'dev'}
  region: eu-central-1
  environment:
    DYNAMODB_TABLE: ${self:service}-${self:provider.stage}
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource: "arn:aws:dynamodb:${self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"

functions:
  create:
    handler: handlers/create.create
    events:
      - http:
          path: items
          method: post
          cors: true

  getAll:
    handler: handlers/get.getAll
    events:
      - http:
          path: items
          method: get
          cors: true

  getOne:
    handler: handlers/get.getOne
    events:
      - http:
          path: items/{id}
          method: get
          cors: true

  update:
    handler: handlers/update.update
    events:
      - http:
          path: items/{id}
          method: put
          cors: true

  delete:
    handler: handlers/delete.delete
    events:
      - http:
          path: items/{id}
          method: delete
          cors: true

resources:
  Resources:
    ItemsTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.environment.DYNAMODB_TABLE}
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
        BillingMode: PAY_PER_REQUEST

Deployment:
Deploy the service using below commands:

# Configure AWS credentials
aws configure

# Deploy to AWS
serverless deploy

Note: You may encounter below error on deployment:
serverless.ps1 cannot be loaded. The file ..\npm\serverless.ps1 is not digitally signed.
To resolve this error, you may execute the below command and retry deployment:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

Once the deployment is successful, the API endpoints will be generated in the format like below:

Create new items (POST /items)
Retrieve all items (GET /items)
Get specific items by ID (GET /items/{id})
Update existing items (PUT /items/{id})
Delete items (DELETE /items/{id})

endpoints:
  POST - https://xxxxx.execute-api.eu-central-1.amazonaws.com/dev/items
  GET - https://xxxxx.execute-api.eu-central-1.amazonaws.com/dev/items
  GET - https://xxxxx.execute-api.eu-central-1.amazonaws.com/dev/items/{id}
  PUT - https://xxxxx.execute-api.eu-central-1.amazonaws.com/dev/items/{id}
  DELETE - https://xxxxx.execute-api.eu-central-1.amazonaws.com/dev/items/{id}

Testing the API
Use curl or Postman to test your endpoints:

# Create an item
curl -X POST https://your-api-url/dev/items \
  -H "Content-Type: application/json" \
  -d '{"name": "Test Item 1", "description": "This is a test item 1"}'

# Get all items
curl https://your-api-url/dev/items

# Get one item
curl https://your-api-url/dev/items/{id}

# Update an item
curl -X PUT https://your-api-url/dev/items/{id} \
  -H "Content-Type: application/json" \
  -d '{"name": "Updated Item 1", "description": "This is an updated item 1"}'

# Delete an item
curl -X DELETE https://your-api-url/dev/items/{id}

Cleanup
To remove all deployed resources:

serverless remove

This command will (check your AWS Console):

Delete the Lambda functions
Remove the API Gateway endpoints
Delete the DynamoDB table
Remove the IAM roles and policies
Clean up any CloudWatch log groups
Remove any other AWS resources that were created as part of your service

Conclusion
We’ve built a production-ready serverless CRUD microservice that’s scalable, maintainable, and follows best practices. This architecture can serve as a foundation for more complex applications, handling millions of requests while maintaining cost efficiency through the serverless model.

serverless Article's

30 articles in total