Introduction

In this blog post, we’ll explore setting up AWS SNS, Lambda, and EventBridge using the AWS CLI. While tools like Terraform offer comprehensive infrastructure management, the AWS CLI remains a crucial skill for DevOps professionals and developers. It enables quick, scriptable, and efficient resource management, making it a vital part of any DevOps toolkit. This guide is designed for beginners, providing a hands-on approach to harnessing the AWS CLI's power.

Why Use the AWS CLI?

Using the AWS CLI offers several advantages:
-- Quick Setup: Ideal for on-the-fly resource management without needing to write extensive code.
-- Scriptable Automation: Enables automation of repetitive tasks, enhancing productivity.
--Direct Integration: Can be easily integrated into scripts and pipelines, providing a direct interface with AWS services.
--Granular Control: Offers detailed control over AWS services, complementing infrastructure-as-code tools like Terraform.
--Essential Skill: Knowing the CLI deepens your understanding of AWS services, making you more versatile in managing cloud infrastructure.

Step-by-Step Guide

SNS Setup

1. Fetch AWS Account ID and Region

   aws sts get-caller-identity --query Account --output text
   aws configure get region

Verify: Ensure you see your AWS account ID and region.

2. Create an SNS Topic

   aws sns create-topic --name MyTopic

Verify:

   aws sns list-topics

Ensure your topic is listed.

3. Subscribe an Email Endpoint to the Topic

   aws sns subscribe --topic-arn arn:aws:sns:your-region:your-account-id:MyTopic --protocol email --notification-endpoint [email protected]

Verify:

   aws sns list-subscriptions-by-topic --topic-arn arn:aws:sns:your-region:your-account-id:MyTopic

Ensure your subscription is listed.

4. Publish a Message to the Topic

   aws sns publish --topic-arn arn:aws:sns:your-region:your-account-id:MyTopic --message "Hello, this is a test message!"

Verify: Check your email for the message.

Common Errors and Troubleshooting: SNS Setup

• SNS Topic Creation Error: If you encounter a "Throttling" error, ensure you are not exceeding AWS request limits. Consider adding retries with exponential backoff.
• Email Subscription Confirmation Not Received: Check your spam folder, and ensure your email address is correct in the subscription command.

Lambda Setup

1. Create the IAM Role for Lambda Execution

   aws iam create-role --role-name LambdaExecutionRole --assume-role-policy-document '{
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "Service": "lambda.amazonaws.com"
         },
         "Action": "sts:AssumeRole"
       }
     ]
   }'

Verify:

   aws iam get-role --role-name LambdaExecutionRole

Ensure the role details are displayed.

2. Attach the Policy to Allow Publishing to SNS

   aws iam put-role-policy --role-name LambdaExecutionRole --policy-name LambdaSNSPolicy --policy-document '{
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": "sns:Publish",
         "Resource": "arn:aws:sns:your-region:your-account-id:MyTopic"
       }
     ]
   }'

Verify:

   aws iam get-role-policy --role-name LambdaExecutionRole --policy-name LambdaSNSPolicy

Ensure the policy details are displayed.

3. Attach the AWSLambdaBasicExecutionRole Policy

   aws iam attach-role-policy --role-name LambdaExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Verify:

   aws iam list-attached-role-policies --role-name LambdaExecutionRole

Ensure the policy is listed.

4. Zip the Python File

   cd /path/to/your/python/file
   zip function.zip lambda_function.py

Verify: Ensure function.zip is created in the directory.

Common Errors and Troubleshooting: Lambda and EventBridge Setup

• Lambda Permission Errors: Ensure the Lambda execution role has the correct permissions.
• EventBridge Target Errors: If the target is not added, double-check the ARN and ensure the Lambda function exists.

5. Create the Lambda Function

   aws lambda create-function --function-name MyLambdaFunction --runtime python3.8 --role arn:aws:iam::your-account-id:role/LambdaExecutionRole --handler custom_handler_name.lambda_handler --zip-file fileb://function.zip

Note: Replace custom_handler_name with your desired handler name. The handler name should match the filename and function name in your Python code.

Verify:

   aws lambda get-function --function-name MyLambdaFunction

Ensure the function details are displayed.

6. Add Environment Variables to Lambda

   aws lambda update-function-configuration --function-name MyLambdaFunction --environment "Variables={API_KEY=XXXXXXXXXXX,SNS_TOPIC_ARN=arn:aws:sns:your-region:your-account-id:MyTopic}"

Note: Replace XXXXXXXXXXX with the value of your API Key.
Verify:

   aws lambda get-function-configuration --function-name MyLambdaFunction

Ensure the environment variables are listed.

7. Add SNS Trigger to Lambda

   aws lambda add-permission --function-name MyLambdaFunction --statement-id sns-invoke --action "lambda:InvokeFunction" --principal sns.amazonaws.com --source-arn arn:aws:sns:your-region:your-account-id:MyTopic

Verify:

   aws sns list-subscriptions-by-topic --topic-arn arn:aws:sns:your-region:your-account-id:MyTopic

Ensure the Lambda function is listed as a subscriber.

8. Create a Directory for Tests

   mkdir tests
   cd tests

9. Create test_event.json

   {
       "key1": "value1",
       "key2": "value2",
       "key3": "value3"
   }

10. Create output.json
This file will be used to store the output of the Lambda function when you test it. Ensure this file is in the tests directory.

11. Test the Lambda Function

```sh
aws lambda invoke --function-name MyLambdaFunction --payload file://tests/test_event.json tests/output.json
```

12. Verify the Output

```sh
cat tests/output.json
```

PS: If you don't have a Python script, check out my GitHub repository here. It contains all the necessary files and instructions to get started. The scripts use NBA-specific API URLs to ensure you receive relevant data for your application.

EventBridge Setup

1. Create the EventBridge Rule

   aws events put-rule --name MyScheduledRule --schedule-expression "cron(0 12 * * ? *)"

Verify:

   aws events list-rules --name-prefix MyScheduledRule

Ensure the rule is listed.

2. Add Lambda as a Target for the Rule

   aws events put-targets --rule MyScheduledRule --targets "Id"="1","Arn"="arn:aws:lambda:your-region:your-account-id:function:MyLambdaFunction"

Verify:

   aws events list-targets-by-rule --rule MyScheduledRule

Ensure the Lambda function is listed as a target.

3. Invoke the Lambda function

   aws lambda invoke --function-name MyLambdaFunction output.txt

Ensure the message is sent to SNS.

Real-World Use Case: Automated Monitoring and Alerts

Imagine setting up an automated alert system for your application logs. By using Lambda and SNS, you can trigger alerts directly from your CloudWatch logs, ensuring immediate action on critical events.

Security Best Practices

• Least Privilege Principle: Assign only necessary permissions to IAM roles.
• Regular Audits: Frequently review your IAM policies for compliance and security.

Conclusion

The AWS CLI is more than a convenience tool; it’s a critical component of your DevOps toolkit. While tools like Terraform excel at managing infrastructure at scale, the CLI offers quick, scriptable access to AWS services, making it invaluable for tasks that require immediate action or detailed control. By mastering the CLI, you enhance your ability to manage cloud resources efficiently, making it an essential skill for modern cloud engineers.