Logo

dev-resources.site

for different kinds of informations.

Folding as a Service with AWS StepFunctions

Published at
1/23/2023
Categories
folding
sam
aws
python
Author
sazecis
Categories
4 categories in total
folding
open
sam
open
aws
open
python
open
Author
7 person written this
sazecis
open
Folding as a Service with AWS StepFunctions

I am an AWS Solution Architect and I had been working with cloud technologies for many years. I had always been interested in using my technical skills to contribute to something meaningful. In 2020, when the COVID-19 pandemic was in full swing, I came across an article about the Folding@home project, which uses spare processing power of personal computers to perform complex simulations of protein folding. I realized that by using my own home computer, I could contribute to the project and help researchers understand how proteins fold and misfold, which in turn can aid in the development of new drugs and treatments for COVID-19 and other diseases.

One day, while I was thinking about how to increase my computer's processing power and efficiency, I had an idea: why not use the cloud? I knew that with the vast resources and flexibility of a cloud computing platform like Amazon Web Services (AWS), I could run simulations on a much larger scale and speed up the discovery process. So, I set up a virtual machine on AWS and started running the Folding@home simulations on it. The results were astounding, my computer's processing power increased, and I was able to perform more complex simulations and make new discoveries at a much faster rate. However, I soon realized that running Folding@home on the cloud was too expensive for my personal budget.

I couldn't let go of the idea. I thought that there might be other people or companies who would like to contribute with some compute resources but don't know how to use the folding. So, I came up with the idea of providing a "Folding as a Service" solution where contributors just specify the amount of money they want to donate and based on that, the infrastructure will be automatically generated and will run the foldings until the credit runs out. This way, I can help more people and organizations contribute to the Folding@home project and potentially researchers find cures for diseases more quickly.

Still in 2020, I took action and put the idea of "Folding as a Service" into practice. I built the service using Step Functions, Lambda, DynamoDB, EC2 and more. The Step Function was designed to take an input of credits in dollars and based on that, it would automatically start and configure a single EC2 instance. Several Lambda tasks were used to start and stop the Folding@home instance, monitor the performance of the instance, and make sure it was running efficiently. DynamoDB was used to keep track of the credit consumption.

I had the idea to put this whole service behind a website and create an easy-to-use user interface which would allow anyone to access the service and contribute to the Folding@home project with a few simple clicks, regardless of their technical expertise. I finished the backend service but because of lack of time did not start to work with the user interface. The solution stood unused until recently when I started to do some open-source contributions.

The other trigger which took me back to my dusty idea happened during the re:Invent conference, where I learned about a new service called AWS Application Composer. I wanted to test this service and explore its capabilities, and I decided to use my old Folding as a Service solution as a demo. I decided to recreate its infrastructure using the AWS Application Composer. Using the Application Composer wasn't so trivial but still it provided me with a good head start, and I was able to use the basic SAM template generated by it. I found that the Application Composer can be a great tool for serverless application designing, planning, and implementation, but as soon you need to include non-serverless resources, like in my case the tool can't be used any more. Still let me show you the initial diagram which I created:

Image description

The SAM template is a special CloudFormation template. CloudFormation is not my favorite tool, but still I decided to stick with it for this particular case. The reason for that was that I wanted to have a project using SAM.

Now these tools allowed me to create the infrastructure as code, update the scripts, and develop a working Folding as a Service solution that runs on EC2 spot instances, and can fold both with CPU or GPU setups. The solution is open-source, and anyone can clone my repository (https://github.com/sazecis/folding-as-a-service), build it, and deploy it to their own AWS account. If anyone is interested in more details on how to deploy the solution, they should read the README file of the repository or check out the diagram below:

Image description

After deployment, the folding process will not start automatically, it needs to be triggered by executing the Step Function and specifying the credit which you want to spend. This solution is ideal for individuals and organizations who have unused AWS credits or wish to contribute to the Folding@home project. The process of setting it up is straightforward, making it easy to contribute to the fight against diseases.

There are still many potentials in this solution, and it can be implemented for other Cloud Providers as well. Implementing a user-friendly interface would bring in more users and be a significant breakthrough. However, I am taking it step by step. And of course, if anyone is interested in joining the project, please feel free to contact me. Join the project and let's make a difference together.

sam Article's
30 articles in total
Favicon
Running lambdas locally using Javascript/Node.js
Favicon
Cut Your AWS Lambda Logging Costs: Filter Logs with AWS SAM
Favicon
Building a "Real-Time" Data Integration Platform on AWS
Favicon
Using Amazon Cognito with the user-password flow
Favicon
SAM Registration and Maintenance Ensuring Your Business Stays Compliant
Favicon
Utilizing the System for Award Management SAM for Government Contracting Success
Favicon
Secure API Gateway with Amazon Cognito using SAM
Favicon
Resources and Properties for AWS SAM
Favicon
Adding Cognito Authentication to our Serverless Dash App
Favicon
Using YAML anchors and aliases in a SAM template
Favicon
First impressions of CloudFormation’s IaC generator and CDK migrate
Favicon
Building Scalable Serverless Applications with AWS SQS and Lambda using SAM
Favicon
How to add CI/CD to my SAM project
Favicon
How to create serverless applications with AWS SAM (Serverless Application Model)
Favicon
Introduction to AWS SAM (Serverless Application Model)
Favicon
Help! How do I set DeletionPolicy to Retain for production only?
Favicon
An efficient way to build your serverless microservices. Part 3. CI/CD with AWS SAM.
Favicon
Leveraging Infrastructure as Code (IaC) for AWS Lambda: A Comparative Analysis of AWS SAM, Terraform, and Serverless Framework
Favicon
AWS Lambda with Rust and SAM
Favicon
Deploying Lambdas with AWS SAM & GitHub Actions: Step by Step
Favicon
Speed up new serverless application development with customized SAM templates
Favicon
Streamline AWS Development with CI/CD, SAM, and GitHub Actions
Favicon
AWS sam #3: sam local + ApiGateway Lambda authorizer
Favicon
✨ Porting Lambda Functions to AWS SAM
Favicon
Store Thumbnails from Your Live Stream Using AWS SAM CLI to Set Up Lambda Function and API Gateway
Favicon
AWS sam #2: sam local + logs
Favicon
AWS sam #1: sam local + DynamoDB
Favicon
Event-driven file management using S3 Notifications and Step Functions
Favicon
Folding as a Service with AWS StepFunctions
Favicon
Elevating Your Serverless Development with AWS SAM

Featured ones: