dev-resources.site
for different kinds of informations.
How to run WebAssembly/WASI application (Spin) with Dapr on Kubernetes
Introduction
As far as we know, WebAssembly (WASM) / WebAssembly System Interface (WASI) is gaining traction these days. It comes with a lot of benefits such as faster startup times (no cold-start like serverless, start-up around milliseconds), near-native performance (WASM format), lightweight (very small sizing), convenience and versatility (build once run everywhere promise), security (sandbox by default).
But WASM/WASI itself is only strong for the computing workload (e.g. int + int => int) just like serverless v2 in the cloud computing world. It is appropriate for heavy and ephemeral calculation/transformation tasks or some kind of trigger tasks by external components such as cloud services. And it might delegate persistent or message communication tasks to its old brothers (containers). That's the reason Containers and WebAssembly should come and complement together, not suited for stand-alone tasks when we intend to use it to build cloud applications.
Having said that, we can use many of the existing containers built by famous companies like Redis, Kafka, and RabbitMQ... Then we can run WebAssembly (WASI) components on Kubernetes with containerd-wasm-shims. But with the very limitation of components for persistence data, message broker, networking management, etc. we might think about how can we leverage some kind of OSS from CNCF like Dapr or KEDA to amplify its power. Imagine that we can use Dapr for service invocation, data binding, pub/sub, distributed workflow, you name it, so we can build out some battery-included applications on the cloud.
This article is the first step in addressing the described above. And it works because of the release of containerd-wasm-shims v0.9.0 just yesterday by DeisLabs team. In this release, there is a very cool feature called Linux Container Side-by-Side in a Pod: You can now run Linux containers alongside WebAssembly containers within the same Kubernetes Pod, and we can use it to run the Dapr sidecar model which runs side-by-side with Spin component. We will deep dive into how to make it work in a minute.
Sample application (Spin)
In this post, we would like to introduce a very simple application written on Spin which returns a list of products (drinks or foods) as below.
GET {{host}}/v1-get-item-types HTTP/1.1
content-type: application/json
[
{
"image": "img/CAPPUCCINO.png",
"itemType": 0,
"name": "CAPPUCCINO",
"price": 4.5
},
// ...
{
"image": "img/CROISSANT_CHOCOLATE.png",
"itemType": 9,
"name": "CROISSANT_CHOCOLATE",
"price": 3.5
}
]
The code in Rust language:
use anyhow::Result;
use bytes::Bytes;
use serde::{Serialize, Deserialize};
use serde_json::json;
use spin_sdk::{
http::{Params, Request, Response},
http_component,
};
#[derive(Debug, Serialize, Clone)]
#[serde(rename_all = "camelCase")]
struct ItemType {
name: String,
item_type: i8,
price: f32,
image: String,
}
#[derive(Debug, Deserialize)]
struct GetItemByTypeModel{
types: String,
}
impl TryFrom<&Option<Bytes>> for GetItemByTypeModel {
type Error = anyhow::Error;
fn try_from(value: &Option<Bytes>) -> std::result::Result<Self, Self::Error> {
match value {
Some(b) => Ok(serde_json::from_slice::<GetItemByTypeModel>(b)?),
None => Err(anyhow::anyhow!("No body")),
}
}
}
#[http_component]
fn handle_product_api(req: Request) -> Result<Response> {
println!("{:?}", req.headers());
let mut router = spin_sdk::http::Router::default();
router.get("/", health_handler);
router.get("/v1-get-item-types", get_item_types_handler);
router.get("/v1-get-items-by-types", get_item_by_types_handler);
router.handle(req)
}
fn health_handler(_req: Request, _params: Params) -> Result<Response> {
Ok(http::Response::builder()
.status(200)
.body(Some("".into()))?)
}
fn get_item_types_handler(_req: Request, _params: Params) -> Result<Response> {
let items = json!(get_item_types());
let result = bytes::Bytes::from(items.to_string());
Ok(http::Response::builder()
.header("Content-Type", "application/json")
.status(200)
.body(Some(result))?)
}
fn get_item_by_types_handler(req: Request, _params: Params) -> Result<Response> {
let Ok(model) = GetItemByTypeModel::try_from(&req.body().clone()) else {
return Ok(http::Response::builder()
.status(http::StatusCode::BAD_REQUEST)
.body(None)?);
};
let mut temp: Vec<ItemType> = Vec::new();
for i in get_item_types() {
let parts = model.types.split(',');
let ii = i.clone();
for j in parts {
if ii.item_type.to_string().as_str() == j {
temp.push(ii.clone())
}
}
}
let result = bytes::Bytes::from(json!(temp).to_string());
Ok(http::Response::builder()
.header("Content-Type", "application/json")
.status(200)
.body(Some(result))?)
}
fn get_item_types() -> Vec<ItemType> {
vec![
ItemType {
name: "CAPPUCCINO".to_string(),
item_type: 0,
price: 4.5,
image: "img/CAPPUCCINO.png".to_string(),
},
// ...
// ...
ItemType {
name: "CROISSANT_CHOCOLATE".to_string(),
item_type: 9,
price: 3.5,
image: "img/CROISSANT_CHOCOLATE.png".to_string(),
},
]
}
The whole source code for product-api can be found at https://github.com/thangchung/dapr-labs/blob/feat/spin-refactor/polyglot/product-api/src/lib.rs
Package the Spin application into Container
To make it work on Kubernetes, we need to package the application with docker buildx and the target should be wasi/wasm. Let's do it below
Login into GitHub artifact hub:
docker login ghcr.io -u <your username>
It asks you to provide the password (PAT), please go to your developer profile to generate it.
Then we are ready to build the wasm/wasi container:
FROM --platform=${BUILDPLATFORM} rust:1.67 AS build
RUN rustup target add wasm32-wasi
COPY . /product
WORKDIR /product
RUN cargo build --target wasm32-wasi --release
FROM scratch
COPY --from=build /product/target/wasm32-wasi/release/product_api.wasm /target/wasm32-wasi/release/product_api.wasm
COPY ./spin.toml /spin.toml
Let builds the image above:
cd product-api
docker buildx build -f Dockerfile --platform wasi/wasm,linux/amd64,linux/arm64 -t ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.1 . --push
This action will build everything that is needed for the Spin app can run on a container, and then push it into the container hub. After that, you can see it at https://github.com/thangchung/dapr-labs/pkgs/container/dapr-labs%2Fproduct-api-spin/124547580?tag=1.0.1
Prepare your Kubernetes (K3d) cluster
Install k3d into your Ubuntu:
wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
Make it shim to be able to run WASM/WASI workloads (slight, spin, wws, lunatic):
k3d cluster create wasm-cluster --image ghcr.io/deislabs/containerd-wasm-shims/examples/k3d:v0.9.0 -p "8081:80@loadbalancer" --agents 2
kubectl apply -f https://github.com/deislabs/containerd-wasm-shims/raw/main/deployments/workloads/runtime.yaml
kubectl apply -f https://github.com/deislabs/containerd-wasm-shims/raw/main/deployments/workloads/workload.yaml
echo "waiting 5 seconds for workload to be ready"
sleep 15
curl -v http://127.0.0.1:8081/spin/hello
curl -v http://127.0.0.1:8081/slight/hello
curl -v http://127.0.0.1:8081/wws/hello
curl -v http://127.0.0.1:8081/lunatic/hello
If you can curl it with 200 statuses, then everything is okay. Move on!
Refs: https://github.com/thangchung/containerd-wasm-shims/blob/main/deployments/k3d/README.md
Install Dapr 1.11.2 into K3d
In this post, we use Dapr v1.11.2, and you should see as:
dapr --version
CLI version: 1.11.0
Runtime version: 1.11.2
For a demo purpose only, so we install it via Dapr CLI to k3d cluster (on production, we might need to use Dapr Helm chart with HA mode):
dapr init -k --runtime-version 1.11.2
Wait a second for Dapr installed on your cluster, then we continue to install Redis component into K3d cluster as well (not need redis for now, but might need in the next posts).
helm install my-redis oci://registry-1.docker.io/bitnamicharts/redis --set architecture=standalone --set global.redis.password=P@ssw0rd
Now we create some Dapr components which bind with Redis above:
kubectl apply -f components-k8s/
Query it:
kubectl get components
Should return:
NAME AGE
baristapubsub 16h
kitchenpubsub 16h
statestore 16h
Okay, let's move on to the final step.
Run Spin app with Dapr on Kubernetes (k3d)
We create the yaml file as below:
apiVersion: apps/v1
kind: Deployment
metadata:
name: product-api
spec:
replicas: 1
selector:
matchLabels:
app: product-api
template:
metadata:
labels:
app: product-api
annotations:
(1) dapr.io/enabled: "true"
(1) dapr.io/app-id: "product-api"
(1) dapr.io/app-port: "80"
(1) dapr.io/enable-api-logging: "true"
spec:
(2) runtimeClassName: wasmtime-spin
containers:
- name: product-api
image: ghcr.io/thangchung/dapr-labs/product-api-spin:1.0.1
command: ["/"]
env:
- name: RUST_BACKTRACE
value: "1"
resources: # limit the resources to 128Mi of memory and 100m of CPU
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: product-api
spec:
type: LoadBalancer
ports:
- protocol: TCP
port: 5001
targetPort: 80
selector:
app: product-api
(1): daprized the application so that Dapr control plane will inject the sidecar automatically for us.
(2): Wasmtime with Spin is chosen for this application.
Then we apply it to create the product-api:
kubectl apply -f iac/kind-spin/product-api-deploy.yaml
Check it work:
kubectl get po
You should see:
NAME READY STATUS RESTARTS AGE
my-redis-master-0 1/1 Running 8 (134m ago) 16h
product-api-8ccbc56b-gvlc2 2/2 Running 0 78m
If you notice, now we can have a sidecar (2/2) work very well with Spin app (WASM/WASI). Thanks, Mossaka and the Deislabs team for working very hard on it.
If you tail the logs of product-api-daprd, you should see:
kubectl logs pod/product-api-8ccbc56b-dxfmj --namespace=default --container=daprd --since=0
...
time="2023-09-04T08:47:10.311158673Z" level=info msg="Dapr trace sampler initialized: DaprTraceSampler(P=0.000100)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.311260921Z" level=info msg="Initialized name resolution to kubernetes" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.311556754Z" level=info msg="Loading components…" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.313604323Z" level=info msg="Component loaded: kubernetes (secretstores.kubernetes/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.318570002Z" level=info msg="Component loaded: baristapubsub (pubsub.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.321000955Z" level=info msg="Component loaded: kitchenpubsub (pubsub.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.3210958Z" level=info msg="Waiting for all outstanding components to be processed" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.325038524Z" level=info msg="Using 'statestore' as actor state store" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:10.325088749Z" level=info msg="Component loaded: statestore (state.redis/v1)" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
...
time="2023-09-04T08:47:10.352600037Z" level=info msg="application protocol: http. waiting on port 80. This will block until the app is listening on that port." app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
...
time="2023-09-04T08:47:16.169135624Z" level=info msg="actor runtime started. actor idle timeout: 1h0m0s. actor scan interval: 30s" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime.actor type=log ver=1.11.2
time="2023-09-04T08:47:16.169271184Z" level=info msg="Configuring workflow engine with actors backend" app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime.wfengine type=log ver=1.11.2
time="2023-09-04T08:47:16.169288176Z" level=info msg="Registering component for dapr workflow engine..." app_id=product-api instance=k3d-wasm-cluster-agent-1 scope=dapr.runtime type=log ver=1.11.2
time="2023-09-04T08:47:16.169440257Z" level=info msg="initializing Dapr workflow component" app_id=product-api component="dapr (workflow.dapr/v1)" instance=k3d-wasm-cluster-agent-1 scope=dapr.contrib type=log ver=1.11.2
time="2023-09-04T08:47:16.17126385Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 5864ms"
Amazing, it could daprized and run product-api successfully. I struggled to make it work for a month (before containerd-wasm-shims v0.9.0), and now it is like a dream 🙌.
Now we get the services:
kubectl get svc
You should get:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 16h
my-redis-headless ClusterIP None <none> 6379/TCP 16h
my-redis-master ClusterIP 10.43.109.123 <none> 6379/TCP 16h
product-api-dapr ClusterIP None <none> 80/TCP,50001/TCP,50002/TCP,9090/TCP 81m
product-api LoadBalancer 10.43.4.58 172.19.0.2,172.19.0.3,172.19.0.4 5001:32134/TCP 81m
For demo purpose, we do a port forward our product-api: 10.43.4.58 on 5001 port to host machine.
And using curl, we can achive our purpose like a breeze:
###
GET http://localhost:5001/v1-get-item-types HTTP/1.1
content-type: application/json
Return
HTTP/1.1 200 OK
content-type: application/json
content-length: 776
date: Mon, 04 Sep 2023 08:27:42 GMT
[
{
"image": "img/CAPPUCCINO.png",
"itemType": 0,
"name": "CAPPUCCINO",
"price": 4.5
},
...
{
"image": "img/CROISSANT_CHOCOLATE.png",
"itemType": 9,
"name": "CROISSANT_CHOCOLATE",
"price": 3.5
}
]
Or,
###
GET http://localhost:5001/v1-get-items-by-types HTTP/1.1
content-type: application/json
{
"types": "1,2,3"
}
Return:
HTTP/1.1 200 OK
content-type: application/json
content-length: 241
date: Mon, 04 Sep 2023 08:29:19 GMT
[
{
"image": "img/COFFEE_BLACK.png",
"itemType": 1,
"name": "COFFEE_BLACK",
"price": 3.0
},
{
"image": "img/COFFEE_WITH_ROOM.png",
"itemType": 2,
"name": "COFFEE_WITH_ROOM",
"price": 3.0
},
{
"image": "img/ESPRESSO.png",
"itemType": 3,
"name": "ESPRESSO",
"price": 3.5
}
]
Ta-da!!! It worked ❤️
This is just the beginning of a series of posts about how can we run WASM/WASI with Dapr on Kubernetes. More fun to come soon.
The source code of this sample can be found at https://github.com/thangchung/dapr-labs/tree/main/polyglot
WebAssembly, Docker container, Dapr, and Kubernetes better together series: part 1, part 2, part 3, part 4.
Featured ones: