For the past couple of months I've been building out a low code/no code AI/ML Platform that lets users deploy their ML models as an API. Imagine the following scenario: you've trained a deep learning model and pushed it to Huggingface. While the model is great for what you trained it for, it lives inside a jupyter notebook but you need a deployable API where users can directly interact with your model. Or even better, you want to hook your model up with one of the LLM's out there but you don't want to spend time coding up the API, think about integration code, etc. That's essentially what I'm trying to solve.

From the user point of view, you see a flowchart like tool, where you configure your blocks(in the case of Huggingface, you'd paste in your model card), connect them to their input blocks(they essentially dictate whether your model accepts a text or image input or both), and hit create, which then deploys a docker container running a FastAPI server for you to then query. Under the hood, there is a graph being created with each node in the graph loading up the resources needed to run your workload. And in this article, I want to talk about the breadth first search algorithm and how it powers my project.

Now consider the following workflow that runs a Financial Bert model for Sentiment Analysis. You have 3 blocks - the input block which will take your input text, the ML block and the output block which will display the end result.

The first step, would be to take this UI and translate it into format that would speak to the backend. This could look something like:

{
  "input": [
      "Input_Block": {
          "run_config": "All user based config",
          "connections": [ "process.Financial_BERT" ],
          "payload": "<Insert Positive news for financial BERT>",
          "implementation": Input_Implementation_Class()
      }
  ],
  "process": [
      "Financial_BERT": {
          "run_config": "All user based config",
          "connections": [ "output.Output_Block" ],
          "payload": "<Insert Positive news for financial BERT>",
          "implementation": HF_MODEL_Class()
      }
  ],
  "output": [
      "Output_Block": {
          "run_config": "All user based config",
          "connections": []
      }
  ]
}

The next step would be taking the payload of one block, passing it into the implementation of another, and you'd know the next block via the connections list. Now, naively you could implement an iterative loop that starts at the input block, traverses its connection by passing the payload around. It would look something like this:

start_node = workflow['input']['Input_Block']

for connection in start_node['connection']:
    # pass payload to process.Financial_BERT.implementation

This approach would work only if we had a linear workflow, where 1 block connects to only one block. But in the real world, one could have N blocks connecting to 1 block, wherein the following block expects inputs from all N blocks before running itself.

The orange block needs inputs A,B and C as params so it can run.So how do we ensure inputs from block A,B and C is passed altogether in the orange block? This where the breadth first search comes to the rescue.

Source: https://commons.wikimedia.org/wiki/File:Breadth-First-Search-Algorithm.gif

The way the BFS algorithm works is by starting at some node, source, it first looks at all its neighbors and traverses the graph in a layer wise fashion. Here's an excerpt from a Hackerearth article that sums it up best:

Source: https://www.hackerearth.com/practice/algorithms/graphs/breadth-first-search/tutorial/

So coming back to our use case, we can do something like this:

We can start at some dummy node, look at all it's immediate neighbors (A,B, and C) and process them, and only when all 3 are processed is when we move on the next layer where the orange block is and pass the 3 block's output as the orange block's input. The way we implement this is through a queue that processes nodes in the order they arrived in. We'll also keep track of all the nodes we already visited and processed so we don't end up reprocessing them. Below is a step by step flow of how its implemented.

Step by step flow of how I used BFS to process nodes

Here's the code block that I used to incorporate BFS in my project:

Source: https://github.com/farhan0167/otto-m8

And if you notice, it looks a lot like what you'd find on Wikipedia on the BFS Algorithm:

Source: https://en.wikipedia.org/wiki/Breadth-first_search

Now that we have a way to pass payloads from one block to another, there's a whole world of possibilities of what can be implemented. At the moment otto-m8 (which translates to automate), aims to provide all these blocks which implement some form of Deep Learning model. It supports LLM's(OpenAI and Ollama) and Hugging Face models. When you create a workflow, you get a deployable Docker container with an exposed REST endpoint. And at its core (obviously an oversimplification) the BFS algorithm paves the way for a user input to pass through all the chained models in your workflow.
Thank you for reading, especially if you've followed along all the way. Please feel free to try out otto-m8 which can be found here on Github: https://github.com/farhan0167/otto-m8, accompanied by the documentation here at https://otto-m8.com