Intro

In my last post, I walked through a coding solution in which we used Node.js to scrape webpages returned in Google Search results and parse them to Markdown for use in LLMs or other use cases. Today, we will do something similar, but we will utilize no-code solutions to return results from SerpApi’s Google Search API and parse to markdown using the Reader API by Jina.ai.

Turning Search Results Into Markdown for LLMs ↗

Reader by Jina

Jina AI’s Reader API (https://jina.ai/reader/) offers an alternative for extracting structured content from web pages, maintaining document structure, and handling diverse content types.

💡 Jina AI provides 1 million test credits to get started, but keep in mind that tokens are used based on the amount of content parsed, not the number of requests made.

Scraping Search Results

Several options integrate well with SerpApi for those who prefer no-code solutions. This section will walk through configuring Make and Zapier to query search results through SerpApi.

Make

Starting with Make, we will create a Google Doc from the data returned by SerpApi and the Reader API. This is a relatively simple example, but Make provides many integrations with AI providers such as OpenAI, Anthropic, and more. Instead of sending the data to a Google Doc, you can replace the last module and send it wherever you need it!

SerpApi has an official Make app, making it much more straightforward than Zapier to configure. While I will only be walking through how to configure the SerpApi Google Search module, for more information on our Make app, check out this blog post:

Announcing SerpApi’s Make App ↗

Create a new scenario and search for the SerpApi app to get started. Select your preferred search engine module; we will use Google for this example.

After selecting your search engine, we can start configuring our call to SerpApi. You’ll add your SerpApi API key by creating a connection. Click “Create a connection.”

Enter your API key and name your connection:

From here, we can configure our query:

You can fetch additional pages by increasing the “Pagination Limit” setting but note that each page will cost 1 SerpApi search credit.

Click “Run once” and verify the data coming back from SerpApi:

Now that we have results we can work with, let’s add an “Iterator” module to allow us to use the data returned in the organic_results array. The Iterator module (documentation) converts arrays into a series of bundles. Each element in the array will output as a separate bundle, allowing us to query the Reader API for each link returned in our Google results.

To query the Reader API for each link returned, let’s add an HTTP “Make a request” module that allows us to send HTTP requests.

Set the URL parameter to https://r.jina.ai/, then click the box again to pull up a list of available data we can pass to this parameter. From our Iterator module, select the “link” attribute. This passes the link from our organic Google search results to the Reader URL. Here’s what that should look like:

Now, we need to set our header for authorization by passing our Bearer token provided by Jina AI:

Before running again, let’s add our final step to create a Google Doc to store the results. Again, this final step can be changed depending on your use case and where you wish to send the data.

Add the Google Docs “Create a Document” module and add a connection to your Google account. Now you can set a name for the document as well as pass data from the Reader API as the content of the doc:

Now lets run our scenario once more to ensure that everything is working correctly.

While testing, we can set a limiter filter after the Iterator module to prevent the flow from continuing more than once.

Click the wrench icon under the link between the Iterator and HTTP modules. Create a condition only to continue if the Iterator’s “position” attribute is numerically equal to “1”:

Make sure to remove the limiter before publishing if you wish to query for all the results. Alternatively, you set the num parameter in the SerpApi Search Google module to limit results returned by SerpApi.

Zapier

While this article is focused on no-code solutions, SerpApi currently does not have a Zapier integration. Meaning we must use a little code to make our API requests to SerpApi. I have provided a code below that you can drop into Zapier Code steps without issue, though some adjustments may be needed to fit your use case better. I recommend starting with the code I’ve provided and then adjusting once you get results from SerpApi and the Reader API.

In this section, we will walk through:

• Use the Schedule Trigger to initiate Zap
• Retrieve search results URLs from SerpApi
• Process webpages from URLs through Reader API to Markdown
• Send formatted content to your desired apps

As this Zap requires API calls to both SerpApi and the Reader API, you will likely need a Trial or Professional/Team plan. Zapier limits code runtime to 1 second on the free plan, and both API requests will likely take longer (documentation).

To get started, add a trigger to your Zap. For this post, we will use the Schedule by Zapier trigger to our Zap every day at 12:00 a.m. You can select a trigger that best fits your needs.

💡 Make sure to test each step before continuing to ensure you’ve created test data for future steps. Zapier will typically force you to test a step after changes before publishing.

Next, we add a Code by Zapier action to make our request to SerpApi. This example uses JavaScript, but you can choose Python if you prefer to use the code below as a guideline.

I’ve included the JavaScript code for this action below:

const { SERPAPI_KEY, QUERY, LOCATION } = inputData;

async function fetchGoogleSearchResults(query, location) {
  try {
    const response = await fetch(
      `https://serpapi.com/search.json?engine=google&q=${encodeURIComponent(
        query
      )}&location=${encodeURIComponent(
        location
      )}&gl=us&hl=en&api_key=${SERPAPI_KEY}`
    );
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    const data = await response.json();
    if (!data) {
      throw new Error("No data returned from SerpAPI");
    }
    return data;
  } catch (error) {
    // Instead of just logging, throw the error
    throw new Error(`SerpAPI request failed: ${error.message}`);
  }
}

try {
  const searchResults = await fetchGoogleSearchResults(QUERY, LOCATION);
  output = {
    organicResults: searchResults?.organic_results || "No Organic Result Found"
  };
  return output;
} catch (error) {
  console.error("Error:", error.message);
  // Return a default response or throw error based on your needs
  return { organic_results: [] };
}

After adding the above code to the action, add your input data:

• SERPAPI_KEY - your SerpApi key goes here. You can find your API key in your SerpApi dashboard.
• QUERY - query parameter for the search you wish to perform.
• LOCATION - location parameter for the search you wish to perform.

If you wish to add additional search parameters, you can set them as Input Data or update the URL in the code directly. Setting them as Input Data allows you to update them more easily in the future.

Your first code step should now look something like this:

Our request from SerpApi will return an array of links for our organic results. To send each link to the Reader API, we must iterate through the links using the Looping by Zapier action.

This action takes an array of values and runs the actions within the loop for each value. For our “Values to Loop,” we can pass the “Organic Results Link” values from our second step to a key called “links”.

Let’s also set the maximum number of loop iterations to 1 for testing to ensure we aren't spending Reader API credits needlessly.

Inside the Looping by Zapier action, we can add another Code by Zapier action to make our request to the Reader API. Paste the JavaScript below in the code section of this action:

const { link, READERAPI_KEY } = inputData;

async function fetchReaderData(link, apiKey) {
  try {
    // Encode URL for path parameter
    const encodedUrl = encodeURIComponent(link);
    const response = await fetch(`https://r.jina.ai/${encodedUrl}`, {
      method: 'GET',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Accept': 'application/json'
      }
    });
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    const data = await response.json();
    if (!data) {
      throw new Error('No data returned from Reader API');
    }
    return data;
  } catch (error) {
    console.error('Reader API request failed:', error.message);
    throw error;
  }
}

try {
  const readerResults = await fetchReaderData(link, READERAPI_KEY);
  output = {
    readerResults
  };
  return output;
} catch (error) {
  return {
    error: error.message,
    content: null,
    title: null,
    metadata: null
  };
}

Again, set your Input Data for this action:

• link - the value of the link from our Looping action.
• READERAPI_KEY - your Jina AI Reader API key, found on the Reader API website.

Once complete, your second Code by Zapier action should look like this:

Finally, we can send our data from the Reader API to an app. For this example, we will use Google Docs, but feel free to explore Zapier’s integrations and find the action that best fits your use case. I recommend trying with Google Docs first to better understand the output data.

Add the “Create Document From Text in Google Docs” action to your Loop. This will create a new Google Doc for each URL the Loop processes. Feel free to adjust as needed.

If you have not done so previously, you may need to authorize Zapier to access your Google account. Once complete, your Setup tab should look something like this:

Click “Continue” or navigate to the “Configure” tab. Here, we can set a name for the document and the Google Drive folder in which you would like to save the document and pass the content returned from the Reader API to the document’s content.

And you’re all set! Make sure to test the Zap thoroughly to ensure there are no issues, and check your Google Drive to ensure the document was created.

Conclusion

No-code tools such as Make and Zapier provide powerful tools for users who might not be comfortable writing or maintaining scripts necessary to scrape valuable data found in search results. While SerpApi isn’t necessarily a no-code solution, it abstracts away many complex concepts required to scrape search engines. When paired with no-code tools, even non-technical users can leverage this data.