Integrations

URLpipe in Pipedream

A dozen lines in a code step, and any URL in your workflow becomes Markdown, a screenshot or an audit.

By Roger Campos · Last updated: September 2026

TL;DR

In Pipedream, call URLpipe from a Node.js code step with fetch: POST to an endpoint such as https://urlpipe.dev/markdown with the key from an environment variable, and return the result for the next step. For slow work, a workflow with an HTTP trigger gives you a public endpoint to pass as report_to.

Start free See the API

Free plan, no credit card. 1,000 credits a month.

Step 1

Store the key

In Pipedream's Settings → Environment Variables, add URLPIPE_API_KEY with a project API key from the URLpipe dashboard. Code steps read it as process.env.URLPIPE_API_KEY, and it never appears in the workflow itself.

Step 2

A code step that reads a page

Add a Node.js code step after your trigger. This one reads the URL from the trigger's event and returns the Markdown as the step's export:

Node.js step — URL to Markdown

export default defineComponent({
  async run({ steps, $ }) {
    const url = steps.trigger.event.body?.url ?? "https://example.com";

    const res = await fetch("https://urlpipe.dev/markdown", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.URLPIPE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ url, sync: true }),
    });

    if (!res.ok) {
      const { error } = await res.json().catch(() => ({}));
      throw new Error(`URLpipe ${res.status}: ${error ?? "request failed"}`);
    }

    $.export("$summary", `Read ${url}`);
    return await res.text(); // the Markdown, as steps.<this_step>.$return_value
  },
});

Swap /markdown for /meta and return await res.json() to get metadata as an object; for /screenshot, return res.headers.get("X-Result-Url") — a link to the image that needs no key and stays valid for 30 days.

Step 3

Async: results into another workflow

A Lighthouse audit usually takes 15–20 seconds and a batch of pages longer. Instead of waiting in a step, create a second workflow with an HTTP / Webhook trigger, copy its endpoint URL, and send it as report_to:

Node.js step — start an audit, don't wait

export default defineComponent({
  async run({ steps }) {
    const res = await fetch("https://urlpipe.dev/lighthouse", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.URLPIPE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        url: steps.trigger.event.body.url,
        device: "mobile",
        report_to: "https://YOUR-ENDPOINT.m.pipedream.net",
        labels: { site: steps.trigger.event.body.site },
      }),
    });
    return await res.json(); // { token, status: "accepted", labels }
  },
});

In the receiving workflow, steps.trigger.event.body is the delivery: success, result (for Lighthouse, result.categories.performance.score and result.metrics), labels and error. The webhook must be a public https (or http) address on the default port — a URL with :5678 or any other port is refused, and so are IP addresses and localhost. Pipedream's endpoint URLs are HTTPS on the default port.

If you turn on webhook signing, the signature is over the exact bytes URLpipe sent, so verify against the raw body rather than re-serialized JSON — see signing & verifying.

Limits

What it costs, and what it doesn't do

Each call is billed at the usual rate: 1 credit for Markdown, 2 for a Lighthouse audit. Test runs that repeat a URL inside seven days are free cache hits.
The API allows 60 requests a minute and 15 per 10 seconds per project; space out loops over long lists.
There is no URLpipe component in Pipedream's registry yet; this recipe is plain fetch.

Endpoints

The endpoints on this page

/markdown API docs /lighthouse API docs

FAQ

Frequently asked questions

How do I keep my URLpipe key out of Pipedream code?

Add it as an environment variable under Settings → Environment Variables and read process.env.URLPIPE_API_KEY in the step.

How do I receive async URLpipe results in Pipedream?

Create a workflow with an HTTP trigger and pass its endpoint URL as report_to. The delivery arrives as steps.trigger.event.body.

Can I get a screenshot as a URL instead of Base64?

Yes. A sync /screenshot response carries an X-Result-Url header, a keyless link valid for 30 days; async deliveries carry it as result_url.

Does a code step's timeout matter?

It does for sync calls, which can take up to 60 seconds. Give the step enough time, or use async with an HTTP trigger for slow work.

Keep reading

Windsurf Convert a web page to Markdown in Node.js HTML to Markdown for LLMs Firecrawl alternatives Web pages into a RAG pipeline Turn any URL into clean Markdown All of integrations

Connect it in five minutes.

Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.

Start free Email the team