🌐Fetching Open Data

Data Requests enable you to retrieve information from public API endpoints that do not require sensitive information like API keys. For APIs that do require keys, see Advanced: API-key Gated Data.

In this walkthrough, we'll use the SEDA Request Starter Kit, which includes all the necessary dependencies.

HTTP Fetch: GET

The httpFetch method is available only during the execution phase. Using it in the tally phase will trigger an error due to the non-deterministic nature of httpFetch. To make an HTTP request, we use the httpFetch method:

import { Console, httpFetch, OracleProgram } from "@seda-protocol/as-sdk/assembly";

export class MyDataRequest extends OracleProgram {
  execution(): void {
    // Call the API swapi.dev and return an HttpResponse object
    const response = httpFetch("https://swapi.dev/api/planets/1");
    
    // Ensure the fetch call has succeeded
    if (!response.ok) {
      Process.error(Bytes.fromUtf8String("Could not fetch API endpoint"));
    }

    Console.log(response.bytes.toUtf8String());
  }
  
  tally(): void {
	  throw new Error("Not implemented");
  }
}

new MyDataRequest().run()

Running this program with the following test will output the result of the execution, showing the httpFetch response as a string. You can place this test in tests/my-dr.test.ts:

import { describe, it} from "bun:test";
import { file } from "bun";
import { testOracleProgramExecution } from "@seda-protocol/dev-tools"

describe("data request execution", () => {
  it("should run MyDataRequest", async () => {
    const wasmBinary = await file("build/debug.wasm").arrayBuffer();

    // Locally executes the Data Request
    const vmResult = await testOracleProgramExecution(
      Buffer.from(wasmBinary),
      // This program doesn't have any inputs
      Buffer.from(""),
    );
    
    console.log(vmResult.stdout);
  });
});

Running the test will compile the binary and execute the test file, showing the output of the API formatted in JSON.

$ bun run test

Outputs:

bun test v1.1.26 (0a37423b)

tests/index.test.mts:
{"name":"Tatooine","rotation_period":"23","orbital_period":"304","diameter":"10465","climate":"arid","gravity":"1 standard","terrain":"desert","surface_water":"1","population":"200000","residents":["https://swapi.dev/api/people/1/","https://swapi.dev/api/people/2/","https://swapi.dev/api/people/4/","https://swapi.dev/api/people/6/","https://swapi.dev/api/people/7/","https://swapi.dev/api/people/8/","https://swapi.dev/api/people/9/","https://swapi.dev/api/people/11/","https://swapi.dev/api/people/43/","https://swapi.dev/api/people/62/"],"films":["https://swapi.dev/api/films/1/","https://swapi.dev/api/films/3/","https://swapi.dev/api/films/4/","https://swapi.dev/api/films/5/","https://swapi.dev/api/films/6/"],"created":"2014-12-09T13:50:49.641000Z","edited":"2014-12-20T20:58:18.411000Z","url":"https://swapi.dev/api/planets/1/"}

✓ data request execution > should run MyDataRequest [1854.33ms]

We’ve successfully created an Oracle Program that fetches data. To make the program more useful, we need to parse the JSON response and work with its properties. The SEDA-SDK utilizes the json-as library, allowing you to define a schema and parse a JSON string into that schema. This library is already included in the starter kit.

Let’s modify our example to integrate this library:

import { Console, httpFetch, OracleProgram } from "@seda-protocol/as-sdk/assembly";

// Annotate the class so that json-as can parse this
@json
class Planet {
  // ! required, since json-as sets these properties.
  name!: string;
  terrain!: string;
}

export class MyDataRequest extends OracleProgram {
  execution(): void {
    const response = httpFetch("https://swapi.dev/api/planets/1");
    // Parse the response bytes to our JSON schema
    const planet = response.bytes.toJSON<Planet>();
    
    // Ensure the fetch call has succeeded
    if (!response.ok) {
      Process.error(Bytes.fromUtf8String("Could not fetch API endpoint"));
    }

    Console.log(`We are on planet ${planet.name}, which is a ${planet.terrain} terrain`);
  }
  
  tally(): void {
	  throw new Error("Not implemented");
  }
}

new MyDataRequest().run()

Running the previous test again now outputs:

bun test v1.1.26 (0a37423b)

tests/index.test.mts:
We are on planet Tatooine, which is a desert terrain

✓ data request execution > should run MyDataRequest [174.21ms]

Now you can call any open API you want and process the data as needed.

HTTP Fetch: POST

The SEDA SDK enables you to send POST requests (as well as PATCH, PUT, and others) with an attached body. This allows you to work with various APIs, including GraphQL or uploading files to IPFS.

import { Bytes, Console, httpFetch, OracleProgram, Process } from "@seda-protocol/as-sdk/assembly";

@json
class GraphQLQuery {
  query!: string;
}

export class MyDataRequest extends OracleProgram {
  execution(): void {
    const queryBody = `
      query ExampleQuery {
        company {
          name
        }
      }
    `;
    
    const graphQLQuery = new GraphQLQuery();
    graphQLQuery.query = queryBody;

    // httpFetch allows you to modify the request headers
    const headers = new Map<string, string>();
    headers.set('content-type', 'application/json');

    // Fetches the GraphQL API endpoint
    const response = httpFetch("https://spacex-production.up.railway.app/", {
      headers,
      method: "POST",
      // The body to send along with the request
      body: Bytes.fromJSON(graphQLQuery),
    });

    // Ensure the fetch call has succeeded
    if (!response.ok) {
      Process.error(Bytes.fromUtf8String("Could not fetch API endpoint"));
    }

    Console.log(response.bytes.toUtf8String());
  }

  tally(): void {
    throw new Error("Not implemented");
  }
}

new MyDataRequest().run()

Running the same test will give you the response of the POST request:

bun test v1.1.26 (0a37423b)

tests/index.test.mts:
{
  exitCode: 0,
  stderr: "",
  stdout: "{\"data\":{\"company\":{\"name\":\"SpaceX\"}}}\n\n",
  result: Uint8Array(0) [  ],
  resultAsString: "",
}
✓ data request execution > should run MyDataRequest [883.96ms]