Class FetchUtils

Fetch binary data from an API endpoint.

The method returns a promise that resolves with an Uint8Array object.

Example

import { FetchUtils } from '@tableau/taco-toolkit/handlers'

const result = await FetchUtils.fetchArrayBuffer('example.com/api/file')

// Result: a Uint8Array object which contains the file content
Copy

Fetch data from an API endpoint.

The method parses the response body as a JSON string, and returns a promise which resolves with the parsing result.

Example

import { FetchUtils } from '@tableau/taco-toolkit/handlers'

const result = await FetchUtils.fetchJson('example.com/api/user', {
   method: 'POST',
   body: {
     id: 'foo',
   }
})

// Result: { id: 'foo', name: 'bar' }
Copy

Ingest a list of data with DataRow type into EPS data storage. The method returns a promise that resolves when data ingestion succeeds.

The ingested data will be associated with the handlerInput that triggers the Fetcher, and uses the name property as the table name for the parsed data.

The API must be utilized with built-in file-based parser taco:data-file-parser that parses the ingested data from the EPS data storage.

Example

// handlerInput
{
   fetcher: 'MyFetcher', // fetcher file name
   parser: 'taco:data-file-parser', // built-in data parser name
   name: 'user-table' // identifier of source object, it will be used as table name
}
Copy

// MyFetcher.ts
import { FetchUtils, DataRow } from '@tableau/taco-toolkit/handlers'

export default class MyFetcher extends Fetcher {
  async *fetch(options: FetchOptions) {
    const users = await FetchUtils.fetchJson('example.com/api/user')

    const rows: DataRow[] = users.map((user) => {
      const { id, name, address: { street, state, country } } = user
      return { id, name, street, state, country }
    })

    await FetchUtils.ingestDataRows(rows)
  }
}
Copy

Fetch CSV data from an API endpoint and ingest it into EPS data storage. The function returns a promise that resolves when data ingestion succeeds.

The function is designated for CSV data only. For a data source that provides CSV data in multiple files (with same schema), the function may ingest the files into same table. If the multiple files all contain header line, setting the option trimColumnHeader to true will let the function skip the header line when ingesting the subsequent files. trimColumnHeader has no effect when loading the first file for a table.

Note that loading multiple files into the same table must be performed sequentially, meaning loadCsvData must be called after the promise returned from the previous call is resolved.

The API must be utilized with built-in file parser taco:csv-file-parser that parses the ingested data from the EPS data storage.

Example

// handlerInput
{
   fetcher: 'MyFetcher', // fetcher file name
   parser: 'taco:csv-file-parser', // built-in csv parser name
   name: 'user-table' // identifier of source object, it will be used as table name
}
Copy

// MyFetcher.ts
import { FetchUtils } from '@tableau/taco-toolkit/handlers'

export default class MyFetcher extends Fetcher {
   async *fetch(options: FetchOptions) {
     yield await FetchUtils.loadCsvData('example.com/api/user.csv', {
       trimColumnHeader: true
     })
   }
}
Copy

Fetch excel data from an API endpoint and ingest it into EPS data storage. The function returns a promise that resolves when data ingestion succeeds.

The function fetches the data in a stream fashion.

The loaded data will be associated with the handlerInput that triggers the Fetcher, and the name property is used to identify the workbook, while the sheet names will be used as the table names.

The API must be utilized with built-in excel file parser taco:excel-file-parser that parses the ingested data from the EPS data storage.

Example

// handlerInput
{
   fetcher: 'MyFetcher', // fetcher file name
   parser: 'taco:excel-file-parser', // built-in parser name
   name: 'user-table' // identifier of source object, which will be used to identify the workbook
}
Copy

// MyFetcher.ts
import { FetchUtils } from '@tableau/taco-toolkit/handlers'

export default class MyFetcher extends Fetcher {
   async *fetch(options: FetchOptions) {
     yield await FetchUtils.loadExcelData('example.com/api/user.xls')
   }
}
Copy

Fetch parquet data from an API endpoint and ingest it into EPS data storage. The function returns a promise that resolves when data ingestion succeeds.

The function fetches the data in a stream fashion.

The loaded data will be associated with the handlerInput that triggers the Fetcher, and uses the name property as the table name for the parsed data fetched from API endpoint/s.

The API must be utilized with built-in parquet file parser taco:parquet-file-parser that parses the ingested data from the EPS data storage.

Example

// handlerInput
{
   fetcher: 'MyFetcher', // fetcher file name
   parser: 'taco:parquet-file-parser', // built-in parser name
   name: 'user-table' // identifier of source object, which will be used as table name
}
Copy

// MyFetcher.ts for single API endpoint
import { FetchUtils } from '@tableau/taco-toolkit/handlers'

export default class MyFetcher extends Fetcher {
   async *fetch(options: FetchOptions) {
     yield await FetchUtils.loadParquetData('example.com/api/user.parquet')
   }
}
Copy

Note: The API can also be used to fetch parquet data from multiple API endpoints and append the data together into a single table. The name property from the handlerInput will be used as the table name for the parsed data fetched from multiple API endpoint.

Example

// MyFetcher.ts for multiple API endpoints
import { FetchUtils } from '@tableau/taco-toolkit/handlers'

export default class MyFetcher extends Fetcher {
   async *fetch(options: FetchOptions) {
     const urls = [
       'example.com/api/user1.parquet',
       'example.com/api/user2.parquet',
       'example.com/api/user3.parquet'
     ]
     const promises = urls.map((url) => FetchUtils.loadParquetData(url))
     await Promise.all(promises)
     yield
   }
}
Copy