Get Initial Ingestion Status

After enabling the Tableau Metadata API using the tsm maintenance metadata-services command, you can use the Backfill API to monitor progress of initial ingestion and get status information for content and lineage backfills.

The steps described below must be performed by a server admin and recommended to be used in conjunction with Step 7: Monitor initial ingestion progress and validate its status in the Enable Tableau Catalogue topic.

Step 1: Authenticate using the REST API

To access the Backfill API, you must first authenticate against Tableau Server and get a token. You can do this using the Tableau REST API. For more information, see Signing In and Signing Out (Authentication)(Link opens in a new window) in the REST API Help.

Alternatively, you can sign in to Tableau Server using your admin credentials.

Step 2: Make a GET request

Make the following GET request or paste the URL into your browser:

http://my_tableau_server/api/metadata/v1/control/backfill/status

The request initially returns information about content backfill. When content backfill is complete, lineage backfill information is returned.

  • For content backfill, the request returns a status summary and additional status information for each content type depending on what content is available on Tableau Server. Indexing for each content type happens concurrently.
  • For lineage backfill, the request returns a status summary.

Status values from the response

The following values are returned by the Backfill API.

  • contentBackfillTotalDurationSeconds and lineageBackfillTotalDurationSeconds is how much time has progressed, in seconds, on the respective backfill type. When backfillComplete is true, contentBackfillTotalDurationSeconds and lineageBackfillTotalDurationSeconds is the total time spent to complete the respective backfill type.

For content backfill: 

  • contentType can show the following content types: PublishedDatasource, Database, DatabaseTable, Metric, Workbook and Flow.
  • contentId is the identifier of the last indexed item.
  • successfullyIngestedCount is the number of items successfully indexed.
  • failedIngestedCount is the number of items that could not be indexed.
  • durationSeconds is the time spent, in seconds, to index items for the content type.
  • totalCount is the total number of items to index.
  • checkpointCreatedTime is the last recorded time, in UTC, an item was indexed. The Backfill API checks for the last indexed item every five minutes.
  • backfillComplete is true when indexing is complete for all items of the content type.

For lineage backfill:

  • totalCount is the total number of lineage relationships to index.
  • processedCount is the number of indexed lineage relationships.
  • lastLineageConnection is the last indexed lineage relationship.
  • backfillComplete is true when indexing is complete for all lineage relationships.

Example response

The request returns JSON text. To view the JSON in a more readable format, you can use a JSON viewer or browser add-on.

{        

"contentBackfillTotalDurationSeconds": 362,

"lineageBackfillTotalDurationSeconds": 14,

"contentBackfillStatuses": [

{

"contentType": "PublishedDatasource",

"contentId": "sites/1/datasources/-631379806-1912815680",

"successfullyIngestedCount": 20,

"failedToIngestCount": 0,

"durationSeconds": 312,

"totalCount": 20,

"checkpointCreatedTime": "2020-07-29T23:50:25.763Z",

"backfillComplete": true

},

{

"contentType": "Database",

"contentId": "sites/1/databases/e1331f9d-4d73-ee04-9edf-96fd1c37cb8e",

"successfullyIngestedCount": 35,

"failedToIngestCount": 0,

"durationSeconds": 26,

"totalCount": 35,

"checkpointCreatedTime": "2020-04-29T23:50:25.769Z",

"backfillComplete": true

},

{

"contentType": "DatabaseTable",

"contentId": "sites/1/tables/d946d084-53a8-09b6-2ad2-93301e6b4b15",

"successfullyIngestedCount": 64,

"failedToIngestCount": 0,

"durationSeconds": 49,

"totalCount": 64,

"checkpointCreatedTime": "2020-04-29T23:50:25.774Z",

"backfillComplete": true

},

{

"contentType": "Metric",

"contentId": "sites/1/metrics/metric1",

"successfullyIngestedCount": 2,

"failedToIngestCount": 0,

"durationSeconds": 254,

"totalCount": 2,

"checkpointCreatedTime": "2020-04-29T23:50:25.779Z",

"backfillComplete": true

},

{

"contentType": "Workbook",

"contentId": "sites/1/workbooks/6749399-1501801290",

"successfullyIngestedCount": 10,

"failedToIngestCount": 0,

"durationSeconds": 267,

"totalCount": 10,

"checkpointCreatedTime": "2020-04-29T23:50:25.784Z",

"backfillComplete": true

},

{

"contentType": "Flow",

"contentId": "sites/1/flows/4",

"successfullyIngestedCount": 4,

"failedToIngestCount": 0,

"durationSeconds": 195,

"totalCount": 4,

"checkpointCreatedTime": "2020-04-29T23:50:25.788Z",

"backfillComplete": true

}

],

"lineageBackfillStatus": {

"totalCount": 45,

"processedCount": 18,

"lastLineageConnection": "CloudFile downstreamWorkbooks Workbook",

"backfillComplete": false

}

}

Thanks for your feedback!