Refresh flow output files from the command line

You can run your flow from the command line to refresh your flow output instead of running the flow from Tableau Prep Builder. You can run one flow at a time using this method. This option is available on both Windows and Mac machines where Tableau Prep Builder is installed.

Note: If you use Single Sign-On authentication, running flows from the command line isn't available. You can run the flows from Tableau Prep Builder instead.

Connector limitations:

  • JDBC or ODBC connectors: Flows that include these connectors can be run from the command line starting in version 2019.2.3.
  • Cloud connectors: Flows that include cloud connectors, such as Google BigQuery, can't be run from the command line. Instead run the flow manually or run the flow on a schedule in Tableau Server or Tableau Online using Tableau Prep Conductor. For more information, see Keep Flow Data Fresh.

For Windows machines, you can also schedule this process using Windows Task Scheduler. For more information, see Task Scheduler(Link opens in a new window) in the Microsoft online help.

When you run flows from the command line, Tableau Prep Builder refreshes all outputs for the flow using the settings for the output steps specified in Tableau Prep Builder. For information about how to specify your output locations, see Create and publish data extracts and data sources(Link opens in a new window). For information about setting your write options (version 2020.2.1 and later), see Configure write options.

Before running the flow

To run the flow from the command line, you'll need administrator privileges on the machine where you are running the flow and you'll need the following information:

  • The path where Tableau Prep Builder is installed.

  • If connecting to databases and publishing output files to a server or a database (version 2020.3.1 and later) - create a credentials .json file that includes all required credentials.

  • The path where the Tableau Flow (.tfl) file is located.

Credentials .json file requirements

Note: Credentials .json files aren't required if the flow connects to and outputs to local files, files stored on a network share or input files that use Windows Authentication (SSPI). For more information about Windows Authentication, see SSPI Model(Link opens in a new window) in the Microsoft online help.

Tableau Prep Builder uses information from the flow file and from the credentials .json file to run the flow when you have remote connections. For example, the database name for your remote connections and the project name for your output files come from the flow, and the server name and the sign in credentials come from the credentials .json file.

  • If you plan to reuse the file, place it in a folder where it won't be overwritten by the Tableau Prep Builder install process.
  • If you are running a flow that connects to database files, published data sources, publishes the output to a server, or publishes the output to a database (version 2020.3.1 and later), the .json file must include the credentials that are required to connect to these locations.
  • The credentials specified in your flow and the credentials included in your .json file must match, otherwise the flow will fail to run.
  • When you run the process, the hostname, port, and username are used to find the matching connection in the Tableau flow file (.tfl) and updated before running the process. Port ID and Site ID are optional if your connections don't require this information.
  • If connecting to a published data source, include hostname, contentUrl, and port (80 for http and 443 for https) in the input connections. The hostname is required to find the matching connection in the Tableau flow file (.tfl), and the contentUrl and port are used to establish the connection to the server.

  • If you connect to Tableau Online, include the port (80 or 443) in the input connections for the pod that you are connecting to. For more information about Tableau Online, see Tableau Bridge connections to Tableau Online(Link opens in a new window) in the Tableau Online help.

Depending on your Tableau Prep Builder version, your credential information may be formatted differently. Click on the tab below to view the credential format for your Tableau Prep Builder version.

Depending on your connections, include your server or database credentials or both. When your flow connects to and outputs to the same server or database, you only need to include a single block in the .json file. If you connect to a server or database that uses different credentials, use a comma delimited array.

Server connections Database connections

Connection block name: "tableauServerConnections"

Include the following data in the array:

  • serverUrl (Server name)
  • contentUrl (Site ID. This appears after /site/ in the URL for Tableau Server or Tableau Online. For example "https://my.server/#/site/mysite" set "contentUrl": "mysite".))
  • port (Port ID)
  • username
  • password

Connection block name: "databaseConnections"

Include the following data in the array:

  • hostname (Server name)
  • port (Port ID)
  • username
  • password

Note: ContentUrl is always required in the .json file for sever connections. If connecting to a default site, for example "https://my.server/#/site/", set ContentUrl to blank. For example "contentUrl": ""

Examples

This section shows different examples of credentials files that you can create using the credentials .json requirements.

Connecting to a server connection

This example shows a .json credentials file that connects to and outputs to a server connection that uses the same credentials:

{
 "tableauServerConnections":[
   {
    "serverUrl":"https://my.server",
    "contentUrl": "mysite",
    "port":443,
    "username": "jsmith",
    "password": "passw0rd$"
   }
 ]
}				

Connecting to a server connection and output to a database connection

This example shows a .json credentials file that connects to a server connection and outputs to a database connection:

{
 "tableauServerConnections":[
   {
    "serverUrl":"https://my.server",
    "contentUrl": "mysite",
    "port":443,
    "username": "jsmith",
    "password": "passw0rd$"
   }
],
 "databaseConnections":[
   {
    "hostname":"example123.redshift.amazonaws.com",
    "port":"5439",
    "username":"jsmith",
    "password":"p@s$w0rd!"
   }
 ]
}

Connecting to and outputting to different database connections

This example shows a .json credentials file that connects to and outputs to different database connections:

{
 "databaseConnections":[
   {
    "hostname":"example123.redshift.amazonaws.com",
    "port": "5439",
    "username": "jsmith",
    "password": "p@s$w0rd!"
   },
   {
    "hostname":"mysql.mydb.tsi.lan",
    "port": "3306",
    "username": "jsmith",
    "password": "mspa$$w0rd"
   }
 ]
}			

Enter an array for your input and output connections.

Note: If using Tableau Prep Builder version 2018.2.2 through 2018.3.1, always include the "inputConnections" and "outputConnections" arrays even if the flow doesn't have remote connections for inputs or outputs. Just leave those arrays blank. If you are using Tableau Prep Builder version 2018.3.2 and later you don't need to include the blank arrays.

Input connections Output connections
  • hostname (Server name)
  • contentUrl (Always required for published data sources. See Output connections for description.)
  • port (Port ID)
  • username
  • password
  • serverUrl
  • contentUrl (Site ID. This appears after /site/ in the URL for Tableau Server or tableau Online. For example "https://my.server/#/site/mysite" set "contentUrl": "mysite".)
  • username
  • password

Examples

This section shows two different examples of credentials files that you can create using the credentials .json requirements.

Connecting to a published data source

This example shows a .json credentials file that connects to a published data source and outputs data to a server that includes a Site ID

Note: If the inputConnection or outputConnection uses the Default site, for example "https://my.server/#/site/", set ContentUrl to blank. For example "contentUrl": ""

{ 
  "inputConnections":[
    {
     "hostname":"https://my.server",
     "contentUrl": "mysite",
     "port":443,								
     "username": "jsmith",
     "password": "passw0rd$"
    }
 ],
  "outputConnections":[
    {
     "serverUrl":"https://my.server",
     "contentUrl":"mysite",
     "username":"jsmith",
     "password":"passw0rd$"
    }
 ]
}

Connecting to two databases

This example shows a .json credentials file that connects to MySQL and Oracle and outputs data to a server that includes a Site ID.

{ 
  "inputConnections":[
    {
     "hostname":"mysql.example.lan",
     "port":1234,
     "username": "jsmith",				
     "password": "passw0rd"
  },
    {
     "hostname":"Oracle.example.lan",
     "port":5678,
     "username": "jsmith",				
     "password": "passw0rd" 
  }
], 
  "outputConnections":[
    {
     "serverUrl":"http://my.server",
     "contentUrl":"mysite",
     "username":"jsmith",
     "password":"passw0rd$"
    } 
  ]
}

Tips for creating your credentials file

To avoid errors when running the flow, make sure your credentials file follows these guidelines:

  • If using Tableau Prep Builder version 2018.2.2 through 2018.3.1, always include the "inputConnections" and "outputConnections" arrays even if the flow doesn't have remote connections for inputs or outputs. Just leave those arrays blank.

    If you are using Tableau Prep Builder version 2018.3.2 and later you don't need to include the blank array.

    • No remote input connection? Include this syntax at the top of the .json file

      {
       "inputConnections":[
        ],
    • No remote output connection? Include this syntax at the bottom of the .json file

    • "outputConnections":[
        ]
      }
  • No port ID for your input connection or the port is specified as part of the server name.

    If there is no port ID for your connection, don't include the "port":xxxx, reference in the .json file, not even "port": "". If the port ID is included in the server name, include the port ID in the host name. For example "hostname": "mssql.example.lan,1234"

  • When referencing the "serverUrl": don't include a "/" at the end of the address. For example, use this "serverUrl": "http://server" not this "serverUrl": "http://server/".

  • If you have multiple input or output connections include the credentials for each one in the file.

  • If connecting to published data sources, make sure to include the hostname and contentUrl in the input connections.

Run the flow

Important: The examples below include the name change for "Tableau Prep" version 2019.1.2 to "Tableau Prep Builder". If you are using an earlier version of the product use "Tableau Prep" instead.

  1. Open the command prompt or terminal command prompt (MacOS) as an Administrator.

  2. Run one of the following commands using the syntax shown below.

    • The flow connects to local files or files stored on a network share and publishes to local files, files stored on a network share or uses Windows authentication:

      Note: If connecting to or outputting to files stored on a network share, use the UNC format for the path: \\server\path\file name. It can't be password protected.

      Windows

                                           "\[Tableau Prep Builder install location]\Tableau Prep Builder <version>\scripts"\tableau-prep-cli.bat -t "path\to\[your flow file name].tfl"
                                      

      Mac

                                          /Applications/Tableau\ Prep\ Builder\ [Tableau Prep Builder version].app/Contents/scripts/./tableau-prep-cli -t path/to/[your flow file name].tfl
                                      
    • The flow connects to databases or publishes to a server:

      Windows

                                          "\[Tableau Prep Builder install location]\Tableau Prep Builder <version>\scripts"\tableau-prep-cli.bat -c "path\to\[your credential file name].json" -t "path\to\[your flow file name].tfl"
                                      

      Mac

                                          /Applications/Tableau\ Prep\ Builder\ [Tableau Prep Builder version].app/Contents/scripts/./tableau-prep-cli -c path/to/[your credential file name].json -t path/to/[your flow file name].tfl
                                      
    • The flow file or credentials file is stored on a network share (use the UNC format for the path: \\server\path\file name):

      Windows

                                          "\[Tableau Prep Builder install location]\Tableau Prep Builder <version>\scripts"\tableau-prep-cli.bat -c "\server\path\[your credential file name].json" -t "\server\path\[your flow file name].tfl"
                                      

      Mac: Map the network share to /Volumes in Finder so that it is persistent, then use /Volumes/.../[your file] to specify the path:

                                          /Applications/Tableau\ Prep\ Builder\ [Tableau Prep Builder version].app/Contents/scripts/./tableau-prep-cli -c /Volumes/.../[your credential file name].json -t path/to/[your flow file name].tfl
                                      

For common errors and resolutions see Common errors when using the command line to run flows.

Run the flow with incremental refresh enabled (version 2020.2.1 and later)

If you don't have Tableau Prep Conductor enabled on your server to schedule your flow runs, you can run your flow using incremental refresh from the command line. Simply include the parameter --incrementalRefresh in your command line as shown in the example below.

Windows

                     "\[Tableau Prep Builder install location]\Tableau Prep Builder <version>\scripts"\tableau-prep-cli.bat --incrementalRefresh -t "path\to\[your flow file name].tfl"
                

Mac

                    /Applications/Tableau\ Prep\ Builder\ [Tableau Prep Builder version].app/Contents/scripts/./tableau-prep-cli --incrementalRefresh -t path/to/[your flow file name].tfl
                

If the input steps in your flow have incremental refresh enabled and the incremental refresh parameters are properly configured, Tableau Prep Builder will do the following:

  • All inputs in the flow that have incremental refresh enabled will run all corresponding outputs using incremental refresh.
  • If no input in the flow has incremental refresh enabled, all outputs will be run using full refresh. A message will show the refresh method details.
  • If some inputs in the flow have incremental refresh enabled, the corresponding outputs will run using incremental refresh. The other outputs will be run using full refresh and a message will show the refresh method details.

For more information about configuring flows to use incremental refresh, see Refresh Flow Data Using Incremental Refresh

Command options

If you want to view the help options, include -h in the command line.

Command options Description Notes
-c , --connections <arg> The connection path to the credentials file. Requires the path to where the credentials file is located.
-d, --debug Debug the flow process. Include this option to view more information to help debug a problem with refreshing the flow. Log files are stored in: My Tableau Prep Builder Repository\Command Line Repository\Logs
-dsv, --disableSslValidation Disable SSL validation (MacOS) When running flows using the command line on the MacOS, a dialog may show asking for the keychain user and password. Starting with Tableau Prep Builder version 2019.3.2, you can pass in this additional parameter to disable this keychain dialog. For example: /Applications/Tableau\ Prep\ Builder\ [Tableau Prep Builder version].app/Contents/scripts/./tableau-prep-cli -dsv -c path/to/[your credential file name].json -t path/to/[your flow file name].tfl
-h, --help View the help for syntax options.

The help option or a syntax error shows the following information:

usage: tableau-prep-cli [-c <arg>] [-d]  [-h] [-t <arg>]
-c,--connections <arg>        Path to a file with all connection information
-d,--debug                    This option is for debugging
-dsv,--disableSslValidation   Disable SSL validation
-h,--help                     Print usage message
-inc,--incrementalRefresh     Run incremental refresh for all outputs that are configured to support it
-t,--tflFile <arg>            The Tableau Prep Builder flow file	
-inc, --incrementalRefresh Run incremental refresh for all outputs that are configured to use it.

Include this option to run incremental refresh for all inputs that are configured to use it. Incremental refresh enables Tableau Prep Builder to retrieve and process only new rows instead of all rows in a flow.

The incremental refresh configuration settings on the input steps determine which flow outputs can be run incrementally. All other outputs will be run using a full refresh and a message will show the refresh method details.

For more information about running flows using incremental refresh, see Refresh Flow Data Using Incremental Refresh .

-t, --tflFile <arg> The .tfl flow file Requires the path to where the .tfl flow file is located.

Syntax examples

The command lines below show four different examples for running a flow using the following criteria:

  • Tableau Prep Builder version: 2020.3.1

    Important: The examples below include the name change for Tableau Prep version 2019.1.2 to Tableau Prep Builder. If you are using an earlier version of the product use "Tableau Prep" instead.

  • Flow name: Flow1.tfl

  • Flow location: C:\Users\jsmith\Documents\My Tableau Prep Builder Repository\Flows

  • Credentials file name: Flow 1.json

  • Credentials file location: C:\Users\jsmith\Desktop\Flow credentials

  • Credentials file location stored on a network share: \tsi.lan\files\Flow credentials

The flow connects to and publishes to local files

Windows

                    "\Program Files\Tableau\Tableau Prep Builder 2020.3.1\scripts"\tableau-prep-cli.bat -t "\C:\Users\jsmith\Documents\My Tableau Prep Builder Repository\Flows\Flow1.tfl"
                

Mac

                    /Applications/Tableau\ Prep\ Builder\ 2020.3.1.app/Contents/scripts/./tableau-prep-cli -t /Users/jsmith/Documents/My\ Tableau\ Prep\ Builder\ Repository/Flows.Flow1.tfl
                

The flow connects to and publishes to local files and uses the short form for incremental refresh

Windows

                    "\Program Files\Tableau\Tableau Prep Builder 2020.3.1\scripts"\tableau-prep-cli.bat -inc -t "\C:\Users\jsmith\Documents\My Tableau Prep Builder Repository\Flows\Flow1.tfl"
                

Mac

                    /Applications/Tableau\ Prep\ Builder\ 2020.3.1.app/Contents/scripts/./tableau-prep-cli -inc -t /Users/jsmith/Documents/My\ Tableau\ Prep\ Builder\ Repository/Flows.Flow1.tfl
                

The flow connects to databases and publishes to a server

Windows

                    "\Program Files\Tableau\Tableau Prep Builder 2020.3.1\scripts"\tableau-prep-cli.bat -c "\C:\Users\jsmith\Desktop\Flow credentials\Flow1.json" -t "\C:\Users\jsmith\Documents\My Tableau Prep Builder Repository\Flows\Flow1.tfl"
                

Mac

                    /Applications/Tableau\ Prep\ Builder\ 2020.3.1.app/Contents/scripts/./tableau-prep-cli -c /Users/jsmith/Desktop/Flow\ credentials/Flow1.json -t /Users/jsmith/Documents/My\ Tableau\ Prep\ Builder\ Repository/Flows.Flow1.tfl
                

The flow publishes to a server and the credentials file is stored on a network share

Windows

                    "\Program Files\Tableau\Tableau Prep Builder 2020.3.1\scripts"\tableau-prep-cli.bat -c "\\tsi.lan\files\Flow credentials\Flow1.json" -t "\C:\Users\jsmith\Documents\My Tableau Prep Builder Repository\Flows\Flow1.tfl"
                

Mac

                    /Applications/Tableau\ Prep\ Builder\ 2020.3.1.app/Contents/scripts/./tableau-prep-cli -c /Volumes/files/Flow\ credentials/Flow1.json -t /Users/jsmith/Documents/My\ Tableau\ Prep\ Builder\ Repository/Flows.Flow1.tfl
                
Thanks for your feedback! There was an error submitting your feedback. Please try again.