Deploy a Coordination Service Ensemble
The Coordination Service is built on Apache ZooKeeper, an open-source project, and coordinates activities on the server, guaranteeing a quorum in the event of a failure, and serving as the source of "truth" regarding the server topology, configuration, and state. The service is installed automatically on the initial Tableau Server node, but no additional instances are installed as you add additional nodes. Because the successful functioning of Tableau Server depends on a properly functioning Coordination Service, we recommend that for server installations of three or more nodes, you add additional instances of the Coordination Service by deploying a new Coordination Service ensemble. This provides redundancy and improved availability in the event that one instance of the Coordination Service has problems.
The hardware you use for Tableau Server can have an effect on how well the Coordination Service runs. In particular:
Memory. The Coordination Service maintains state information in memory. By design, the memory footprint is small, and is typically not a factor in overall server performance.
Disk speed. Because the service stores state information on disk, it benefits from fast disk speed on the individual node computers.
Connection speed between nodes. The service communicates continuously between cluster nodes; a fast connection speeds between nodes helps with efficient synchronization.
Because the Coordination Service is I/O intensive, if you are running Tableau Server on computers that meet or just exceed the minimum hardware requirements, you may want to configure a Coordination Service ensemble that puts the service on nodes that are not being used for other server processes. This reduces the chance of delays due to I/O contention between server processes. For information on how to deploy an ensemble on dedicated Coordination Service-only nodes, see Configure Tableau Server for High Availability with Coordination Service-Only Nodes.
To ensure that the Coordination Service can work properly, the service requires a quorum—a minimum number of instances of the service. This means that the number of nodes in your installation impacts how many instances of the Coordination Service you want to configure in your ensemble.
Number of Coordination Service instances to use
The maximum number of Coordination Service instances you can have in an ensemble on Tableau Server depends on how many Tableau Server nodes you have in your deployment. Configure a Coordination Service ensemble based on these guidelines:
|Total number of server nodes||Recommended number of Coordination Service nodes in ensemble (must be 1, 3, or 5)||Notes|
|1-2 nodes||1 node||This is the default and requires no changes unless you want to move the Coordination Service off your initial node and onto your additional node.|
|3-4 nodes||3 nodes|
|5 or more nodes||5 nodes||Five is the maximum number of Coordination Service instances you can install.|
If you install a total of three or more nodes, you should also deploy a Coordination Service ensemble. If you do not, you will get a warning message every time you make changes to the server configuration or topology. You can ignore this message, but as a best practice you should deploy a multi-node Coordination Service ensemble.
When you install Tableau Server, a single instance of the Coordination Service is installed on the initial node. TSM and Tableau Server depend on the Coordination Service to function properly, so to provide redundancy and ensure availability on multi-node installations, configure additional instances of the Coordination Service by deploying a Coordination Service ensemble. Coordination Service ensembles are installed with one, three, or five instances of the Coordination Service. In a three-node installation of Tableau Server, the recommended number of Coordination Service instances is three, one on each node.
Do not attempt to deploy a Coordination Service ensemble if there are other changes pending. Discard or apply any pending changes before deploying a new Coordination Service ensemble.
Note: This operation includes steps that you may need to perform using the TSM command line.
The following steps illustrate how to deploy a new Coordination Service ensemble on an existing three-node Tableau Server cluster and clean up the old ensemble.
On the initial node, open a command prompt as administrator.
Type this command to sign in to Tableau Server as a TSM administrator:
tsm login -u <username>
You will be prompted for your password.
Type this command to stop Tableau Server:
Some TSM processes will continue to run, including the Administration Controller and Administration Agent.
Confirm there are no pending changes:
tsm pending-changes list
If there are pending changes, you need to either discard the changes or apply them. Applying pending changes will take some time:
Discard the changes
tsm pending-changes discard
Apply the changes:
tsm pending-changes apply
Wait until the command completes and you are returned to the system prompt.
Get the node IDs for each node in the cluster:
tsm topology list-nodes -v
tsm topology deploy-coordination-servicecommand to add a new Coordination Service ensemble by adding the Coordination Service to specified nodes. You must specify the node(s) that the Coordination Service should be added to. The command also makes the new ensemble the "production" ensemble (the ensemble in use).
For example, deploy the Coordination Service to all three nodes of a three-node cluster, where the nodes are node1, node2, and node3:
tsm topology deploy-coordination-service -n node1,node2,node3
Wait until the new Coordination Service ensemble is running and the server is ready for the next step.
Important. If you attempt to clean up the old Coordination Service ensemble before the server is in the proper state, you can put the server into an unrecoverable state and may need to completely reinstall Tableau.
Check the status of the server:
tsm status -v
If the deployment is not complete, you may see processes showing as running when they are not, and the Coordination Service showing a status of "unavailable" while the service is synchronizing between nodes on the cluster. Tableau Server may show as being in an error state while this is happening. You may also get an error message: "Could not connect to TSM Controller at '<host>:8850'." This is normal when the server is returning to a valid state.
Check the status of the server periodically until you are prompted to sign in again.
When you are prompted, sign in to TSM and continue to check the server status until you see a status of "STOPPED" for each node. If the status of a node shows as "ERROR" you need to wait. When each node status is "STOPPED" you should also see the following services running:
On the initial node:
Two instances of the Coordination Service on the initial node, both with a status of "running".
The Administration Controller with a status of "running". (The Administration Controller is only installed on the initial node.)
The Administration Agent with a status of "running".
Additional services on the initial node, all with a status of "running": Service Manager, License Manager, Client File Service.
On the additional nodes:
One or more instance of the Coordination Service on each additional node you specified when you deployed the new ensemble, all with a status of "running". If you are deploying a new ensemble to nodes that already had Coordination Service running, you will see two instances of the service.
The Administration Agent on every node, with a status of "running".
If you do not see a status of "running" for all of the above, wait a few minutes and run the status command again.
Note: If there is a problem with an instance of the Coordination Service (if it shows as stopped for example), you can toggle back to your previous Coordination Service ensemble using the
tsm topology toggle-coordination-servicecommand. To do this, the rest of the services should be in the state described above, including Administration Controller and Agent. You can toggle back to the previous ensemble only if you have not run the
cleanup-coordination-servicecommand. Tableau Server cannot be running when you use this command.
Remove the old ensemble. This step is required. You cannot run Tableau Server with multiple Coordination Service ensembles configured.
tsm topology cleanup-coordination-service
Tableau Server must be stopped when you use this command, but some TSM services will be running (see above).
Start Tableau Server: