External Message Queue Service (RabbitMQ) for Tableau Resource Monitoring Tool
The Tableau Resource Monitoring Tool uses RabbitMQ as its message queue service to collect data from Agents and bring them to the RMT Server. This information in the queue is processed and eventually stored in the RMT repository (PostgreSQL database).
Before the 2022.3 release, the RabbitMQ message queue service was automatically installed with the RMT Server and this was the only configuration available. Starting in RMT version 2022.3, you can reconfigure RMT Server to use an externally hosted RabbitMQ service. When RMT Server is configured to use an externally hosted message queue service, we refer to it as the external message queue service.
With this new option added in version 2022.3, RMT Server can be configured in the following ways:
- Continue to use the locally installed message queue service: This means that RabbitMQ that is automatically installed with RMT Server on the same machine will be used as the message service for RMT Server, and no further changes are made to this configuration.
- Configure RMT Server to use an external message queue service: This means that RAbbitMQ that RMT uses is hosted external to RMT Server. Currently only AWS AMQ is supported as the hosting platform for RabbitMQ. Since all messages from the Agents go through RabbitMQ, by hosting this externally frees up resources from the machine where RMT Server is installed.
To setup and manage the external message queue service, you should have a good understanding of the AWS AMQ platform. We recommend reviewing the documentation on AWS site. You will also need to know how to configure and manage the external message queue service for RMT. This topic will provide you with that information in the sections below.
New installation of Resource Monitoring Tool
The instructions provided in this section apply to a new installation of Resource Monitoring Tool. If you have an existing installation, and want to move your local RabbitMQ to an externally hosted configuration, see the Existing installations of Tableau Resource Monitoring Tool section of this topic.
Use the following steps to install Tableau Resource monitoring tool and reconfigure RMT Server to use an external message queue service:
-
Create Amazon AMQ for Rabbit MQ to host the external message queue service with the following recommendations:
-
For engine type, use RabbitMQ engine. For the version of Rabbit MQ, see the Product compatibility section.
-
Use the same default version as the one used when RabbitMQ is installed locally. For more information, see the product compatibility table.
-
Use a single-Instance broker.
-
Instance specification: mq.m5.large, 2 vCPU/8 GiB RAM.
-
Create a RabbitMQ username/password.
To learn more, see Working with Amazon MQ for Rabbit MQ on AWS documentation site.
-
-
Test the connection from RMT Server to the message broker by copying the RabbitMQ web console URL from the AWS MQ page, and paste it into a web browser on RMT Server. Log in via the username and password you created when you set up the broker.
-
Follow the instructions in this topic to install RMT Server, but skip the steps for creating an environment. You will do that later after configuring RMT Server to use the external repository.
-
Run rmtadmin setup as follows to configure the external Rabbit MQ message queue service:
rmtadmin master-setup --mq-config=external --mq-server=aws_amq_servername --mq-vhost='/' --mq-port=5671 --mq-username=aws_amq_username --mq-password='aws_amq_password' --mq-tls-certificate-host=aws_amq_servername
-
Now create an environment and download the bootstrap file.
-
Run the following command to create and environment:
rmtadmin create-env --name=<myenvironment> --api-username=<TableauServer API user name> --api-password=<password for the Tableau Server API user account>
-
Download the bootstrap file to register Agents:
rmtadmin bootstrap-file --env=<myenvironment> --filename<The absolute or relative path including the file name>
The steps are described in full detail in the Install the RMT Server Using Command Line topic.
-
-
Re-register Agents on Tableau Server Nodes using the instructions in Install the Agent Using Command Line topic.
Existing installations of Tableau Resource Monitoring Tool
There are two paths to migrating an existing RMT installations that are currently using a local message queue service to an external message queue service:
-
Migration with environment recreation: This involves creating the external Rabbit MQ service, configuring RMT Server to use the external message queue service, recreating the environments, and redoing all the custom configurations like incident thresholds, and re-registering the Agents.
When using this method, anything coming from Agent, hardware metrics, TS logs, viz loads, hyper queries in the queue but not yet processed will be lost during migration. This is very similar to doing a new install of RMT described in the section above except that you will not be installing RMT in this case..
-
Manual migration with minimal data loss: This is specifically for customers who are able to independently able to implement the instructions and do not want to deal with recreating the environments, and recreating incident thresholds. This method will preserve event data from being lost during the transition, but process counter information captured from Tableau Server node hardware during the transition will not be preserved. This migration is manual and not a built-in migration feature for Resource Monitoring Tool.
The instructions to do this is described below:
Migrating to external AWS AMQ
Step1: Create Amazon AMQ for Rabbit MQ to host the external message queue service with the following recommendations.
- Use RabbitMQ engine type.
- Default Version (3.9.13) is equal to RMT 22.2 bundled version.
- Use Single-Instance broker.
- Use default instance size (mq.m5.large, 2 vCPU/8 GiB RAM).
- Set RabbitMQ username/password.
To learn more, see Working with Amazon MQ for Rabbit MQ on AWS documentation site.
Step 2: Test the connection from RMT Server
Copy the RabbitMQ web console URL from the AWS MQ page, and paste it into a web browser on RMT Server. Log in via the username and password you created when you set up the broker.
Step 3: Stop RMT Agent services and de-register Agents
Run the following commands on each node where Agent is running. Don't stop RMT Server, it should be kept running.:
rmtadmin stop
rmtadmin deregister
Step 4: Verify the queue.
On RMT Server, run the rmtadmin status
, and review the output. When all of the values in the Total column for each of the queues reaches 0 (or nearly 0) and remains at that value, proceed to the next step.
Step 5 : Configure RMT Server to use the new Amazon MQ service
The aws_amq_servername value will be the same as the endpoint listed in Amazon MQ’s page, except that the protocol prefix and port at the end should be removed.. See the following example:
Replace: “amqps://b-9512e888-a4a3-4b79-a9c0-07418c101941.mq.us-west-2.amazonaws.com:5671
” with: “B-9512e888-a4a3-4b79-a9c0-07418c101941.mq.us-west-2.amazonaws.com
”
Now, run rmtadmin master-setup
, substituting the user name and password you created for Amazon MQ.
rmtadmin master-setup --mq-config=external --mq-server=aws_amq_servername --mq-vhost='/' --mq-port=5671 --mq-username=aws_amq_username --mq-password=<aws_amq_password> --mq-tls-certificate-host=<aws_amq_servername>
Note the following:
- Replace the name in the --mq-vhost argument below only if you created your own virtual host (doing so is optional), .
- Similarly change the port if the service is set up to run on a non-default port.
- The value for --mq-tls-certificate-host value must be the server name for your AWS MQ instance, as used earlier in the --mq-server argument, though it can be a wildcard for additional flexibility.
Step 6: Start the RMT Server
Navigate to the RMT Server installation folder: cd 'c:\Program Files\Tableau\Tableau Resource Monitoring Tool\master\'
Navigate to the RMT Server installation folder: sudo /var/opt/tableau/tabrmt/master
Start RMT Server: rmtadmin start
Step 7: Re-register Agents
- Download a new bootstrap file from RMT Server.
- Accessing the Agent web interface by navigating to http://localhost:9002/setup/register, import the bootstrap file.
- Click Test Message Queue. Review the previous inputs if you receive an error.
- Click Connect to Message Queue
- Under the Server section, click the dropdown with “New Environment Server” pre-selected, and rather than accepting the default, pick the server you are trying to register from the list. Scroll to the bottom and click “Register Agent” button
Step 8: Verify that RMT Server is running
Log into RMT Server web interface tool and verify that new data is being processed.
Upgrade best practices
Here are the general steps you need to follow if you want to upgrade to a version 2022.3 or later and migrate to using an external message queue service at the same time.
Migration with environment recreation:
- Upgrade RMT Server and all Agents to 2022.3 or later
- Create an Amazon AMQ broker service.
- Configure RMT Server to use the external message queue service
- Recreate environments and re-register all the Agents
- Recreate any custom configurations.
Note: You will lose some event and hardware processing data and you will also need to reconfigure the incident thresholds
Manual migration with minimal data loss:
- Upgrade RMT Server and all Agents to version 2022.3 or later
- Follow the steps detailed in the migration section of this topic. This mainly involves creating a Amazon AMQ broker service an pointing RMT Server to the external message queue service.
- Re-register the agents.
Upgrade steps with enabling TLS for RabbitMQ
Since the agents in versions earlier than 2022.3 have been communicating using unencrypted connection to the RabbitMQ message queue service, when upgrading to version 2022.3 or later, those agents need to be updated to use the new secure connection details. The steps to do this are as follows:
-
After completing the upgrade steps described in the above section, stop all agents by running the following command:
rmtadmin stop --agent
-
Download the bootstrap file for the environment by running the following command:
rmtadmin bootstrap-file --env<myenvironment> --filename <The absolute or relative path including the file name>
-
Run the following command on each of the machines where the Agent is installed:
rmtadmin rotate-mq-certificate <BOOTSTRAP_FILE> --username=<RMT Server Username> --password-file=<RMT Server Password file name>
- Restart each Agent machine after successfully running the
rmtadmin rotate-mq-certificate
command.
Product Compatibility
This table lists RMT version 2022.3 and later only, since the external message queue is only available from version 2022.3 and later.
RMT Version | RabbitMQ version shipped with RMT | Supoort RabbitMQ version for external message queue service |
22.3 | 3.10.5 | 3.10.5 |
Who can do this
To install Resource Monitoring Tool, you must have all the following:
- Administrator permissions on the machine you are installing Resource Monitoring Tool.
- Tableau Server Administrator site role.
- Resource Monitoring Tool Administrator account.