Ansible Playbook Instructions

This document introduce some important variables and ansible host inventory file from the git repository https://github.com/TenDBCluster/TenDBCluster-Ansible.

Host Inventory Instructions

As an example, we define a topology of TenDB Cluster as:

                          +-------------------------+   +-------------------------+
                          |   +-----------------+   |   |   +-----------------+   |
                          |   |   tendb-spt0    | <-|---|-- |   tendb-spt0-1  |   |
                          |   +-----------------+   |   |   +-----------------+   |
                          |                         |   |                         |
+-----------------+       |   +-----------------+   |   |   +-----------------+   |
| tspider-node-01 |       |   |   tendb-spt1    | <-|---|-- |   tendb-spt1-1  |   |
+-----------------+       |   +-----------------+   |   |   +-----------------+   |
                          +-------------------------+   +-------------------------+

                          +-------------------------+   +-------------------------+
+-----------------+       |   +-----------------+   |   |   +-----------------+   |
| tspider-node-02 |       |   |   tendb-spt2    | <-|---|-- |   tendb-spt2-1  |   |
+-----------------+       |   +-----------------+   |   |   +-----------------+   |
                          |                         |   |                         |
                          |   +-----------------+   |   |   +-----------------+   |
                          |   |   tendb-spt3    | <-|---|-- |   tendb-spt3-1  |   |
                          |   +-----------------+   |   |   +-----------------+   |
                          +-------------------------+   +-------------------------+


+----------------++----------------++----------------+                             
| tdbctl-node-01 || tdbctl-node-02 || tdbctl-node-03 |                             
+----------------++----------------++----------------+

In this topology a small grid represents a TenDB/TSpider/Tdbctl instance and a large grid represents a host. From it we can see there are two TenDB instances on one host.

Accordingly, hosts.tendbcluster is defined as:

[all:vars]
ansible_ssh_user=root
ansible_ssh_port=22
ansible_ssh_private_key_file=/root/.ssh/tendbcluster_rsa
# set the ssh public key to remote hosts
ansible_ssh_public_key_file=/root/.ssh/tendbcluster_rsa.pub

# the shard_number must be the number of tendb:children
[tendbcluster:vars]
tendbcluster_name=test
tendbcluster_shard_num=4
node_status=1

mysql_home_dir=/home/mysql/mysqldata
mysql_log_dir=/home/mysql/mysqllog
innodb_buffer_pool_size_mb={{innodb_buffer_pool_size_mb_auto}}
mysql_data_dir="{{mysql_home_dir}}/{{mysql_port}}"

################
[tendbcluster:children]
tendb
tspider
tdbctl

# all this child must be defined later
[tendb:children]
SPT0
SPT1
SPT2
SPT3

################
#### SPIDER ####
# the ansible_host:mysql_port MUST be unique in this inventory
[tspider]
tspider-node-01 ansible_host=192.168.1.106 mysql_port=25000
tspider-node-02 ansible_host=192.168.1.26 mysql_port=25000

################
#### TDBCTL ####
# MGR will be setup when tdbctl node num >=3
# you need to set group_replication_local_address to different port when installing tdbctl on the same machine
[tdbctl]
tdbctl-node-01 ansible_host=192.168.1.26  mysql_port=26000 group_replication_local_address={{ansible_host}}:23306
tdbctl-node-02 ansible_host=192.168.1.2   mysql_port=26000 group_replication_local_address={{ansible_host}}:23306
tdbctl-node-03 ansible_host=192.168.1.106 mysql_port=26000 group_replication_local_address={{ansible_host}}:23306

################
#### SPT0 ####
[SPT0]
tendb-spt0 ansible_host=192.168.1.95 mysql_shard=SPT0 mysql_port=20000 role=master
tendb-spt0-1 ansible_host=192.168.1.147 mysql_shard=SPT0 mysql_port=20000 role=slave master=tendb-spt0

#### SPT1 ####
[SPT1]
tendb-spt1 ansible_host=192.168.1.95 mysql_shard=SPT1 mysql_port=20001 role=master
tendb-spt1-1 ansible_host=192.168.1.147 mysql_shard=SPT1 mysql_port=20001 role=slave master=tendb-spt1

#### SPT2 ####
[SPT2]
tendb-spt2 ansible_host=192.168.1.2 mysql_shard=SPT2 mysql_port=20002 role=master
tendb-spt2-1 ansible_host=192.168.1.130 mysql_shard=SPT2 mysql_port=20002 role=slave master=tendb-spt2

#### SPT3 ####
[SPT3]
tendb-spt3 ansible_host=192.168.1.2 mysql_shard=SPT3 mysql_port=20003 role=master
tendb-spt3-1 ansible_host=192.168.1.130 mysql_shard=SPT3 mysql_port=20003 role=slave master=tendb-spt3

Explanations:

  1. All inventory_hostnames must be distinct.

  2. Hostname-Port (ansible_host : mysql_port) pairs of Tdbctl, TSpider and TenDB nodes must be distinct. The server_id under the same shard of TenDB is auto generated; therefore, in a GTID master-slave environment, server_ids are guaranteed distinct. Please note that if some TenDB nodes are added or deleted, or the order of hosts are changed, the server_id generated based on groups[mysql_shard].index(inventory_hostname) will change accordingly; in this case, if TenDB nodes are not restarted, it may lead to duplicate server_ids. The safest way to approach this is to set the server_id variable for every host.

  3. The mysql_shard variable of hosts under the same shard of TenDB must be the same, and has to follow a format of SPT+sequence_number.

  4. Each host of TenDB must have a role configured to either a master or slave. If a master/slave switch takes place later, the configured roles have to be updated in time, otherwise it may cause problems for the entire cluster.

  5. For TenDB, we suggest a slave use the same port as its master's. A master and a slave should be deployed on different hosts.

  6. In TSpider, the autoinc_value variable auto increases every time a new node joins. If not in TSpider, it is maintained based on indexes. Therefore, it is not recommended to change the indexes. Similarly, the server_id in TSpider is auto generated and auto increases as such.

  7. If there are at least 3 Tdbctl nodes, Tdbctl will be automatically configured to MGR mode during installation. In production, we suggest Tdbctl be in MGR mode, otherwise DDL operations will fail if Tdbctl ever undergoes a breakdown.

  8. Global variables:

    • tendbcluster_shard_num Number of shards, i.e., number of TenDB instances. Each shard must be given a precise definition (eg. SPT0, SPT1, SPT2, SPT3 as defined above).

    • tendbcluster_name Name of the current TenDBCluster, used to identify a cluster during monitoring.

    • node_status (to be implemented) Status of a node (set to 1 by default).

      • 1: Working normally or expected to work normally.
      • 2: Expected to work normally but confirmed to have broken down.
      • 3: Waiting for retiring and switching offline.
      • 4: Currently offline, can be safely cleared out of the inventory.
    • tendbcluster:vars Key variables that have great influence in cluster performance, such as mysql_home_dir (directory path for MySQL data storage) and innodb_buffer_pool_size_mb (size of InnoDB buffer pool).

      • mysql_home_dir, mysql_data_dir When multiple instances are deployed on a same host, mysql_port is used to separate data storage directories. If your hosts have multiple high-performance disks, you can directly set the mysql_data_dir variable for each host.
      • innodb_buffer_pool_size_mb By default, the size of buffer pool of each instance is calculated based on the number of instances on a host inferred from the inventory. The innodb_buffer_pool_size_pct_total variable in group_vars is set at 0.8 by default, making 80% of total memory being used. If you want to set a different buffer pool for a specific instance, you can overwrite the global configuration by setting the value of this variable for your host (unit: MB).

group_vars Variable Instructions

Ansible will render the variables defined from group_vars/ . You may read ansible ariable precedence to find out the variables' priority.

group_vars/all:

fileserver: http://tendbcluster.com/downloads
ansible_python_interpreter: /usr/bin/python

# iface: eth1
mysql_version: "tendb-3.3"
mysql_pkg: "mysql-5.7.20-linux-x86_64-{{mysql_version}}-gcs"
mysql_pkg_md5: 01b0f74ff7a1d6a9b6fa31190adbac0a

tspider_version: tspider-3.6.1
tspider_pkg: "mariadb-10.3.7-linux-x86_64-{{tspider_version}}-gcs"
tspider_pkg_md5: 326c4874e56be4d6d84e6f60e08af050

tdbctl_version: tdbctl-2.1
tdbctl_pkg: "mysql-5.7.20-linux-x86_64-{{tdbctl_version}}"
tdbctl_pkg_md5: 3b127981aa5835e914f71d6777982646

checksum_enabled: false

innodb_buffer_pool_size_pct_total: 0.8

# my.cnf [mysqld]
mycnf_mysqld:
  innodb_flush_log_at_trx_commit: 0
  character-set-server: utf8mb4

# my.cnf [mysql]
mycnf_mysql:
  default-character-set: utf8mb4

backup_dir: /data/dbbak
mysql_charset: utf8mb4
mysql_conf_dir: /home/mysql/etc

mysql_socket: "{{mysql_data_dir}}/mysql.sock"
mysql_pid_file: "{{mysql_data_dir}}/mysqld.pid"
mysql_conf_file: "{{mysql_conf_dir}}/my.cnf.{{mysql_port}}"

# user admin
tendbcluster_user_admin: tendbcluster
tendbcluster_user_admin_pass: xxxxxx
# user repl
tendbcluster_user_repl: repl_user
tendbcluster_user_repl_pass: xxxxxx

Note:

  • fileserver The http url for downloading packages for installation.

  • mysql_version, mysql_pkg, mysql_pkg_md5 TenDB package's version, name and MD5 checksum. Similarly, TSpider and Tdbctl packages are specified as such.

  • innodb_buffer_pool_size_pct_total Percentage of TenDB's memory usage to total memory on a host. For example, say a host has a total memory of 16384 MB, the size of buffer pool is then set as int(16384 * 0.8) = 13107 MB. If 4 shards are deployed on this host, then each instance's innodb_buffer_pool_size is int(13107 / 4) = 3276 MB. You can also manually set a specific size in the host inventory; in this case, the result of auto calculation will be overwritten.

  • mycnf_mysqld Use it to customize the mysqld section in the my.cnf file. If you want to have different configurations for TenDB and TSpider, modify group_vars/tendb and group_vars/tspider respectively. Note that the generation of my.cnf file does not completely depend on mycnf_mysqld; there are some built-in configurations in the roles/tendb/templates/my.cnf.tendb.j2 file.

  • backup_dir Directory for MySQL backups. It is mainly used to store backups from mysqldump during "build slave".

  • tendbcluster_user_admin, tendbcluster_user_repl User accounts that have to be created for the cluster to run. They are automatically created during the deployment of cluster and are used for maintaining the cluster.

    • tendbcluster_user_admin Name of the admin user. The access IP is restricted during the creation of the admin user. Tdbctl will use this user account to create tables in each node.
    • tendbcluster_user_admin_pass Password of the admin user. We suggest using ansible-vault for password encryption.
    • tendbcluster_user_repl Name of the MySQL replication user. Replication privileges are granted to this user from any access IP (using %).
    • tendbcluster_user_repl_pass Password of the MySQL replication user.
    • tendbcluster_user_root_pass Password of the MySQL root account. Its value is the same as tendbcluster_user_admin_pass's by default. Root account is not needed after deployment.

results matching ""

    No results matching ""