You are here

News

How to install multiple MySQL instances on a single host using MyEnv?

Abdel-Mawla Gharieb - Thu, 2014-10-16 16:36

We have been asked several times by MySQL users about how to install multiple MySQL instances on a single host.
Typically, this is required when testing different MySQL versions or MySQL servers (MySQL server, Percona server and MariaDB server) while no available resources are available.
Sometimes, it is even required to install multiple MySQL instances on a single production server.

In this article, I'll go through the steps needed to install multiple MySQL instances on a single host (using the tar balls binaries) and how our popular tool MyEnv can make such process so easy.

Prepare MySQL environment [root@centos-temp ~]# groupadd mysql [root@centos-temp ~]# useradd -g mysql mysql [root@centos-temp ~]# su - mysql [mysql@centos-temp:~]$ mkdir ~/product [mysql@centos-temp:~]$ mkdir ~/data [mysql@centos-temp ~]$ cd /downloads [mysql@centos-temp ~]$ wget http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.21-linux-glibc2.5-x86_64.tar.gz [mysql@centos-temp ~]$ cd ~/product [mysql@centos-temp ~]$ tar xf /downloads/mysql-5.6.21-linux-glibc2.5-x86_64.tar.gz [mysql@centos-temp ~]$ ln -s mysql-5.6.21-linux-glibc2.5-x86_64 mysql-5.6.21 Install MyEnv

MyEnv can be downloaded from here and the installation steps are listed here.

Install the first instance (named master)
  • Prepare the first instance: [mysql@centos-temp ~]$ mkdir ~/data/master [mysql@centos-temp ~]$ vi ~/data/master/my.cnf [mysqld] port = 3306 basedir = /home/mysql/product/mysql-5.6.21 datadir = /home/mysql/data/master socket = /home/mysql/data/master/master.sock user = mysql
  • Launch the myenv installer: [mysql@centos-temp ~]$ ~/product/myenv/bin/installMyEnv.sh PHP is installed on /usr/bin/php Starting MyEnv installer: /home/mysql/product/myenv-1.1.2/bin/installMyEnv.php Configuration file /etc/myenv/myenv.conf does NOT exist. Copy from template or abort (T, a):
  • Since this is the first instance, the myenv config file does not exist yet, we take the template (t): Copy from template or abort (T, a): t Copy /home/mysql/product/myenv-1.1.2/etc/myenv.conf.template to /etc/myenv/myenv.conf
  • Then MyEnv will detect that no instances are there, we choose the first option (a) to add a new instance: No instance exists yet. An instance is the same as a mysqld process. What do you want to do next? o Add a new instance, o change an existing instance, o delete an existing instance, o save configuration and exit or o quit without saving (A/c/d/s/q)? a
  • MyEnv will ask for the new instance name (master): You have to enter a new instance name. An instance name can consist of 0-9, a-z, A-Z, - and _. Examples: test, prod, dev, [mysqld1] mysqld-3306 Enter a new instance name: master
  • MyEnv will ask for the MySQL basedir and datadir: Changing instance master: Set basedir. The basedir is the directory where your MySQL binaries are located. Example: /home/mysql/product/myenv /home/mysql/product/myenv-1.1.2 /home/mysql/product/mysql-5.6.21 /home/mysql/product/mysql-5.6.21-linux-glibc2.5-x86_64 /usr/local/mysql /opt/mysql [/usr] basedir = /home/mysql/product/mysql-5.6.21 /home/mysql/product/mysql-5.6.21 seems to exist and has already some files in it . Omitting installation of binary tar ball. Set datadir. The datadir is the directory where your MySQL data are located. Example: /usr/local/mysql/data, /opt/mysql/data or /var/lib/mysql or /home/mysql/product/mysql-5.6.21/data or [/home/mysql/data/master] or /home/mysql/data/master datadir = /home/mysql/data/master WARNING: directory /home/mysql/data/master already exist.
  • MyEnv will detect that no mysql database is created yet in the datadir and ask if it should install it for us or not (using the normal mysql_install_db script) Shall I install a mysql database under /home/mysql/data/master (Y/n)? y Installing MySQL instance. Please be patient, this can take a while... /home/mysql/product/mysql-5.6.21/scripts/mysql_install_db --datadir=/home/mysql/data/master --basedir=/home/mysql/product/mysql-5.6.21
  • For the new instance, we should specify the port, the socket, and the location of my.cnf (all these must be specific per instance): Set port. Example: 3307 or [3306] port = 3306 Set socket. Example: /tmp/mysql-3306.sock /tmp/mysql-master.sock [/tmp/mysql.sock] /var/run/mysqld/mysql.sock socket = /home/mysql/data/master/master.sock Choose location of my.cnf: Example: /etc/my.cnf /etc/mysql/my.cnf /home/mysql/data/master/my-3306.cnf [/home/mysql/data/master/my.cnf] my.cnf = /home/mysql/data/master/my.cnf
  • MyEnv will list the just added instance and ask what should be the next step, we should save and exit (s): The following instances are available: master An instance is the same as a mysqld process. What do you want to do next? o Add a new instance, o change an existing instance, o delete an existing instance, o save configuration and exit or o quit without saving (a/c/d/S/q)? s Writing the configuration file... Rename /etc/myenv/myenv.conf to /etc/myenv/myenv.conf.2014-10-15-05-50-12 Write /etc/myenv/myenv.conf
  • MyEnv will propose to add the following lines to the .bash_profile file, we should confirm (y): Do you want to add the following code to your ~/.bash_profile file? . /etc/myenv/MYENV_BASE MYENV_PWD=`pwd` cd $MYENV_BASE/bin . myenv.profile cd $MYENV_BASE; $MYENV_BASE/bin/showMyEnvStatus.php; cd - > /dev/null cd $MYENV_PWD (Y/n)? y Writing /etc/myenv/MYENV_BASE
  • We should add the myenv startup script as shown here into the appropriate path (the provided commands should be executed manually from the root/privileged-user after that): Please copy yourself manually the MyEnv start/stop script to the following location: shell> sudo cp /home/mysql/product/myenv-1.1.2/bin/myenv.server /etc/init.d/myenv done (Y)? y and link it to your O/S runlevels: RedHat/CentOS: shell> sudo chkconfig --add myenv; chkconfig myenv on Ubuntu/Debian: shell> sudo update-rc.d myenv defaults SLES/OpenSuSE: shell> sudo chkconfig --add myenv done (Y)? y Now source your profile as follows: . ~/.bash_profile
  • Now, we should logout and login so that the .bash_profile code will be executed (we can copy the startup script in the meantime): [mysql@centos-temp ~]$> exit [root@centos-temp ~]# cp /home/mysql/product/myenv-1.1.2/bin/myenv.server /etc/init.d/myenv [root@centos-temp ~]# chkconfig --add myenv [root@centos-temp ~]# su - mysql Up : Down : master (5.6.21) master ( 3306) : test mysql@centos-temp:~ [master, 3306]>
  • Now, we can start the just installed instance and check if it is started correctly or not: mysql@centos-temp:~ [master, 3306]> start . SUCCESS! mysql@centos-temp:~ [master, 3306]> up Up : master (5.6.21) Down : master ( 3306) : test mysql@centos-temp:~/data/master [master, 3306]>

Cool! We have got the first instance installed. let's install the second one ...

Install the second instance (named slave1)
  • Prepare the second instance: mysql@centos-temp:~ [master, 3306]> mkdir ~/data/slave1 mysql@centos-temp:~ [master, 3306]> vi ~/data/slave1/my.cnf [mysqld] port = 3307 basedir = /home/mysql/product/mysql-5.6.21 datadir = /home/mysql/data/slave1 socket = /home/mysql/data/slave1/slave1.sock user = mysql
  • Launch the MyEnv installer and add the new instance: mysql@centos-temp:~ [master, 3306]> product/myenv/bin/installMyEnv.sh PHP is installed on /usr/bin/php Starting MyEnv installer: /home/mysql/product/myenv-1.1.2/bin/installMyEnv.php Configuration file /etc/myenv/myenv.conf already exists. Use this, overwrite with template or abort (U,t,a): u The following instances are available: master An instance is the same as a mysqld process. What do you want to do next? o Add a new instance, o change an existing instance, o delete an existing instance, o save configuration and exit or o quit without saving (A/c/d/s/q)? a You have to enter a new instance name. An instance name can consist of 0-9, a-z, A-Z, - and _. Examples: test, prod, dev, [mysqld1] mysqld-3306 Enter a new instance name: slave1 Changing instance slave1: Set basedir. The basedir is the directory where your MySQL binaries are located. Example: /home/mysql/product/myenv /home/mysql/product/myenv-1.1.2 /home/mysql/product/mysql-5.6.21 /home/mysql/product/mysql-5.6.21-linux-glibc2.5-x86_64 /usr/local/mysql /opt/mysql [/usr] basedir = /home/mysql/product/mysql-5.6.21 /home/mysql/product/mysql-5.6.21 seems to exist and has already some files in it. Omitting installation of binary tar ball. Set datadir. The datadir is the directory where your MySQL data are located. Example: /usr/local/mysql/data, /opt/mysql/data or /var/lib/mysql or /home/mysql/product/mysql-5.6.21/data or [/home/mysql/data/slave1] or /home/mysql/data/slave1 datadir = /home/mysql/data/slave1 WARNING: directory /home/mysql/data/slave1 already exist. Shall I install a mysql database under /home/mysql/data/slav1 (Y/n)? y Installing MySQL instance. Please be patient, this can take a while... /home/mysql/product/mysql-5.6.21/scripts/mysql_install_db --datadir=/home/mysql/data/slave1 --basedir=/home/mysql/product/mysql-5.6.21 Set port. Example: 3307 or [3306] port = 3307 Set socket. Example: /tmp/mysql-3307.sock /tmp/mysql-slave1.sock [/tmp/mysql.sock] /var/run/mysqld/mysql.sock socket = /home/mysql/data/slave1/slave1.sock Choose location of my.cnf: Example: /etc/my.cnf /etc/mysql/my.cnf /home/mysql/data/slave1/my-3307.cnf [/home/mysql/data/slave1/my.cnf] my.cnf = /home/mysql/data/slave1/my.cnf The following instances are available: master slave1 An instance is the same as a mysqld process. What do you want to do next? o Add a new instance, o change an existing instance, o delete an existing instance, o save configuration and exit or o quit without saving (a/c/d/S/q)? s Writing the configuration file... Rename /etc/myenv/myenv.conf to /etc/myenv/myenv.conf.2014-10-15-06-44-04 Write /etc/myenv/myenv.conf Now source your profile as follows: . ~/.bash_profile
  • Then we logout and login again to load the new configs: mysql@centos-temp:~ [master, 3306]> exit logout [root@centos-temp product]# su - mysql Up : master (5.6.21) Down : slave1 (5.6.21) master ( 3306) : test slave1 ( 3307) : test mysql@centos-temp:~ [master, 3306]>

MyEnv says that we have two instances (master and slave1), master is up and running while slave1 is stopped. Let's start it then ..

  • First, all commands we execute are sent to the master instance, we should change to the slave1 instance: mysql@centos-temp:~ [master, 3306]> slave1 mysql@centos-temp:~ [slave1, 3307]> start . SUCCESS! mysql@centos-temp:~ [slave1, 3307]>
  • Both instances are now up and running ... mysql@centos-temp:~ [slave1, 3307]> up Up : master (5.6.21) slave1 (5.6.21) Down : master ( 3306) : test slave1 ( 3307) : test mysql@centos-temp:~ [slave1, 3307]>
Instances navigation and MySQL connection

After we installed the two instances we need to know how can we navigate between them to open MySQL connections.

  • Open a MySQL connection to the master instance: mysql@centos-temp:~ [master, 3306]> mysql -u root Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.6.21 MySQL Community Server (GPL) Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. root@localhost master [(none)] SQL>
  • While the instance name is already printed in the terminal we need to double check that by a MySQL command (the used port) and exit after that: root@localhost master [(none)] SQL> SHOW GLOBAL VARIABLES LIKE 'port'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | port | 3306 | +---------------+-------+ 1 row in set (0.00 sec) root@localhost master [(none)] SQL> exit Bye mysql@centos-temp:~ [master, 3306]>
  • Open a MySQL connection to the slave1 instance and check the used port: mysql@centos-temp:~ [master, 3306]> slave1 mysql@centos-temp:~ [slave1, 3307]> mysql -u root Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 5 Server version: 5.6.21 MySQL Community Server (GPL) Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. root@localhost slave1 [(none)] SQL> SHOW GLOBAL VARIABLES LIKE 'port'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | port | 3307 | +---------------+-------+ 1 row in set (0.00 sec) root@localhost slave1 [(none)] SQL>
  • Navigate back to the master instance: mysql@centos-temp:~ [slave1, 3307]> master mysql@centos-temp:~ [master, 3306]>

More information about MyEnv commands can be checked here.

Notes:
  • Using the same procedures above we can add several instances using either the same binaries or some others.
  • We used the smallest configurations in the my.cnf file, you are free to make your own changes.

Migration between MySQL/Percona Server and MariaDB

Shinguz - Wed, 2014-10-08 21:40
Taxonomy upgrade extras: migrationsidegradeupgrademysqlmariadbpercona serverconversioncompatibility

This week we did some migrations from MariaDB 10.0 to Percona Server 5.6 at the IT department of a big German bank.

We were perfectly aware that since version 10.0 the MariaDB code base started to diverge slightly away from the MySQL and Percona Server code base which are still pretty close to each other.

Because of the Percona Server option enforce_storage_engine we wanted to do this migration.

We stopped the MariaDB 10.0 server replaced the binaries by the Percona Server 5.6 binaries and started the Percona 5.6 server again. After successfully starting the instance we found some error messages in the MySQL error log. By running the mysql_upgrade command some of the problems were fixed but not all of them. Still left problems were:

  • The MariaDB binary logs provoked some error messages for the Percona Server: [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 25, event_type: -93 [Warning] Error reading GTIDs from binary log: -1 [ERROR] Incorrect definition of table mysql.db: expected column 'User' at position 2 to have type char(16), found type char(80). [ERROR] Incorrect definition of table mysql.event: expected column 'definer' at position 3 to have type char(77), found type char(141). [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set... A purge of the binary logs solved this issue.
  • The tables mysql.event, mysql.innodb_table_stats and mysql.innodb_index_stats where not fixed by mysql_upgrade (a bug to fix for Percona and MySQL/Oracle?). We had to replace those tables manually by copying from an other already working Percona 5.6 Server.

Later in the FromDual technology labs we investigated further and tried the other way from Percona Server 5.6 to MariaDB 10.0. In this direction we found some other errors in the MySQL error log which also where not completely resolved by the mysql_upgrade utility:

  • The mysql.innodb_table_stats and mysql.innodb_index_stats tables where recreated manually (here a bug to fix for the MariaDB people?).
  • All error messages from tables affected by the following message: InnoDB: in InnoDB data dictionary has unknown flags 40/50/52. could be silenced by a run of the OPTIMIZE TABLE command (which can become quite expensive for very big tables).

Sidegrades from MySQL 5.6 to Percona Server 5.6 and back did not provoke any error message written to the MySQL log files. Sidegrades from MariaDB 10.0 to MySQL 5.6 and vice versa behaved exactly the same as MariaDB 10.0 to Percona Server 5.6 and back.

from/to: MySQL 5.6 MariaDB 10.0 Percona Server 5.6 MySQL 5.6 - 2 tables, OPTIMIZE OK MariaDB 10.0 binlog, 3 tables - binlog, 3 tables Percona Server 5.6 OK 2 tables, OPTIMIZE -

During our tests we got rid of the error messages. If they caused any technical harm to the tables or the data we cannot say so far. Further testing and experience from real life is needed. Any feedback is welcome!

Observations

It looks like MariaDB 10.0 understands MySQL/Percona Server replication but not the other way around. So replication from MariaDB 10.0 to MySQL 5.6 does probably not work (different implementation of GTID)?

Recommendation

To make sure a sigdegrade between these 3 MySQL branches/forks is seamlessly possible the best method seems to be to dump/restore (NOT xtrabackup!) the data. This can be an issue with huge databases (hundreds of Gbyte).

Further aid

Also have a look at our MySQL compatibility matrix and our MySQL upgrade check-list.

If you need any help to convert MySQL to MariaDB to Percona Server or the other way do not hesitate to contact the FromDual consultancy team. We will be pleased to assist you as a neutral and vendor independent consulting company.

MySQL Environment MyEnv 1.1.1 has been released

FromDual.en - Mon, 2014-09-08 08:29

FromDual has the pleasure to announce the release of the new version 1.1.1 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.1.0 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Upgrade from 1.0.5 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.1 MyEnv
  • Function to check upgrade requirements to v1.1 implemented.
MyEnv Installer
  • none
MyEnv Utilities
  • none
MySQL Backup Manager
  • End tag added to mysql_bman code in certain places where it was missing.
  • innodbackup bug fixed, pigz bug fixed.
  • Warning message more verbose explaining solution in case of missing xtrabackup binaries.
  • pigz as alternative compression mechanism implemented.
  • Log mysqldump error to stderr.
  • my_exec replaces exec/system calls.
  • Return code exits fixed.
  • Cleanup date is formated nicer and cleanup bug is fixed.
  • copy_preserve introduced to preserve timestamp and archive code cleaned up.
  • Retention time can be 0 or any value out of h, d, w, m.
  • Binary log begin and end ts is written to log file.
MySQL Backup Manager Catalog
  • Catalog DB handle was sometimes called without a catalog (ugly bug introduced in v1.1.0!).
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalogrelease

MySQL Environment MyEnv 1.1.1 has been released

FromDual.en - Mon, 2014-09-08 08:29
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalog

FromDual has the pleasure to announce the release of the new version 1.1.1 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.1.0 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/
Upgrade from 1.0.5 to 1.1.1 # cd ${HOME}/product # tar xf /download/myenv-1.1.1.tar.gz # rm -f myenv # ln -s myenv-1.1.1 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.1 MyEnv
  • Function to check upgrade requirements to v1.1 implemented.
MyEnv Installer
  • none
MyEnv Utilities
  • none
MySQL Backup Manager
  • End tag added to mysql_bman code in certain places where it was missing.
  • innodbackup bug fixed, pigz bug fixed.
  • Warning message more verbose explaining solution in case of missing xtrabackup binaries.
  • pigz as alternative compression mechanism implemented.
  • Log mysqldump error to stderr.
  • my_exec replaces exec/system calls.
  • Return code exits fixed.
  • Cleanup date is formated nicer and cleanup bug is fixed.
  • copy_preserve introduced to preserve timestamp and archive code cleaned up.
  • Retention time can be 0 or any value out of h, d, w, m.
  • Binary log begin and end ts is written to log file.
MySQL Backup Manager Catalog
  • Catalog DB handle was sometimes called without a catalog (ugly bug introduced in v1.1.0!).

MySQL Environment MyEnv 1.1.0 has been released

FromDual.en - Sat, 2014-08-23 19:46

FromDual has the pleasure to announce the release of the new version 1.1.0 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.0.5 to 1.1.0 # cd ${HOME}/product # tar xf /download/myenv-1.1.0.tar.gz # rm -f myenv # ln -s myenv-1.1.0 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.0 MyEnv
  • Error with !includedir fixed (Bug #143).
  • tput is hidden in interactive mode now. No more nasty warning messages.
  • Start/stop script did not pass error code correctly.
  • Timezone complain on myenv_start_stop.php fixed with temporary fix.
  • Init script should not through errors on SuSE any more.
  • Init script is improved. Myenv should only start if network is available (runlevel).
  • mysqladmin complains now if my.cnf has wrong parameters and does not show the schema down any more.
  • MYSQL_HOME was changed from basedir to datadir, consider this for upgrade, this is a significant change, impact is not clear!
  • Under some circumstances up did not show all schemata, fixed.
MyEnv Installer
  • No changes.
MyEnv Utilities
  • NUMA maps script from Jeremy Cole added.
  • block_galera_node.sh fixed for more than 1 back-end.
MySQL Backup Manager
  • For many backup types cleanup, archive and compress functionality added.
  • Schema backup per-schema writes log pos now to log file and catalog.
  • Binlog policy added.
  • Backup structure is not created any more for cleanup and catalog operations.
  • Binlog backup should work now with and without default location.
  • Passing strange characters in password is handled more robust now.
  • Schema names more robust with back-ticks.
  • memory-table-check bug fixed.
  • Command line is displayed in output (without exposing password).
  • Cleanup of archivedir is possible now.
  • Backup for configuration file implemented.
  • Physical backup based on xtrabackup/mysqlbackup implemented.
  • All exit's replaced by returns. Proper error handling should be possible.
  • Backup logging implemented.
  • Password from command line is not exposed any more to log file. Back-port from v1.0.5.
  • Per schema privilege backup implemented.
MySQL Backup Manager Catalog
  • Catalog creation and catalog upgrade integrated.
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalogrelease

MySQL Environment MyEnv 1.1.0 has been released

FromDual.en - Sat, 2014-08-23 19:46
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackupcatalog

FromDual has the pleasure to announce the release of the new version 1.1.0 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened in the MySQL Backup Manager (mysql_bman) where we added a backup catalog.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.0.5 to 1.1.0 # cd ${HOME}/product # tar xf /download/myenv-1.1.0.tar.gz # rm -f myenv # ln -s myenv-1.1.0 myenv

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Replace the following files:

  • cp ${HOME}/product/myenv/etc/variables.conf.template /etc/myenv/variables.conf
  • cp ${HOME}/product/myenv/bin/myenv.server /etc/init.d/myenv
Changes in MyEnv 1.1.0 MyEnv
  • Error with !includedir fixed (Bug #143).
  • tput is hidden in interactive mode now. No more nasty warning messages.
  • Start/stop script did not pass error code correctly.
  • Timezone complain on myenv_start_stop.php fixed with temporary fix.
  • Init script should not through errors on SuSE any more.
  • Init script is improved. Myenv should only start if network is available (runlevel).
  • mysqladmin complains now if my.cnf has wrong parameters and does not show the schema down any more.
  • MYSQL_HOME was changed from basedir to datadir, consider this for upgrade, this is a significant change, impact is not clear!
  • Under some circumstances up did not show all schemata, fixed.
MyEnv Installer
  • No changes.
MyEnv Utilities
  • NUMA maps script from Jeremy Cole added.
  • block_galera_node.sh fixed for more than 1 back-end.
MySQL Backup Manager
  • For many backup types cleanup, archive and compress functionality added.
  • Schema backup per-schema writes log pos now to log file and catalog.
  • Binlog policy added.
  • Backup structure is not created any more for cleanup and catalog operations.
  • Binlog backup should work now with and without default location.
  • Passing strange characters in password is handled more robust now.
  • Schema names more robust with back-ticks.
  • memory-table-check bug fixed.
  • Command line is displayed in output (without exposing password).
  • Cleanup of archivedir is possible now.
  • Backup for configuration file implemented.
  • Physical backup based on xtrabackup/mysqlbackup implemented.
  • All exit's replaced by returns. Proper error handling should be possible.
  • Backup logging implemented.
  • Password from command line is not exposed any more to log file. Back-port from v1.0.5.
  • Per schema privilege backup implemented.
MySQL Backup Manager Catalog
  • Catalog creation and catalog upgrade integrated.

Galera Cluster VS PXC VS MariaDB Galera Cluster - Benchmarking

Abdel-Mawla Gharieb - Thu, 2014-08-07 15:36

It is not clear for many MySQL users that Percona XtraDB Cluster (PXC) and MariaDB Galera Cluster depend on the same Galera library i.e used in Galera Cluster for MySQL which is provided by Codership team:

  • Galera Cluster: MySQL Server (by Oracle) + Galera library.
  • Percona XtraDB Cluster: Percona Server + Galera library.
  • MariaDB Galera Cluster: MariaDB Server + Galera library.

But the question is, are there any performance differences between the three of them ?

Let's discover that by doing some simple benchmark to test MySQL write performance in Galera Cluster, PXC and MariaDB Galera Cluster installations.

System Information: HW configurations (AWS Servers): Nodes Servers HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 8, # of threads 16, HT enabled).
  • Memory: 16GB RAM.
  • Storage: HDD 120GB/ 5400RPM.
Load balancer Server HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2651 v2 @ 1.80GHz (# of cores 4, # of threads 8, HT enabled).
  • Memory: 16GB RAM.
  • Storage: HDD 10GB/ 5400RPM.
Load generator Server HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 16, # of threads 32, HT enabled).
  • Memory: 32GB RAM.
  • Storage: HDD 10GB/ 5400RPM.
Software configurations:
  • OS : Red Hat Enterprise Linux Server release 6.5 (Santiago)
  • Sysbench : 0.5.3
  • GLB : 1.0.0
  • Galera Cluster : 5.5.34 and 5.6.16
  • Percona XtraDB Cluster : 5.5.37 and 5.6.19
  • MariaDB Galera Cluster : 5.5.38 and 10.0.12
  • Galera Library : 3.5
Test Information:
  • The testing environment consists of 5 AWS servers, three servers for a three-node cluster (each node is installed on a single server), one server for the load balancer and the final server for the load generator in which sysbench is installed to send requests to the load balancer from.
  • Sysbench command: sysbench --num-threads=64 --max-requests=1000 --db-driver=mysql --test=/usr/share/doc/sysbench/tests/db/oltp.lua --mysql-table-engine=InnoDB --mysql-user=dev --mysql-password='test' --mysql-host=load_balancer_ip run .
  • Table structure which was used by sysbench tests: mysql> show create table sbtest.sbtest\G CREATE TABLE `sbtest` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `k` int(10) unsigned NOT NULL DEFAULT '0', `c` char(120) NOT NULL DEFAULT '', `pad` char(60) NOT NULL DEFAULT '', PRIMARY KEY (`id`), KEY `k` (`k`) ) ENGINE=InnoDB AUTO_INCREMENT=8574 DEFAULT CHARSET=latin1
  • The my.cnf used is something like: [mysqld] key_buffer_size = 16M max_allowed_packet = 16M thread_stack = 192K thread_cache_size = 8 innodb_buffer_pool_size = 8G innodb_flush_log_at_trx_commit = 0 expire_logs_days = 10 max_binlog_size = 100M server-id = 1 log-bin = mysql-bin binlog_format = ROW auto_increment_increment = 3 auto_increment_offset = 1 log_slave_updates default_storage_engine = InnoDB # Path to Galera library wsrep_provider = /usr/lib64/galera/libgalera_smm.so # Cluster connection URL contains the IPs of node#1, node#2 and node#3 wsrep_cluster_address = gcomm://nodeB-IP,nodeC-IP innodb_autoinc_lock_mode = 2 # Node #1 address wsrep_node_address = nodeA-IP # Cluster name wsrep_cluster_name = test_cluster # SST method wsrep_sst_method = rsync # Authentication for SST method wsrep_sst_auth = "sst:password"

Notes:

  • The number of threads used in this test is 64 as it generated the highest throughput on all cluster installations.
  • Each throughput value for each test case is generated by the average of ten (10) times execution.
Testing Results:









The raw results in Transactions / Sec might be useful:


sync_binlog=0innodb_flush_log_ at_trx_commitGalera Cluster 5.5.34PXC 5.5.37MariaDB Galera Cluster 5.5.38Galera Cluster 5.6.16PXC 5.6.15MariaDB Galera Cluster 10.0.120525.119534.022534.249519.575532.19520.7361125.615131.748341.384157.001162.783174.972526.761528.858524.039511.817526.06521.024sync_binlog=10242.201249.622262.516220.313229.807220.97196.82996.759148.815111.995114.8113.0562224.476210.904217.142209.139201.596214.311
Conclusion

According to the above results:

  • innodb_flush_log_at_trx_commit = 1 significantly slows down Galera.
  • sync_binlog also cuts in half the throughput.
  • All other are more or less equal in throughput.

Galera Cluster VS PXC VS MariaDB Galera Cluster - Benchmarking

Abdel-Mawla Gharieb - Thu, 2014-08-07 15:36

It is not clear for many MySQL users that Percona XtraDB Cluster (PXC) and MariaDB Galera Cluster depend on the same Galera library i.e used in Galera Cluster for MySQL which is provided by Codership team:

  • Galera Cluster: MySQL Server (by Oracle) + Galera library.
  • Percona XtraDB Cluster: Percona Server + Galera library.
  • MariaDB Galera Cluster: MariaDB Server + Galera library.

But the question is, are there any performance differences between the three of them ?

Let's discover that by doing some simple benchmark to test MySQL write performance in Galera Cluster, PXC and MariaDB Galera Cluster installations.

System Information: HW configurations (AWS Servers): Nodes Servers HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 8, # of threads 16, HT enabled).
  • Memory: 16GB RAM.
  • Storage: HDD 120GB/ 5400RPM.
Load balancer Server HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2651 v2 @ 1.80GHz (# of cores 4, # of threads 8, HT enabled).
  • Memory: 16GB RAM.
  • Storage: HDD 10GB/ 5400RPM.
Load generator Server HW configurations:
  • CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (# of cores 16, # of threads 32, HT enabled).
  • Memory: 32GB RAM.
  • Storage: HDD 10GB/ 5400RPM.
Software configurations:
  • OS : Red Hat Enterprise Linux Server release 6.5 (Santiago)
  • Sysbench : 0.5.3
  • GLB : 1.0.0
  • Galera Cluster : 5.5.34 and 5.6.16
  • Percona XtraDB Cluster : 5.5.37 and 5.6.19
  • MariaDB Galera Cluster : 5.5.38 and 10.0.12
  • Galera Library : 3.5
Test Information:
  • The testing environment consists of 5 AWS servers, three servers for a three-node cluster (each node is installed on a single server), one server for the load balancer and the final server for the load generator in which sysbench is installed to send requests to the load balancer from.
  • Sysbench command: sysbench --num-threads=64 --max-requests=1000 --db-driver=mysql --test=/usr/share/doc/sysbench/tests/db/oltp.lua --mysql-table-engine=InnoDB --mysql-user=dev --mysql-password='test' --mysql-host=load_balancer_ip run .
  • Table structure which was used by sysbench tests: mysql> show create table sbtest.sbtest\G CREATE TABLE `sbtest` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `k` int(10) unsigned NOT NULL DEFAULT '0', `c` char(120) NOT NULL DEFAULT '', `pad` char(60) NOT NULL DEFAULT '', PRIMARY KEY (`id`), KEY `k` (`k`) ) ENGINE=InnoDB AUTO_INCREMENT=8574 DEFAULT CHARSET=latin1
  • The my.cnf used is something like: [mysqld] key_buffer_size = 16M max_allowed_packet = 16M thread_stack = 192K thread_cache_size = 8 innodb_buffer_pool_size = 8G innodb_flush_log_at_trx_commit = 0 expire_logs_days = 10 max_binlog_size = 100M server-id = 1 log-bin = mysql-bin binlog_format = ROW auto_increment_increment = 3 auto_increment_offset = 1 log_slave_updates default_storage_engine = InnoDB # Path to Galera library wsrep_provider = /usr/lib64/galera/libgalera_smm.so # Cluster connection URL contains the IPs of node#1, node#2 and node#3 wsrep_cluster_address = gcomm://nodeB-IP,nodeC-IP innodb_autoinc_lock_mode = 2 # Node #1 address wsrep_node_address = nodeA-IP # Cluster name wsrep_cluster_name = test_cluster # SST method wsrep_sst_method = rsync # Authentication for SST method wsrep_sst_auth = "sst:password"

Notes:

  • The number of threads used in this test is 64 as it generated the highest throughput on all cluster installations.
  • Each throughput value for each test case is generated by the average of ten (10) times execution.
Testing Results:









The raw results in Transactions / Sec might be useful:


sync_binlog=0innodb_flush_log_ at_trx_commitGalera Cluster 5.5.34PXC 5.5.37MariaDB Galera Cluster 5.5.38Galera Cluster 5.6.16PXC 5.6.15MariaDB Galera Cluster 10.0.120525.119534.022534.249519.575532.19520.7361125.615131.748341.384157.001162.783174.972526.761528.858524.039511.817526.06521.024sync_binlog=10242.201249.622262.516220.313229.807220.97196.82996.759148.815111.995114.8113.0562224.476210.904217.142209.139201.596214.311
Conclusion

According to the above results:

  • innodb_flush_log_at_trx_commit = 1 significantly slows down Galera.
  • sync_binlog also cuts in half the throughput.
  • All other are more or less equal in throughput.

FromDual invites MySQL community to company meeting 2014 in Barcelona

FromDual.en - Wed, 2014-08-06 14:29

FromDual holds its annual company meeting this year in Barcelona, Spain.

We are pleased to invite everybody interested in MySQL technologies (MySQL, Galera Cluster, FromDual Tools, Percona Cluster, MariaDB, etc.) to participate on Thursday evening September 11 at the HCC MONTBLANC, Via Laietana 61, to meet, exchange ideas and discuss MySQL related topics.

The event starts at 18:00, we will meet in the hotel lobby. The planned schedule is:

  • How to Implement GTID Replication in MySQL 5.6 (25') and 5' Questions and Answers.
  • MySQL backup/restore for anonymized exports (25') and 5' Questions and Answers.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.

Please feel free to send us your suggestion about your presentation. Any technical or non-technical MySQL related topic is welcome. For example how you use MySQL in your company or special problems you have faced and solved (or not solved yet), research work you have done on MySQL products, business cases you solve with MySQL products, evaluations or experience you have made, etc. For the proposal please send us a mail.

Please also send us a short notice if you plan to participate or join us with MeetUp.

This gives us the possibility to arrange and organize all the infrastructure with the Hotel.

The event is free of costs for all participants.

We are pleased if you can make it to the event,
Your FromDual Team

FromDual invites MySQL community to company meeting 2014 in Barcelona

FromDual.en - Wed, 2014-08-06 14:29

FromDual holds its annual company meeting this year in Barcelona, Spain.

We are pleased to invite everybody interested in MySQL technologies (MySQL, Galera Cluster, FromDual Tools, Percona Cluster, MariaDB, etc.) to participate on Thursday evening September 11 at the HCC MONTBLANC, Via Laietana 61, to meet, exchange ideas and discuss MySQL related topics.

The event starts at 18:00, we will meet in the hotel lobby. The planned schedule is:

  • How to Implement GTID Replication in MySQL 5.6 (25') and 5' Questions and Answers.
  • MySQL backup/restore for anonymized exports (25') and 5' Questions and Answers.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.
  • Break 15 min
  • Quick presentation (15') of YOUR project or company.
  • Quick presentation (15') of YOUR project or company.

Please feel free to send us your suggestion about your presentation. Any technical or non-technical MySQL related topic is welcome. For example how you use MySQL in your company or special problems you have faced and solved (or not solved yet), research work you have done on MySQL products, business cases you solve with MySQL products, evaluations or experience you have made, etc. For the proposal please send us a mail.

Please also send us a short notice if you plan to participate or join us with MeetUp.

This gives us the possibility to arrange and organize all the infrastructure with the Hotel.

The event is free of costs for all participants.

We are pleased if you can make it to the event,
Your FromDual Team

FromDual: Tools for MySQL and Galera - Backup - Monitoring - Operations

FromDual.en - Sun, 2014-07-27 10:46

FromDual tools provide valuable additional functionality which facilitate and optimize daily operations of your MySQL databases. Since our last newsletter a lot of things have changed in the FromDual tools.


To the MySQL Environment (MyEnv) numerous improvements and suggestions of our customers were added. The most important changes were introduced to the MySQL Backup Manager (mysql_bman).


With the MySQL Ops Center we meet the wishes of our numerous customers, which desire a graphical user interface for operating complex MySQL environments.
These users are often little used in handling MySQL but want nevertheless operate more complex MySQL installations as Master/Slave- or Master/Master-replications.


In the MySQL Performance Monitor (mpm) numerous little bugs have been fixed which were reported to us by our customers.


Note: In our MySQL Service Contracts, Business Hour (5x9) and All around the Clock (7x24) the usage and support of our tools is included.
If you would like to know more about our service prices, we are pleased to send you an offer.



MyEnv v1.0.5

The MySQL Environment (MyEnv) gets more and more popularity in the MySQL eco-system. MyEnv is optimized for MySQL and mimics the popular TVD BasEnv which is popular with bigger Oracle database users.


With MyEnv you can easily consolidate several MySQL instances (mysqld) on one single machine. Thanks to MyEnv this complicated configuration is a piece of cake. Furthermore MyEnv is more and more common with customers testing their applications against different MySQL versions (5.5, 5.6 and 5.7) or different MySQL branches (Galera Cluster, MariaDB, Percona Server).


The most important improvements in MyEnv v1.0.5 are:

  • Old PHP functions were replaced to achieve better compatibility with PHP 5.4 and 5.5.
  • MyEnv overview (up) of installed MySQL instances was polished and numerous smaller bugs were fixed.
  • Extensions for active/passive fail-over clusters and Oracle Enterprise Monitor Agents for MySQL were integrated.
  • The user guidance of the MyEnv installer was made more user friendly.
  • Problems of MyEnv with SuSE Linux Enterprise Server (SLES) were removed.
  • The tools for MySQL Partitions were extended and improved.

All improvements in detail you can find in the Release Notes.


Here you can download MyEnv.



MySQL Backup Manager v1.0.5

The MySQL Backup Manager (mysql_bman) is actually getting most of interest from our customers. It significantly eases backups for MySQL for all different types of backups.


At this point we would like to quote a MySQL user:

"MySQL Backup Manager is a very nice tool! Congratulations for FromDual! I made my own shell script for catalog and maintained backups by xtrabackup, but mysql_bman is the best! Xtrabackup + mysql_bman!!!"


In mysql_bman version v1.0.5 the following improvements were integrated:

  • Security improvements (password is not exposed any more).
  • Every instance can be tagged with a name and uniquely identified.
  • The MySQL Backup Manager considers now the MySQL configuration file ~/.my.cnf.
  • The compression of backups can be disabled to support de-duplicating drives.
  • The option --no-memory-table-check was introduced to allow inconsistent backups with MEMORY tables.

Download (included in MyEnv).



MySQL Ops Center v0.2

Our MySQL customers have requested a simple user interface to operate and administer many MySQL databases. This is the reason why FromDual has launched the MySQL Ops Center.
The Ops Center can centrally operate and control complex MySQL configurations like Master/Slave or Master/Master set-ups, monitor, start, stop and reconfigure the replication.
With the MySQL Ops Center you can also start and stop easily virtual IP's and move them to an other host.


The most important features, which were added to the first public preview-release of the MySQL Ops Center v0.2. are:

  • Starting and stopping of MySQL databases on remote machines by a central management console.
  • Starting and stopping of the MySQL replication.
  • Starting and stopping of a virtual IP (VIP).
  • Fail-over of VIP from active master to slave (master/slave replication) or passive master (master/master replication).
  • Configuration of the master/slave replication.

The MySQL Ops Center can be downloaded here. Further information you can find at MySQL Ops Center.



MySQL Performance Monitor v0.9.3

The MySQL Performance Monitor (mpm) was optimized in many places. Further know bugs were fixed and the mpm agent was made ready for the newest Zabbix version v2.2:

  • Bugs related to sha/sha1 encryption were fixed.
  • A stopped database is better detected now.
  • DRBD informations were improved.
  • New behaviour of zabbix_senders in Zabbix v2.2 is handled correctly now.
  • New measuring points was added (Galera Cluster) and wrong ones fixed.

You can download the latest version of MySQL Performance Monitor from here here and for more information about the manual installation just follow up the steps on the installation guide. To check all changes and improvements of MySQL Performance Monitor check out the Release Notes.

Taxonomy upgrade extras: galeraBackupmanagermonitoringOperationsMyEnvfromdual_brman

FromDual: Tools for MySQL and Galera - Backup - Monitoring - Operations

FromDual.en - Sun, 2014-07-27 10:46
Taxonomy upgrade extras: galeraBackupmanagermonitoringOperations

FromDual tools provide valuable additional functionality which facilitate and optimize daily operations of your MySQL databases. Since our last newsletter a lot of things have changed in the FromDual tools.


To the MySQL Environment (MyEnv) numerous improvements and suggestions of our customers were added. The most important changes were introduced to the MySQL Backup Manager (mysql_bman).


With the MySQL Ops Center we meet the wishes of our numerous customers, which desire a graphical user interface for operating complex MySQL environments.
These users are often little used in handling MySQL but want nevertheless operate more complex MySQL installations as Master/Slave- or Master/Master-replications.


In the MySQL Performance Monitor (mpm) numerous little bugs have been fixed which were reported to us by our customers.


Note: In our MySQL Service Contracts, Business Hour (5x9) and All around the Clock (7x24) the usage and support of our tools is included.
If you would like to know more about our service prices, we are pleased to send you an offer.



MyEnv v1.0.5

The MySQL Environment (MyEnv) gets more and more popularity in the MySQL eco-system. MyEnv is optimized for MySQL and mimics the popular TVD BasEnv which is popular with bigger Oracle database users.


With MyEnv you can easily consolidate several MySQL instances (mysqld) on one single machine. Thanks to MyEnv this complicated configuration is a piece of cake. Furthermore MyEnv is more and more common with customers testing their applications against different MySQL versions (5.5, 5.6 and 5.7) or different MySQL branches (Galera Cluster, MariaDB, Percona Server).


The most important improvements in MyEnv v1.0.5 are:

  • Old PHP functions were replaced to achieve better compatibility with PHP 5.4 and 5.5.
  • MyEnv overview (up) of installed MySQL instances was polished and numerous smaller bugs were fixed.
  • Extensions for active/passive fail-over clusters and Oracle Enterprise Monitor Agents for MySQL were integrated.
  • The user guidance of the MyEnv installer was made more user friendly.
  • Problems of MyEnv with SuSE Linux Enterprise Server (SLES) were removed.
  • The tools for MySQL Partitions were extended and improved.

All improvements in detail you can find in the Release Notes.


Here you can download MyEnv.



MySQL Backup Manager v1.0.5

The MySQL Backup Manager (mysql_bman) is actually getting most of interest from our customers. It significantly eases backups for MySQL for all different types of backups.


At this point we would like to quote a MySQL user:

"MySQL Backup Manager is a very nice tool! Congratulations for FromDual! I made my own shell script for catalog and maintained backups by xtrabackup, but mysql_bman is the best! Xtrabackup + mysql_bman!!!"


In mysql_bman version v1.0.5 the following improvements were integrated:

  • Security improvements (password is not exposed any more).
  • Every instance can be tagged with a name and uniquely identified.
  • The MySQL Backup Manager considers now the MySQL configuration file ~/.my.cnf.
  • The compression of backups can be disabled to support de-duplicating drives.
  • The option --no-memory-table-check was introduced to allow inconsistent backups with MEMORY tables.

Download (included in MyEnv).



MySQL Ops Center v0.2

Our MySQL customers have requested a simple user interface to operate and administer many MySQL databases. This is the reason why FromDual has launched the MySQL Ops Center.
The Ops Center can centrally operate and control complex MySQL configurations like Master/Slave or Master/Master set-ups, monitor, start, stop and reconfigure the replication.
With the MySQL Ops Center you can also start and stop easily virtual IP's and move them to an other host.


The most important features, which were added to the first public preview-release of the MySQL Ops Center v0.2. are:

  • Starting and stopping of MySQL databases on remote machines by a central management console.
  • Starting and stopping of the MySQL replication.
  • Starting and stopping of a virtual IP (VIP).
  • Fail-over of VIP from active master to slave (master/slave replication) or passive master (master/master replication).
  • Configuration of the master/slave replication.

The MySQL Ops Center can be downloaded here. Further information you can find at MySQL Ops Center.



MySQL Performance Monitor v0.9.3

The MySQL Performance Monitor (mpm) was optimized in many places. Further know bugs were fixed and the mpm agent was made ready for the newest Zabbix version v2.2:

  • Bugs related to sha/sha1 encryption were fixed.
  • A stopped database is better detected now.
  • DRBD informations were improved.
  • New behaviour of zabbix_senders in Zabbix v2.2 is handled correctly now.
  • New measuring points was added (Galera Cluster) and wrong ones fixed.

You can download the latest version of MySQL Performance Monitor from here here and for more information about the manual installation just follow up the steps on the installation guide. To check all changes and improvements of MySQL Performance Monitor check out the Release Notes.

FromDual Performance Monitor for MySQL 0.9.3 has been released

FromDual.en - Wed, 2014-07-09 12:25

FromDual has the pleasure to announce the release of the new version 0.9.3 of its popular Database Performance Monitor for MySQL, Galera Cluster, MariaDB and Percona Server mpm.

This release contains various minor bug fixes and improvements.

You can download mpm from here.

In the inconceivable case that you find a bug in mpm please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

New installation of mpm v0.9.3

Please follow our mpm installation guide.

Upgrade from 0.x to 0.9.3 # cd /download # tar xf mysql_performance_monitor-0.9.3.tar.gz # cd /opt # tar xf /download/mysql_performance_monitor_agent-0.9.3.tar.gz # rm -f mpm # ln -s mysql_performance_monitor_agent-0.9.3 mpm

No other upgrade requirements are known.

Changes in mpm v0.9.3 mpm agent
  • Typos fixed.
  • Kill trap reports to the log file as well now.
mpm agent and MaaS
  • Example for timeshift feature added to configuration template.
MySQL module
  • DB down not detected (bug #27/#138).
InnoDB module
  • InnoDB Status module: SHA fix (bug #139).
Master module
  • Missing values in cache file fixed.
mpm templates for Zabbix
  • No changes.
Taxonomy upgrade extras: mysqlperformancemonitormonitoringmpmmaasperformance monitorreleasefpmmm

FromDual Performance Monitor for MySQL 0.9.3 has been released

FromDual.en - Wed, 2014-07-09 12:25
Taxonomy upgrade extras: mysqlperformancemonitormonitoringmpmmaasperformance monitor

FromDual has the pleasure to announce the release of the new version 0.9.3 of its popular Database Performance Monitor for MySQL, Galera Cluster, MariaDB and Percona Server mpm.

This release contains various minor bug fixes and improvements.

You can download mpm from here.

In the inconceivable case that you find a bug in mpm please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

New installation of mpm v0.9.3

Please follow our mpm installation guide.

Upgrade from 0.x to 0.9.3 # cd /download # tar xf mysql_performance_monitor-0.9.3.tar.gz # cd /opt # tar xf /download/mysql_performance_monitor_agent-0.9.3.tar.gz # rm -f mpm # ln -s mysql_performance_monitor_agent-0.9.3 mpm

No other upgrade requirements are known.

Changes in mpm v0.9.3 mpm agent
  • Typos fixed.
  • Kill trap reports to the log file as well now.
mpm agent and MaaS
  • Example for timeshift feature added to configuration template.
MySQL module
  • DB down not detected (bug #27/#138).
InnoDB module
  • InnoDB Status module: SHA fix (bug #139).
Master module
  • Missing values in cache file fixed.
mpm templates for Zabbix
  • No changes.

Replication Troubleshooting - Classic VS GTID

Abdel-Mawla Gharieb - Fri, 2014-07-04 15:05

In previous posts, I was talking about how to set up MySQL replication, Classic Replication (based on binary logs information) and Transaction-based Replication (based on GTID). In this article I'll summarize how to troubleshoot MySQL replication for the most common issues we might face with a simple comparison how can we get them solved in the different replication methods (Classic VS GTID).

There are two main operations we might need to do in a replication setup:

  • Skip or ignore a statement that causes the replication to stop.
  • Re-initialize a slave when the Replication is broke and could not be started anymore.
Skip or Ignore statement

Basically, the slave should be always synchronized with its master having the same copy of data, but for some reasons there might be inconsistency between both of them (unsafe statement in SBR, Slave is not read_only and was modified apart of replication queries, .. etc) which causes errors and stops the replication, e.g. if the master inserted a record which was already inserted on the slave (Duplicate entry) or updated/deleted a row which was not exist on the slave, ... etc.

To solve this issue, we have to either reverse what we have done on the slave (e.g. delete the inserted rows) if that was made by mistake and is known or we can skip executing those statements on the slave and continue the replication again (I'll focus on skipping a statement in this post as it needs different interaction in Classic and GTID replication).

Sample error messages (from SHOW SLAVE STATUS output): Last_SQL_Error: Could not execute Write_rows event on table test.t1; Duplicate entry '4' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000304, end_log_pos 285 Last_SQL_Error: Could not execute Update_rows event on table test.t1; Can't find record in 't1', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000304, end_log_pos 492 Last_SQL_Error: Could not execute Delete_rows event on table test.t1; Can't find record in 't1', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000304, end_log_pos 688 How to solve that issue ?
CLASSIC REPLICATION

Solving this problem is a straight forward process in the classic replication setup, what only we need is to issue the following SQL commands on the slave's:

SQL> SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; SQL> START SLAVE;
GTID REPLICATION

Solving this problem is not a straight forward in GTID replication like it is in the Classic replication and the variable SQL_SLAVE_SKIP_COUNTER wont be useful in this area anymore.

To get this problem solved in a GTID replication we will need to inject an empty transaction as follows:

  • Check which transaction is causing the problem: SQL> SHOW SLAVE STATUS\G . . Retrieved_Gtid_Set: b9b4712a-df64-11e3-b391-60672090eb04:1-7 Executed_Gtid_Set: 4f6d62ed-df65-11e3-b395-60672090eb04:1, b9b4712a-df64-11e3-b391-60672090eb04:1-6 Auto_Position: 1

    Retrieved_Gtid_Set means the retrieved GTIDs from the master

    Executed_Gtid_Set means the executed GTIDs on the slave.

    According to the above output, the slave retrieved GTIDs from 1:7 (b9b4712a-df64-11e3-b391-60672090eb04:1-7) and executed only from 1:6 (b9b4712a-df64-11e3-b391-60672090eb04:1-6), so the problem is in transaction number 7.

  • Inject an empty transaction: SQL> SET GTID_NEXT='b9b4712a-df64-11e3-b391-60672090eb04:7'; SQL> BEGIN;COMMIT; SQL> SET GTID_NEXT='AUTOMATIC'; SQL> START SLAVE;

    BE CAUTIOUS: The first part of Executed_Gtid_Set (4f6d62ed-df65-11e3-b395-60672090eb04:1) is the local executed GTIDs (not received from the master) while the second part (b9b4712a-df64-11e3-b391-60672090eb04:1-6) is the executed GTIDs which retrieved from the master (check the master's UUID by either checking the UUID value in "Retrieved_Gtid_Set" which is basically for the master's UUID or by issuing SHOW GLOBAL VARIABLES LIKE 'server_uuid'; on the master server). So we should make sure that we are using the master's UUID when injecting an empty transaction, otherwise, the problem will still remain and the slave wont be started.

Note:

After starting the slave successfully in either classic or GTID replication we might need to use a combination of Percona tools pt-table-checksum and pt-table-sync to fix the inconsistency problem.

Re-initialize/ re-build a slave

For many reasons, we might end up with only re-build a slave to get the replication working, e.g. if we stopped a slave for a while where the master purged the binary log file that is needed by that slave or there are many duplicate entry errors so that pt-table-checksum and pt-table-sync could not be used then we have to re-initialize the slave from the beginning by having a fresh backup from the master server and restore it on the slave. Lets check how can we do that in both replication methods.

How to solve that issue ?
CLASSIC REPLICATION
Sample error message:
Last_IO_Errno: 1236 Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

Fix steps:

  • Backup the master server by the following command: shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --master-data=2 --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Restore the backup file on the slave: shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Get the binary logs information when the backup was taken: shell> head -n 50 /path/to/backupdir/full_backup-$TIMESTAMP.sql|grep "CHANGE MASTER TO" CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000011', MASTER_LOG_POS=120;
  • Issue the "CHANGE MASTER TO" command using the new information: SQL> CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000011', MASTER_LOG_POS=120;
  • Start the slave: SQL> START SLAVE;

NOTE:

Xtrabackup tool could be used instead of mysqldump,especially, if the database size is big. Check out this link for more information.

GTID REPLICATION
Sample error message:
Last_IO_Errno: 1236 Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'

Fix steps:

  • Backup the master server by the following command: shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Check the GTID value when the backup was taken: shell> head -n 50 /path/to/backupdir/full_backup-$TIMESTAMP.sql|grep PURGED SET @@GLOBAL.GTID_PURGED='b9b4712a-df64-11e3-b391-60672090eb04:1-8';
  • Reset the GTID_EXECUTED and GTID_PURGED values on the slave: SQL> RESET MASTER;
  • Restore the backup file on the slave: shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Make sure that the values of GTID_EXEUCTED and GTID_PURGED are the correct ones: SQL> SHOW GLOBAL VARIABLES LIKE 'gtid_executed'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_executed | b9b4712a-df64-11e3-b391-60672090eb04:1-8 | +---------------+------------------------------------------+ 1 row in set (0.00 sec) SHOW GLOBAL VARIABLES LIKE 'gtid_purged'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_purged | b9b4712a-df64-11e3-b391-60672090eb04:1-8 | +---------------+------------------------------------------+ 1 row in set (0.01 sec)
  • Start the slave: SQL> START SLAVE;

NOTES:

  • If we didn't reset the GTID_EXECUTED and GTID_PURGED values on the slave before restoring the backup file, the following error will be appeared:
    shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql. ERROR 1840 (HY000): @@GLOBAL.GTID_PURGED can only be set when @@GLOBAL.GTID_EXECUTED is empty.

    The above error indicates that the statement at the beginning of the backup file - which is "SET @@GLOBAL.GTID_PURGED='b9b4712a-df64-11e3-b391-60672090eb04:1-8';" - failed because GTID_PURGED cannot be set unless GTID_EXECUTED is empty. Since GTID_EXECUTED is a read only variable, the only way to empty its value is to issue "RESET MASTER" on the slave server before restoring the backup file.

  • Xtrabackup tool could be used as well instead of mysqldump to get this problem solved and without the need to reset GTID_EXECUTED and GTID_PURGED values . Check out this link for more information.
Conclusion

While GTID provides many benefits over the classic replication but it has different troubleshooting and fix strategies which must be known first before deploying GTID in production systems.

Taxonomy upgrade extras: GTIDreplication

Replication Troubleshooting - Classic VS GTID

Abdel-Mawla Gharieb - Fri, 2014-07-04 15:05

In previous posts, I was talking about how to set up MySQL replication, Classic Replication (based on binary logs information) and Transaction-based Replication (based on GTID). In this article I'll summarize how to troubleshoot MySQL replication for the most common issues we might face with a simple comparison how can we get them solved in the different replication methods (Classic VS GTID).

There are two main operations we might need to do in a replication setup:

  • Skip or ignore a statement that causes the replication to stop.
  • Re-initialize a slave when the Replication is broke and could not be started anymore.
Skip or Ignore statement

Basically, the slave should be always synchronized with its master having the same copy of data, but for some reasons there might be inconsistency between both of them (unsafe statement in SBR, Slave is not read_only and was modified apart of replication queries, .. etc) which causes errors and stops the replication, e.g. if the master inserted a record which was already inserted on the slave (Duplicate entry) or updated/deleted a row which was not exist on the slave, ... etc.

To solve this issue, we have to either reverse what we have done on the slave (e.g. delete the inserted rows) if that was made by mistake and is known or we can skip executing those statements on the slave and continue the replication again (I'll focus on skipping a statement in this post as it needs different interaction in Classic and GTID replication).

Sample error messages (from SHOW SLAVE STATUS output): Last_SQL_Error: Could not execute Write_rows event on table test.t1; Duplicate entry '4' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000304, end_log_pos 285 Last_SQL_Error: Could not execute Update_rows event on table test.t1; Can't find record in 't1', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000304, end_log_pos 492 Last_SQL_Error: Could not execute Delete_rows event on table test.t1; Can't find record in 't1', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000304, end_log_pos 688 How to solve that issue ?
CLASSIC REPLICATION

Solving this problem is a straight forward process in the classic replication setup, what only we need is to issue the following SQL commands on the slave's:

SQL> SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; SQL> START SLAVE;
GTID REPLICATION

Solving this problem is not a straight forward in GTID replication like it is in the Classic replication and the variable SQL_SLAVE_SKIP_COUNTER wont be useful in this area anymore.

To get this problem solved in a GTID replication we will need to inject an empty transaction as follows:

  • Check which transaction is causing the problem: SQL> SHOW SLAVE STATUS\G . . Retrieved_Gtid_Set: b9b4712a-df64-11e3-b391-60672090eb04:1-7 Executed_Gtid_Set: 4f6d62ed-df65-11e3-b395-60672090eb04:1, b9b4712a-df64-11e3-b391-60672090eb04:1-6 Auto_Position: 1

    Retrieved_Gtid_Set means the retrieved GTIDs from the master

    Executed_Gtid_Set means the executed GTIDs on the slave.

    According to the above output, the slave retrieved GTIDs from 1:7 (b9b4712a-df64-11e3-b391-60672090eb04:1-7) and executed only from 1:6 (b9b4712a-df64-11e3-b391-60672090eb04:1-6), so the problem is in transaction number 7.

  • Inject an empty transaction: SQL> SET GTID_NEXT='b9b4712a-df64-11e3-b391-60672090eb04:7'; SQL> BEGIN;COMMIT; SQL> SET GTID_NEXT='AUTOMATIC'; SQL> START SLAVE;

    BE CAUTIOUS: The first part of Executed_Gtid_Set (4f6d62ed-df65-11e3-b395-60672090eb04:1) is the local executed GTIDs (not received from the master) while the second part (b9b4712a-df64-11e3-b391-60672090eb04:1-6) is the executed GTIDs which retrieved from the master (check the master's UUID by either checking the UUID value in "Retrieved_Gtid_Set" which is basically for the master's UUID or by issuing SHOW GLOBAL VARIABLES LIKE 'server_uuid'; on the master server). So we should make sure that we are using the master's UUID when injecting an empty transaction, otherwise, the problem will still remain and the slave wont be started.

Note:

After starting the slave successfully in either classic or GTID replication we might need to use a combination of Percona tools pt-table-checksum and pt-table-sync to fix the inconsistency problem.

Re-initialize/ re-build a slave

For many reasons, we might end up with only re-build a slave to get the replication working, e.g. if we stopped a slave for a while where the master purged the binary log file that is needed by that slave or there are many duplicate entry errors so that pt-table-checksum and pt-table-sync could not be used then we have to re-initialize the slave from the beginning by having a fresh backup from the master server and restore it on the slave. Lets check how can we do that in both replication methods.

How to solve that issue ?
CLASSIC REPLICATION
Sample error message:
Last_IO_Errno: 1236 Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

Fix steps:

  • Backup the master server by the following command: shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --master-data=2 --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Restore the backup file on the slave: shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Get the binary logs information when the backup was taken: shell> head -n 50 /path/to/backupdir/full_backup-$TIMESTAMP.sql|grep "CHANGE MASTER TO" CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000011', MASTER_LOG_POS=120;
  • Issue the "CHANGE MASTER TO" command using the new information: SQL> CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000011', MASTER_LOG_POS=120;
  • Start the slave: SQL> START SLAVE;

NOTE:

Xtrabackup tool could be used instead of mysqldump,especially, if the database size is big. Check out this link for more information.

GTID REPLICATION
Sample error message:
Last_IO_Errno: 1236 Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'

Fix steps:

  • Backup the master server by the following command: shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Check the GTID value when the backup was taken: shell> head -n 50 /path/to/backupdir/full_backup-$TIMESTAMP.sql|grep PURGED SET @@GLOBAL.GTID_PURGED='b9b4712a-df64-11e3-b391-60672090eb04:1-8';
  • Reset the GTID_EXECUTED and GTID_PURGED values on the slave: SQL> RESET MASTER;
  • Restore the backup file on the slave: shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql
  • Make sure that the values of GTID_EXEUCTED and GTID_PURGED are the correct ones: SQL> SHOW GLOBAL VARIABLES LIKE 'gtid_executed'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_executed | b9b4712a-df64-11e3-b391-60672090eb04:1-8 | +---------------+------------------------------------------+ 1 row in set (0.00 sec) SHOW GLOBAL VARIABLES LIKE 'gtid_purged'; +---------------+------------------------------------------+ | Variable_name | Value | +---------------+------------------------------------------+ | gtid_purged | b9b4712a-df64-11e3-b391-60672090eb04:1-8 | +---------------+------------------------------------------+ 1 row in set (0.01 sec)
  • Start the slave: SQL> START SLAVE;

NOTES:

  • If we didn't reset the GTID_EXECUTED and GTID_PURGED values on the slave before restoring the backup file, the following error will be appeared:
    shell> mysql -u root -p < /path/to/backupdir/full_backup-$TIMESTAMP.sql. ERROR 1840 (HY000): @@GLOBAL.GTID_PURGED can only be set when @@GLOBAL.GTID_EXECUTED is empty.

    The above error indicates that the statement at the beginning of the backup file - which is "SET @@GLOBAL.GTID_PURGED='b9b4712a-df64-11e3-b391-60672090eb04:1-8';" - failed because GTID_PURGED cannot be set unless GTID_EXECUTED is empty. Since GTID_EXECUTED is a read only variable, the only way to empty its value is to issue "RESET MASTER" on the slave server before restoring the backup file.

  • Xtrabackup tool could be used as well instead of mysqldump to get this problem solved and without the need to reset GTID_EXECUTED and GTID_PURGED values . Check out this link for more information.
Conclusion

While GTID provides many benefits over the classic replication but it has different troubleshooting and fix strategies which must be known first before deploying GTID in production systems.

Replication channel fail-over with Galera Cluster for MySQL

Shinguz - Thu, 2014-06-19 07:05
Taxonomy upgrade extras: channelgaleraclusterfail-overreplicationmasterslave

Sometimes it could be desirable to replicate from a Galera Cluster to a single MySQL slave or to an other Galera Cluster. Reasons for this measure could be:

  • An unstable network between two Galera Cluster locations.
  • A separation of a reporting slave and the Galera Cluster so that heavy reports on the slave do not affect the Galera Cluster performance.
  • Mixing different sources in a slave or a Galera Cluster (fan-in replication).

This article is based on earlier research work (see MySQL Cluster - Cluster circular replication with 2 replication channels) and uses the old MySQL replication style (without MySQL GTID).

Preconditions
  • Enable the binary logs on 2 nodes of a Galera Cluster (we call them channel masters) with the log_bin variable.
  • Set log_slave_updates = 1 on ALL Galera nodes.
  • It is recommended to have small binary logs and relay logs in such a situation to reduce overhead of scanning the files (max_binlog_size = 100M).
Scenarios

   

Let us assume that for some reason the actual channel master of channel 1 breaks. As a consequence the slave of channel 1 does not receive any replication events any more. But we have to keep the replication stream up and running. So we have to switch the replication channel to channel master 2.

Switching replication channel

First for security reasons we should stop the slave of replication channel 1 first:

mysql> STOP SLAVE;

Then we have to find the actual relay log on the slave:

mysql> pager grep Relay_Log_File mysql> SHOW SLAVE STATUS\G mysql> nopager Relay_Log_File: slave-relay-bin.000019

Next we have to find the last applied transaction on the slave:

mysql> SHOW RELAYLOG EVENTS IN 'slave-relay-bin.000019'; | slave-relay-bin.000019 | 3386717 | Query | 5201 | 53745015 | BEGIN | | slave-relay-bin.000019 | 3386794 | Table_map | 5201 | 53745067 | table_id: 72 (test.test) | | slave-relay-bin.000019 | 3386846 | Write_rows | 5201 | 53745142 | table_id: 72 flags: STMT_END_F | | slave-relay-bin.000019 | 3386921 | Xid | 5201 | 53745173 | COMMIT /* xid=1457451 */ | +------------------------+---------+-------------+-----------+-------------+--------------------------------+

This is transaction 1457451 which is the same on all Galera nodes.

On the new channel master of channel 2 we have to find now the matching binary log. This can be done best by matching times between the relay log and the binary log of master of channel 2.

On slave:

shell> ll *relay-bin* -rw-rw---- 1 mysql mysql 336 Mai 22 20:32 slave-relay-bin.000018 -rw-rw---- 1 mysql mysql 3387029 Mai 22 20:37 slave-relay-bin.000019

On master of channel 2:

shell> ll *bin-log* -rw-rw---- 1 mysql mysql 2518737 Mai 22 19:57 bin-log.000072 -rw-rw---- 1 mysql mysql 143 Mai 22 19:57 bin-log.000073 -rw-rw---- 1 mysql mysql 165 Mai 22 20:01 bin-log.000074 -rw-rw---- 1 mysql mysql 62953648 Mai 22 20:40 bin-log.000075

It looks like binary log 75 of master 2 matches to relay log of our slave.

Now we have to find the same transaction on the master of channel 2:

mysql> pager grep -B 6 1457451 mysql> SHOW BINLOG EVENTS IN 'bin-log.000075'; mysql> nopager | bin-log.000075 | 53744832 | Write_rows | 5201 | 53744907 | table_id: 72 flags: STMT_END_F | | bin-log.000075 | 53744907 | Xid | 5201 | 53744938 | COMMIT /* xid=1457450 */ | | bin-log.000075 | 53744938 | Query | 5201 | 53745015 | BEGIN | | bin-log.000075 | 53745015 | Table_map | 5201 | 53745067 | table_id: 72 (test.test) | | bin-log.000075 | 53745067 | Write_rows | 5201 | 53745142 | table_id: 72 flags: STMT_END_F | | bin-log.000075 | 53745142 | Xid | 5201 | 53745173 | COMMIT /* xid=1457451 */ | +----------------+----------+-------------+-----------+-------------+---------------------------------------+

We successfully found the transaction and want the position of the next transaction 53745173 where we should continue replicating.

As a last step we have to set the slave to the master of replication channel 2:

mysql> CHANGE MASTER TO master_host='master2', master_port=3306, master_log_file='bin-log.000075', master_log_pos=53745173; mysql> START SLAVE;

After a while the slave has caught up and is ready for the next fail-over back.

Discussion

We found during our experiments that an IST of a channel master does not lead to a gap or loss of events in the replication stream. So restarting a channel master does not require a channel fail-over as long as an IST can be used for resyncing the channel master with the Galera Cluster.

The increase of wsrep_cluster_conf_id is NOT an indication that a channel fail-over is required.

A SST resets the binary logs so after the SST a slave will not replicate any more. So using this method should be safe to use. If you find any situation where you experience troubles with channel fail-over please let us know.

MySQL Environment MyEnv 1.0.5 has been released

FromDual.en - Fri, 2014-06-13 18:29

FromDual has the pleasure to announce the release of the new version 1.0.5 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened on the MySQL Backup Manager (mysql_bman) utility.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.0.x to 1.0.5 # cd ${HOME}/product # tar xf /download/myenv-1.0.5.tar.gz # rm -f myenv # ln -s myenv-1.0.5 myenv
Upgrade from 1.0.2 or older to 1.0.3 or newer

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Exchange the MyEnv section in ~/.bash_profile (make a backup of this file first?) by the following new one:

# BEGIN MyEnv # Written by the MyEnv installMyEnv.php script. . /etc/myenv/MYENV_BASE MYENV_PWD=`pwd` cd $MYENV_BASE/bin . myenv.profile cd $MYENV_BASE; $MYENV_BASE/bin/showMyEnvStatus.php; cd - > /dev/null cd $MYENV_PWD # END MyEnv Changes in MyEnv 1.0.5 MyEnv
  • Schema output in up was still ugly.
  • Instance output is split correctly similar to up/down display.
  • Instance list is now shorter when short instance names are used.
  • ignore-passive option added for myEnv to ignore passive databases in an active/passive fail-over cluster. Based on existence of datadir.
  • Upgrade instructions have been improved and denormalized.
  • Only display existing OEM agents, criteria is directory in oratab must exist.
  • Up instances are not reported with missing mysqladmin command (Galera binary tar balls) but it was not visible what is the reason. Reason is displayed as an error message now.
MyEnv Installer
  • Lists each basedir candidate in a separate line when adding a new instance. More conveniant for reading if many basedirs are available.
MyEnv Utilities
  • block_galera_node.sh: Insert instead of Append used for firewall rules. Only block load-balancer ports and not everything else.
  • block_galera_node.sh made more flexible.
MySQL Backup Manager
  • Cleanup job errors with missing target. Fixed for MGB.
  • Password on command line is not exposed anymore to log file.
  • Instance name optionally added to binary-log backup file names.
  • Binary logs are not cleaned-up because they are not copied with bck_ prefix (Bug #143).
  • Config file example in --help output done more nicely.
  • More strict option checking implemented.
  • All schemas with non transactional tables are shown instead of just the first one.
  • Help typo fixed and example improved.
  • --ignore-memory-table-check implemented to avoid error exit with MEMORY tables.
  • Preparation work for blocking MyISAM backup done.
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackuprelease

MySQL Environment MyEnv 1.0.5 has been released

FromDual.en - Fri, 2014-06-13 18:29
Taxonomy upgrade extras: myenvoperationMySQL Operationsmulti instanceconsolidationBackup

FromDual has the pleasure to announce the release of the new version 1.0.5 of its popular MySQL, MariaDB and Percona Server multi-instance environment MyEnv.

The majority of improvements happened on the MySQL Backup Manager (mysql_bman) utility.

You can download MyEnv from here.

In the inconceivable case that you find a bug in MyEnv please report it to our Bugtracker.

Any feedback, statements and testimonials are welcome as well! Please send them to feedback@fromdual.com.

Upgrade from 1.0.x to 1.0.5 # cd ${HOME}/product # tar xf /download/myenv-1.0.5.tar.gz # rm -f myenv # ln -s myenv-1.0.5 myenv
Upgrade from 1.0.2 or older to 1.0.3 or newer

If you are using plug-ins for showMyEnvStatus create all the links in the new directory structure:

cd ${HOME}/product/myenv ln -s ../../utl/oem_agent.php plg/showMyEnvStatus/

Exchange the MyEnv section in ~/.bash_profile (make a backup of this file first?) by the following new one:

# BEGIN MyEnv # Written by the MyEnv installMyEnv.php script. . /etc/myenv/MYENV_BASE MYENV_PWD=`pwd` cd $MYENV_BASE/bin . myenv.profile cd $MYENV_BASE; $MYENV_BASE/bin/showMyEnvStatus.php; cd - > /dev/null cd $MYENV_PWD # END MyEnv Changes in MyEnv 1.0.5 MyEnv
  • Schema output in up was still ugly.
  • Instance output is split correctly similar to up/down display.
  • Instance list is now shorter when short instance names are used.
  • ignore-passive option added for myEnv to ignore passive databases in an active/passive fail-over cluster. Based on existence of datadir.
  • Upgrade instructions have been improved and denormalized.
  • Only display existing OEM agents, criteria is directory in oratab must exist.
  • Up instances are not reported with missing mysqladmin command (Galera binary tar balls) but it was not visible what is the reason. Reason is displayed as an error message now.
MyEnv Installer
  • Lists each basedir candidate in a separate line when adding a new instance. More conveniant for reading if many basedirs are available.
MyEnv Utilities
  • block_galera_node.sh: Insert instead of Append used for firewall rules. Only block load-balancer ports and not everything else.
  • block_galera_node.sh made more flexible.
MySQL Backup Manager
  • Cleanup job errors with missing target. Fixed for MGB.
  • Password on command line is not exposed anymore to log file.
  • Instance name optionally added to binary-log backup file names.
  • Binary logs are not cleaned-up because they are not copied with bck_ prefix (Bug #143).
  • Config file example in --help output done more nicely.
  • More strict option checking implemented.
  • All schemas with non transactional tables are shown instead of just the first one.
  • Help typo fixed and example improved.
  • --ignore-memory-table-check implemented to avoid error exit with MEMORY tables.
  • Preparation work for blocking MyISAM backup done.

GTID In Action

Abdel-Mawla Gharieb - Thu, 2014-06-12 14:09

In a previous post I was talking about How to Setup MySQL Replication using the classic method (based on binary logs information). In this article I'll go through the transaction-based replication implementation using GTID in different scenarios.

The following topics will be covered in this blog:

What is the concept of GTID protocol?

GTID is a Global Transaction IDentifier which introduced in MySQL 5.6.5. It's not only unique on the server it was originated but it's unique among all servers in a replication setup.
GTID also guarantee consistency because once a transaction is committed on a server, any other transaction having the same GTID will be ignored, i.e. a committed transaction on a master will be applied only once on the slaves.

GTID consists of two parts separated by a column {source_id:transactions_id}.

WHERE

  • source_id: Normally the server's UUID on which the transaction originates. e.g. "b9b4712a-df64-11e3-b391-60672090eb04" .
  • transaction_id: A sequence number determining the order of the committed transaction.

The following is the GTID for the third transaction on a server having the uuid "b9b4712a-df64-11e3-b391-60672090eb04":
b9b4712a-df64-11e3-b391-60672090eb04:3

As a new protocol in MySQL there is a set of new related variables, the following are the most important ones (IMHO):

  • gtid-mode: ON|OFF to enable or disable GTID, this is not a Boolean variable (0 and 1 are not acceptable).
  • enforce-gtid-consistency: prevent executing the non transactionally safe statements, like:
    • CREATE TABLE .. SELECT.
    • CREATE TEMPORARY TABLE (inside a transaction).
    • Statements that update nontransactional tables inside a transaction.
  • gtid_purged: The set of transactions that have been purged from the binary logs.
  • gtid_executed: The set of transactions which already executed on that server.
  • gtid_next: The GTID which will be used for the next transaction.
GTID Replication Implementation Fresh Installations

Fresh installation means that there's no data yet in the master or in other words, we are building a replication setup from scratch.

The implementation process is divided into two parts:

MASTER'S SIDE CONFIGURATION:
  • Add the following variables to the MySQL configuration file (my.cnf): [mysqld] server-id=1 log-bin=mysql-bin binlog_format=ROW gtid-mode=on enforce-gtid-consistency log-slave-updates
  • Restart MySQL so that configuration changes take place: shell> service mysql restart
  • Create a MySQL user to be used by the slave: SQL> GRANT REPLICATION SLAVE ON *.* TO 'slave_user_name'@'slave_ip' IDENTIFIED BY 's3cret';

SLAVE'S SIDE CONFIGURATION:
  • Add the following variables to the my.cnf file: [mysqld] server-id=2 log-bin=mysql-bin binlog_format=ROW relay_log=relay-log skip-slave-start gtid-mode=on enforce-gtid-consistency log-slave-updates
  • Restart MySQL so that configuration changes take place: shell> service mysql restart
  • Set the master information on the slave's:

    Unlike the classic method, we don't need the master's binary log information and only what we need is to specify MASTER_AUTO_POSITION=1 instead:

    SQL> CHANGE MASTER TO -> MASTER_HOST='master_ip', -> MASTER_PORT=3306, -> MASTER_USER='slave_user_name', -> MASTER_PASSWORD='s3cret', -> MASTER_AUTO_POSITION=1;
  • Start replication: SQL> START SLAVE;
  • Check the replication status: SQL> show slave status\G Slave_IO_State: Waiting for master to send event Master_Host: 127.0.0.1 Master_User: gtid_repl Master_Port: 3320 Connect_Retry: 60 Master_Log_File: mysql-bin.000007 Read_Master_Log_Pos: 191 Relay_Log_File: relay-log.000004 Relay_Log_Pos: 401 Relay_Master_Log_File: mysql-bin.000007 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: . . . . Retrieved_Gtid_Set: b9b4712a-df64-11e3-b391-60672090eb04:1-2 Executed_Gtid_Set: b9b4712a-df64-11e3-b391-60672090eb04:1-2 Auto_Position: 1
Adding New Slave

It's a very simple process to add a new slave to a running replication (or setup replication with existing data) where GTID is being used:

  • Backup the master server shell> mysqldump -u root -p --all-databases --flush-privileges --single-transaction --flush-logs --triggers --routines --events --hex-blob >/path/to/backupdir/full_backup-$TIMESTAMP.sql
  • On the new slave, use the same MySQL configuration as described above (except the server id which should be unique) and restart it.
  • Restore the backup file taken from the master.
  • Use change master to with MASTER_AUTO_POSITION=1
  • Start the slave.

Is it so simple like that!! How did the slave know the backup position? What if some transactions were executed on the master after that backup?

Actually, when GTID is enabled, mysqldump includes the last transaction ID (GTID) at the time of taking the backup:

-- -- GTID state at the beginning of the backup -- SET @@GLOBAL.GTID_PURGED='b9b4712a-df64-11e3-b391-60672090eb04:1-7';

After restoring the backup the variable GTID_EXECUTED will be equal to GTID_PURGED (the above value) and when the slave starts it first sends the range of GTIDs it has executed (GTID_EXECUTED) to the master so that the master can sends back every missing transaction which was not applied yet on the slave.


NOTE:

If the database size is big (100GB or so) then using Xtrabackup tool instead of mysqldump here will be a very good idea. Check out this link for more information on how to use Xtrabackup tool to restore a slave server having GTID enabled.

Migration from classic replication to GTID replication How to perform the migration?

To migrate an already running replication using the classical method to GTID replication, the following steps should be done:

  • Ensure that all servers (master and slaves) are in the same point by setting the master server as read only (SET GLOBAL read_only=ON;) and wait until all slaves catch up the master's data.
  • Shutdown MySQL on all servers and add the GTID variables to the configuration files.
  • Beside the GTID variables, add read-only to the master's configuration and skip-slave-start to the slaves configurations.
  • Start MySQL service on all servers.
  • Issue the change master command with MASTER_AUTO_POSITION=1 on all slaves and then start them.
  • Make the master writable again by SET GLOBAL read_only=OFF; (don't forget to remove/hash it from the master's my.cnf file as well).
Is online migration from classic to GTID replication available?

At the time of writing this article, the online migration is not applicable - as you can see from the above steps - we have to shutdown ALL servers at the same time and that is because of two reasons:

  • GTID can NOT be enabled online because GTID_MODE is a read only variable (having this variable to be dynamic is already in Oracle's plan).
  • Replication can NOT be established between two or more servers having different values for GTID_MODE, i.e. either GTID is enabled on ALL servers or disabled on ALL servers.
Workaround ??

There's a feature request (by MySQL Devs team at Booking.com) to have an extra GTID mode (ANONYMOUS_IN-GTID_OUT) which allows a slave to receives anonymous transactions (transactions from master having GTID_MODE = OFF which do not have GTIDs) and assigns GTIDs for those transactions. In this case, this slave could be used as an intermediate server between master having GTID disabled and slaves having GTID enabled (it will be slave for the master and master for the other slaves)

The online migration steps would be:

  • Restart a slave (lets name it slaveA) using the GTID_MODE = ANONYMOUS_IN-GTID_OUT.
  • Rolling restart to the other slaves to use the normal GTID_MODE=ON and pointing them to slaveA as a new master.
  • Point the application to write to slaveA instead of the old master.
  • Restart the old master to use GTID_MODE=ON and having slaveA as a master.

Note: This is not yet available in Oracle binaries

More information on this could be find here.

GTID Benefits
  • Simplifies the setup of MySQL replication as master's binary logs information is not needed anymore (binary log file name and position).
  • Consistency is guaranteed between master and slave as the committed transaction on the master will be applied only once on the slave.
  • Simple to determine whether masters and slaves are consistent or not.
  • Fail-over process is much easier. When the master fail to operate, no need to calculate a slave's binary logs information before promoting it to be new master. MASTER_AUTO_POSITION=1 will do the job as all transactions in all servers inside the replication have the same GTID.
  • Automatic fail-over scripts is now much easier to implement.

To know how to troubleshoot GTID replication, check out the Replication Troubleshooting - Classic VS GTID blog.

Taxonomy upgrade extras: GTIDreplication

Pages

Subscribe to FromDual aggregator - FromDual all (en)