Upgrading an Existing EDB Failover Manager Cluster 3.0 to new EFM 3.1

Published on Tue, Jun 19, 2018

As you all know, EDB Failover Manager(EFM) provides the high availability insfrastructure for EDB Postgres clusters. Failover Manager monitors the members of a Postgres cluster, identifies and verifies database failures quickly and reliably, and if needed promotes a standby node to become the cluster master and issues alerts.

Every release of EFM has new features and functionalities and to take benefit of it you may need to upgrade your existing cluster to new version.

The Failover Manager 3.1 installer has an upgrade utility called “upgrade-conf” in “/usr/edb/efm-3.1/bin” directory, that locates the .properties & .nodes file of pre-existing EFM version 2.0,2.1 or 3.0 and creates a new copy of configuration files in /etc/edb/efm-3.1/ directory to be used by EFM 3.1 version.

Syntax:

    /usr/edb/efm-3.1/bin/efm upgrade-conf <clustername>

Let’s see in play by upgrading the existing EFM cluster 3.0 to new EFM 3.1 version.

  1. Check the existing EFM cluster status before proceeding to upgrade to EFM 3.1.

     [root@witness efm]# /usr/edb/efm-3.0/bin/efm cluster-stauts efm
     Cluster Status: efm
     VIP:
         Agent Type  Address              Agent  DB       Info
         --------------------------------------------------------------
         Master      MASTER_IP            UP     UP
         Standby     STANDBY_IP           UP     UP
         Witness     WITNESS_IP           UP     N/A
     Allowed node host list:
         STANDBY_IP MASTER_IP WITNESS_IP 
     Membership coordinator: STANDBY_IP
     Standby priority host list:
         STANDBY_IP
     Promote Status:
         DB Type     Address              XLog Loc         Info
         --------------------------------------------------------------
         Master      MASTER_IP        0/160001A8
         Standby     STANDBY_IP       0/160001A8
         Standby database(s) in sync with master. It is safe to promote.
    

    You can understand with the output that I have Master,Standby and Witness nodes in my EFM cluster configuration.

  2. Install latest version EFM 3.1 binaries on Master,Standby & Witness nodes. EFM binaries comes in rpms, to download you have to enable the EDB repository and follow the instructions mentioned here

     yum install edb-efm31
    
  3. After installing new EFM 3.1 binaries on all the 3 nodes, invoke “–help” to see the context of “upgrade-conf”.

     # /usr/edb/efm-3.1/bin/efm --help
    
     <trimmed other options>
    
     upgrade-conf
         Will create a 3.1 compatible .properties and .nodes file based on existing files.
         Must be run with root privileges for default configuration.
         Full command: efm upgrade-conf <cluster name>
         To upgrade files from a non-sudo configuration, include the -source switch to
         specify the path to the files. The new files will be written to the directory from
         which the command is invoked, and will be owned by the user executing the command.
         Full command: efm upgrade-conf <cluster name> -source <directory>
    
  4. “upgrade-conf” utility should be executed on Master,Standby and Witness nodes. The utility will create a latest EFM 3.1 version configuration files(.properties & .nodes) from the pre-existing EFM 3.0 version. Below steps are executed on the Master Node, follow the same “upgrade-conf” steps on the Standby & Witness node.

    EFM 3.0 configuration files

     [root@master ]# ls -l /etc/edb/efm-3.0/
     total 40
     -rw-r--r--. 1 efm efm   180 Jun 18 19:00 efm.nodes
     -rw-r--r--. 1 efm efm 16087 Jun 17 08:07 efm.properties
     -rw-r--r--. 1 efm efm   139 Feb 22 02:20 efm.nodes.in
     -rw-r--r--. 1 efm efm 15771 Feb 22 02:20 efm.properties.in
    

    Run upgrade-conf command

     [root@witness bin]# /usr/edb/efm-3.1/bin/efm upgrade-conf efm
     Checking directory /etc/edb/efm-3.0
     Processing efm.properties file
     Removing log.dir value to use default.
    
     The following properties were added in addition to those in previous installed version:
         notification.level
         stop.failed.master
         stable.nodes.file
         virtualIp.interface
         virtualIp.single
         lock.dir
    
     Checking directory /etc/edb/efm-3.0
     Processing efm.nodes file
    
     Upgrade of files is finished. The owner and group for properties and nodes files have been set as 'efm'.
    

    Check for the new configuration files in /etc/edb/efm-3.1

     [root@master ]# ls -l /etc/edb/efm-3.1/
     total 48
     -rw-r--r--. 1 efm  efm    179 Jun 18 19:16 efm.nodes
     -rw-r--r--. 1 efm  efm  17321 Jun 18 19:16 efm.properties
     -rw-r--r--. 1 root root   139 May 17 23:55 efm.nodes.in
     -rw-r--r--. 1 root root 17066 May 17 23:55 efm.properties.in
    

    Note: During “upgrade-conf” step, old version EFM service should be running.

  5. Now, stop the Old EFM service on all the nodes

     systemctl stop efm-3.0.service
    
     or 
    
     Use "stop-cluster" option to stop all EFM services
    
     [root@master ]# /usr/edb/efm-3.0/bin/efm stop-cluster efm
     Stop cluster command sent to 3 nodes.
    
  6. Now start the EFM 3.1 service on all the nodes.

     [root@master efm-3.1]# systemctl start efm-3.1.service
    

    Note: If you are running EFM cluster with different cluster name then you need to edit/modify 3.1 version unit file with clustername and apply the unit file changes by running “systemctl daemon-reload”

Thats all, it took 5 steps to upgraded the existing EFM cluster to the new version EFM 3.1.