Evidian SafeKit brings high availability to Firebird (an open source SQL relational database). This article explains how to implement quickly a Firebird cluster without shared disk and without specific skills. The high availabity module firebird.safe and a free trial are offered in the installation instructions section.
This clustering solution is recognized as the simplest to implement by our customers and partners. It is also a complete solution that solves hardware failures (20% of problems) including the complete failure of a computer room, software failures (40% of problems) including smooth upgrade server by server and human errors (40% of problems) thanks to its simplicity.
On the previous figure, the server 1/PRIM runs Firebird services (any edition). Users are connected to the virtual IP address of the mirror cluster. SafeKit replicates files opened by Firebird services in real time. Only changes in the files are replicated across the network, thus limiting traffic (byte-level file replication). Names of file directories containing Firebird services data are simply configured in SafeKit. There are no pre-requisites on disk organization for the two servers. Directories to replicate may be located in the system disk. SafeKit implements synchronous replication with no data loss on failure contrary to asynchronous replication.
In case of server 1 failure, there is an automatic failover on server 2 with restart of Firebird services. Then, when server 1 is restarted, SafeKit implements automatic failback with reintegration of data without stopping Firebird services on server 2. Finally, the system returns to synchronous replication between server 2 and server 1. The administrator can decide to swap the role of primary and secondary and return to a server 1 running Firebird services. The swap can also be done automatically by configuration.
On both Windows servers
On both Linux servers
The configuration is presented with the web console connected to 2 Windows servers but it is the same thing with 2 Linux servers.
Important: all the configuration is made from a single browser.
Launch the web console in a browser by connecting to http://localhost:9010 (next image)
Enter IP address of the first node and click on Confirm (next image)
Click on New node and enter IP address of the second node (next image)
Click on the red floppy disk to save the configuration (previous image)
In the Configuration tab, click on firebird.safe then enter firebird as the module name and Confirm: next images with firebird instead of xxx
Click on Validate (next image)
Change the path of replicated directories only if necessary (next image) and enter a virtual IP address. A virtual IP address is a new unused IP address in the same IP network as the IP addresses of the two nodes. The virtual IP address automatically switches in case of failover.
For information:
Click on Validate (previous image)
Click on Configure (previous image)
Check the success green message on both servers and click on Next (previous image). On Linux, you may have an error at this step if replicated directories are mount points. See this article to solve the problem.
Select the node with the most up-to-date replicated directories and click on start it to make the first resynchronization in the right direction (previous image). Before this operation, we suggest you to make a copy of replicated directories before starting the cluster to avoid any errors.
Start the second node (previous image) which becomes SECOND green (next image) after resynchronisation of all replicated directories (binary copy from node 1 to node 2).
The cluster is operational with Firebird services running on the PRIM node and nothing running on the SECOND node (previous image). Only modifications inside files are replicated in real-time in this state.
Be careful, components which are clients of the Firebird services must be configured with the virtual IP address. The configuration can be made with a DNS name (if a DNS name has been created and associated with the virtual IP address).
Check with Windows Microsoft Management Console (MMC) or with Linux command lines that the Firebird services are started on the primary server and stopped on the secondary server. Put Firebird services with Boot Startup Type = Manual (SafeKit controls start of Firebird services).
Stop the PRIM node by scrolling down the menu of the primary node and by clicking on Stop. Check that there is a failover on the SECOND node. And check the failover of Firebird services with Windows Microsoft Management Console (MMC) or with Linux command lines.
To understand what happens in the cluster, check the SafeKit logs of the primary server and the secondary server.
To see the module log of the primary server (next image):
To see the application log of the primary server (next image):
To see the logs of the secondary server (previous image), click on W12R2server75/SECOND (it will become blue) on the left side and repeat the same operations. In the secondary module log, you will find the volume and the reintegration time of replicated data.
In Advanced Configuration tab, you can edit internal files of the module: bin/start_prim and bin/stop_prim and conf/userconfig.xml (next image on the left side). If you make change in the internal files here, you must apply the new configuration by a right click on the blue icon/xxx on the left side (next image): the interface will allow you to redeploy the modified files on both servers.
Configure boot start (next image on the right side) configures the automatic boot of the module when the server boots. Do this configuration on both servers once the high availability solution is correctly running.
For getting support on the call desk of https://support.evidian.com, get 2 Snaphots (2 .zip files), one for each server and upload them in the call desk tool (next image).
userconfig.xml
<!DOCTYPE safe>
<safe>
<service mode="mirror" defaultprim="alone" maxloop="3" loop_interval="24" failover="on">
<!-- Heartbeat Configuration -->
<!-- Names or IP addresses on the default network are set during initialization in the console -->
<heart pulse="700" timeout="30000">
<heartbeat name="default" ident="flow">
</heartbeat>
</heart>
<!-- Virtual IP Configuration -->
<!-- Replace
* VIRTUAL_TO_BE_DEFINED by the name of your virtual server
-->
<vip>
<interface_list>
<interface check="on" arpreroute="on">
<real_interface>
<virtual_addr addr="VIRTUAL_TO_BE_DEFINED" where="one_side_alias" />
</real_interface>
</interface>
</interface_list>
</vip>
<!-- Software Error Detection Configuration -->
<errd polltimer="10">
<!-- Firebird Server -->
<proc name="firebird.exe" atleast="1" action="restart" class="prim" />
</errd>
<!-- File Replication Configuration -->
<!--
* C:\Program Files\Firebird\Firebird_3_0\examples\empbuild: directory of employee.fdb sample database
* replace the path with your database directory
-->
<rfs async="second" acl="off" nbrei="3">
<replicated dir="C:\Program Files\Firebird\Firebird_3_0\examples\empbuild" mode="read_only" />
</rfs>
<!-- User scripts activation -->
<user nicestoptimeout="300" forcestoptimeout="300" logging="userlog" />
</service>
</safe>
start_prim.cmd
@echo off
rem Script called on the primary server for starting application services
rem For logging into SafeKit log use:
rem "%SAFE%\safekit" printi | printe "message"
rem stdout goes into Application log
echo "Running start_prim %*"
set res=0
net start "Firebird Server - DefaultInstance" > nul
if not %errorlevel% == 0 (
%SAFE%\safekit printi "Firebird start failed"
) else (
%SAFE%\safekit printi "Firebird started"
)
if %res% == 0 goto end
:stop
set res=%errorlevel%
"%SAFE%\safekit" printe "start_prim failed"
rem uncomment to stop SafeKit when critical
rem "%SAFE%\safekit" stop -i "start_prim"
:end
stop_prim.cmd
@echo off
rem Script called on the primary server for stopping application services
rem ----------------------------------------------------------
rem
rem 2 stop modes:
rem
rem - graceful stop
rem call standard application stop with net stop
rem
rem - force stop (%1=force)
rem kill application's processes
rem
rem ----------------------------------------------------------
rem For logging into SafeKit log use:
rem "%SAFE%\safekit" printi | printe "message"
rem stdout goes into Application log
echo "Running stop_prim %*"
set res=0
rem default: no action on forcestop
if "%1" == "force" goto end
net stop "Firebird Server - DefaultInstance" > nul
%SAFE%\safekit printi "Firebird stopped"
rem wait a little for a real stop of services
%SAFEBIN%\sleep 10
:enduserconfig.xml
<!DOCTYPE safe>
<safe>
<service mode="mirror" defaultprim="alone" maxloop="3" loop_interval="24" failover="on">
<!-- Heartbeat Configuration -->
<!-- Names or IP addresses on the default network are set during initialization in the console -->
<heart pulse="700" timeout="30000">
<heartbeat name="default" ident="flow">
</heartbeat>
</heart>
<!-- Virtual IP Configuration -->
<!-- Replace
* VIRTUAL_TO_BE_DEFINED by the name of your virtual server
-->
<vip>
<interface_list>
<interface check="on" arpreroute="on">
<real_interface>
<virtual_addr addr="VIRTUAL_TO_BE_DEFINED" where="one_side_alias"/>
</real_interface>
</interface>
</interface_list>
</vip>
<!-- Software Error Detection Configuration -->
<errd polltimer="10">
<!-- Firebird Server -->
<proc name="firebird" atleast="1" action="restart" class="prim" />
</errd>
<!-- File Replication Configuration -->
<!--
* /opt/firebird/examples/empbuild: directory of employee.fdb sample database
* replace the path with your database directory
-->
<rfs mountover="off" async="second" acl="off" nbrei="3">
<replicated dir="/opt/firebird/examples/empbuild" mode="read_only" />
</rfs>
<!-- User scripts activation -->
<user nicestoptimeout="300" forcestoptimeout="300" logging="userlog" />
</service>
</safe>
start_prim
#!/bin/sh
# Script called on the primary server for starting applications
# For logging into SafeKit log use:
# $SAFE/safekit printi | printe "message"
#---------- Clean Firebird residual processes
# Call this function before starting any Firebird databases
# to clean eventual resual Firebird processes
clean_Firebird()
{
retval=0
$SAFE/safekit printw "Cleaning Firebird processes"
# kill started Firebird processes
ps -e -o pid,comm | grep firebird | $AWK '{print "kill " $1}'| sh >/dev/null 2>&1
return $retval
}
#---------- Firebird Databases
# Call this function for starting Firebird Server
start_Firebird()
{
retval=0
$SAFE/safekit printw "Starting Firebird Server"
# Firebird - Database Starting
service firebird start
if [ $? -ne 0 ] ; then
$SAFE/safekit printw "Firebird server start failed"
else
$SAFE/safekit printw "Firebird server started"
fi
return $retval
}
# stdout goes into Application log
echo "Running start_prim $*"
res=0
[ -z "$OSNAME" ] && OSNAME=`uname -s`
OSNAME=`uname -s`
case "$OSNAME" in
Linux)
AWK=/bin/awk
;;
*)
AWK=/usr/bin/awk
;;
esac
# TODO
# remove Firebird boot start
# Clean Firebird residual processes
clean_Firebird || res=$?
# Start Firebird databases
start_Firebird || res=$?
if [ $res -ne 0 ] ; then
$SAFE/safekit printi "start_prim failed"
# uncomment to stop SafeKit when critical
# $SAFE/safekit stop -i "start_prim"
fi
exit 0
stop_prim
#!/bin/sh
# Script called on the primary server for stopping application services
# For logging into SafeKit log use:
# $SAFE/safekit printi | printe "message"
#----------------------------------------------------------
#
# 2 stop modes:
#
# - graceful stop
# call standard application stop
#
# - force stop ($1=force)
# kill application's processes
#
#----------------------------------------------------------
#---------- Clean Firebird residual processes
# Call this function on force stop
# to clean eventual resual Firebird processes
clean_Firebird()
{
retval=0
$SAFE/safekit printw "Cleaning Firebird processes "
# kill started Firebird
ps -e -o pid,comm | grep firebird | $AWK '{print "kill -9 " $1}'| sh >/dev/null 2>&1
return $retval
}
#---------- Firebird databases
# Call this function for stopping Firebird databases
stop_Firebird()
{
retval=0
if [ "$1" = "force" ] ; then
# Firebird databases force stop
clean_Firebird
return $retval
fi
# Firebird databases graceful stop
$SAFE/safekit printw "Stopping Firebird server"
service firebird stop
if [ $? -ne 0 ] ; then
$SAFE/safekit printw "Firebird server stop failed"
else
$SAFE/safekit printw "Firebird server stopped"
fi
return $retval
}
# stdout goes into Application log
echo "Running stop_prim $*"
res=0
[ -z "$OSNAME" ] && OSNAME=`uname -s`
case "$OSNAME" in
Linux)
AWK=/bin/awk
;;
*)
AWK=/usr/bin/awk
;;
esac
mode=
if [ "$1" = "force" ] ; then
mode=force
shift
fi
# Stop Firebird server
stop_Firebird $mode || res=$?
[ $res -ne 0 ] && $SAFE/safekit printi "stop_prim failed"
exit 0
