Redis backup - linux shell script

[Halvhjearne]

i just threw this together, cause i wanted to backup my database once in a while, however i do not want be held responsible for anything you do to your system.

 

pls remeber to backup anything you hold dearly before using this and NEVER use this with root permissions!

 

http://pastebin.com/iBiZkvUb

 

if setup correctly the script will backup the database to a folder named the day of the week, database name/folder will be something like:

/whatever/dayoftheweek/dump_2015_26-07_14.00-01.rdb

If you do not plan to use the same user, that runs redis-server to run the backup script, then you will need to make sure the dump.rdb is written with permissions to allow this.

 

by default redis-server will be starting with --umask 007 flag, to change it you need to change the redis-server init script (mine was located at /etc/init.d/redis-server).

 

find the line that starts like this:

if start-stop-daemon --start --quiet --umask 003

this is whats important:

--umask 007

we will need to change this to something like:

--umask 003

now it will write the file with read permissions for all users (-rw-rw-r--).

 

now make sure to restart redis-server:

service redis-server restart

change the path's on top of the script to determin where the database is located:

# set path to where your redis.rdb file is located
REDISDBDIR=/var/lib/redis

and where you want the files to end up:

# set path to where you want the db file to end up
DBBACKUPDIR=/media/9c2af2c6-2d7e-4383-9255-6fa2b6d58a28/SysDisk/armabackup

if you want it to delete or show you old database files you can use/uncomment the lines on the bottom:

##########################################################
# Change how long you want to save a database copy below #
##########################################################
 
#show old db's, uncomment to use:
                            #-mmin = +minutes
#find ${DBBACKUPDIR} -type f -mmin +240
                            #-mtime = +days
#find ${DBBACKUPDIR} -type f -mtime +3
################## IMPORTANT: ############################
# delete db's older than 7days, change the 7 below       #
# or comment out with a # infront of the line to disable #
##########################################################
#find $DBBACKUPDIR -type f -mtime +7 -delete

have fun

 

Regards

Halv

[slayer]

This is incorrect practice of doing backup of redis.

 

You need to call

redis-cli bgsave 

to force redis write database from memory to disk, then wait 10-15 minutes and copy rdb file to your backup directory. If you just copy rdb file from redis directory it may be corrupted or mistimed.

[Halvhjearne]

This is incorrect practice of doing backup of redis.

 

You need to call

redis-cli bgsave 

to force redis write database from memory to disk, then wait 10-15 minutes and copy rdb file to your backup directory. If you just copy rdb file from redis directory it may be corrupted or mistimed.

 

i doubt it would become corrupted, but you are right it might be a good idea to save the db before you copy it.

 

however, if enough keys has been changed recently (wich will usually happend right before a restart) and you setup redis to save correctly, then i dont see a problem with using just this.

[slayer]

When you copy file and redis write data simultaneously to file, your backup rdb file will become incompleted and corrupted.

 

This is default config from debian package of redis

save 900 1
save 300 10
save 60 10000

Redis will save db to disk:

every 900 seconds when 1 key have been changed or have been created

every 300 seconds when10 keys have been changed or have been created

every 60 seconds when 10000 keys have been changed or have been created

 

You can calculate probability of copying corrupted file.

[Halvhjearne]

When you copy file and redis write data simultaneously to file, your backup rdb file will become incompleted and corrupted.

 

This is default config from debian package of redis

save 900 1
save 300 10
save 60 10000

Redis will save db to disk:

every 900 seconds when 1 key have been changed or have been created

every 300 seconds when10 keys have been changed or have been created

every 60 seconds when 10000 keys have been changed or have been created

 

You can calculate probability of copying corrupted file.

 

i would love to see that calculation.

 

considdering that you will not actually change a file that is in use on a linux system, then i doubt it will happend very often that a copy of the database will be corrupt.

 

nevertheless i have now been using this for a couple of days and i wont mind checking at some point how many is corrupted (if any), whenever i get some time.

[slayer]

Sorry i am wrong about data corruption, redis devs changed algorithm of saving rdb file in some version and data corruption is not possible. Redis saves database not directly to /var/lib/redis. It creates new temp file and after database fully flushed to disk, it moves new file to /var/lib/redis/. I use redis from early versions in web development and the problem with data corruption existed in the past