Automated Backup Script (OpenSource)

Hi Guys,

We've put together an unofficial script that should assist many users in getting their backup processes more or less automated. this script s been written in a way that users can take the parts that they need (sections marked off in the code), and incorporate them into their backup processes, or they can use the whole script as it is.

Please note that this script is not to replace your usual backup process.

When copying the script, please save it in a text file named CommitCRMAutoBackup.cmd, located on a different drive than the CommitCRM system (in any folder you wish). This is so that the backups don't get created on the drive that hosts CommitCRM, and the separation between the system & your backups is preserved.

Once you have the script saved, please download zip.exe (obtainable from google), and save the file in the same folder as the script.

The last step is to run the backup script, while adding the path to your commit folder as a command line parameter (No Backslash at the end of the path).

I.E. D:\>CommitCRMAutoBackup.cmd C:\Commit

Here's the script:

@echo off
REM ================================================== =====================
REM
REM
REM Disclaimer:
REM This script has been created as a functional sample, in order to help to automate
REM your backup process. The script has been divided into sections so that you can easily
REM re-engineer the script to better automate your current backup method. This script
REM is not meant to replace your current backup method in any way, Commit Business
REM Solutions Ltd or any of its employees do not take any responsibility for the data
REM stored in any backup and we highly suggest to keep all backup archives as separate
REM as possible from the main CommitCRM data, as well as implementing additional
REM safety & backup measures so that your data will not be in any risk of being lost.
REM
REM What this script does:
REM The script performs the following steps:
REM 1) Gather CommitCRM folder from Command-Line Parameter
REM 2) Create Database "Stop" file, for a safe backup (Disconnects all users from the system)
REM 3) Pause for 5 minutes. (Should be enough for all connections to be termiated)
REM 4) Compress CommitCRM data into a Zip archive (Archive is created in the same folder as the script
REM is run from)
REM 5) Delete Database Stop file
REM
REM Important Note: This backup script does not backup any of your Documents files, including ones
REM “attached” to CommitCRM;
REM please remember to backup your documents files in a separate archive.
REM
REM Once the Database stop file has been deleted, the database is reactivated, and your users will
REM be able to connect to the system again.
REM
REM
REM Parameters:
REM %1: Local path to Commit CRM. When running the Backup script, the script needs you to input
REM the Commit Folder Path in the command line right after the script name. Path must not end with
REM a BackSlash \
REM
REM
REM General Usage Example:
REM :\>CommitCRMAutoBackup.cmd C:\Commit
REM
REM
REM Dependencies:
REM This script needs to be saved in a separate drive from the CommitCRM System files.
REM Please copy Zip.exe in the same folder with this script.
REM Zip.exe (Free Compression Tool) is easily obtainable from many places on the internet. We
REM suggest finding it in Google.
REM
REM ================================================== ================

if ###%1###==###### goto NoParam
goto ParamsFound

:NoParam
echo.
echo Missing path. Enter the local path to the Commit CRM root directory.
echo Example: CommitCRMAutoBackup.cmd C:\Commit
echo ================================================== ================
Echo Zip.exe (Free Compression Tool) is easily obtainable from many places
echo on the internet. We suggest finding it in Google.
goto End

:ParamsFound
REM = Close Datamase Connections======================================
if exist "%1\DbSys\CommitStop.ini" del "%1\DbSys\CommitStop.ini"
echo [TERMINATE] > "%1\DbSys\CommitStop.ini"
echo TimeBeforeAutoTerminate=1 >> "%1\DbSys\CommitStop.ini"
echo Stopping database connections...
echo Waiting 5 Minutes...
REM=============================================== ====================

REM = This section pauses the script for 5 minutes.===================
echo Time remaining until Backup - 5:00
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 4:30
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 4:00
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 3:30
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 3:00
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 2:30
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 2:00
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 1:30
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 1:00
Ping -n 30 127.0.0.1 >NUL
echo Time remaining until Backup - 0:30
Ping -n 30 127.0.0.1 >NUL
echo Backing up your data momentarily.
Ping -n 5 127.0.0.1 >NUL
REM=============================================== ====================

REM = Backup Command=========================================== =======
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set mydate=%%c-%%a-%%b)
For /f "tokens=1-2 delims=/:" %%a in ('time /t') do (set mytime=%%a%%b)
zip -u CommitCRM_Backup_%mydate%_%mytime% %1\db\*.* %1\dbsys\*.* %1\filters\*.* -x commitstop
REM=============================================== ====================

REM = Reactivate Database========================================== ===
if exist "%1\DbSys\CommitStop.ini" del "%1\DbSys\CommitStop.ini"
echo Database reactivated...
REM=============================================== ====================

:End

Do all Commit users have to be out of the system to run this script? In other words, if I schedule it to run at midnight, and everyone leaves Commit open, will that be a problem?

Sweet, thanks for the script!

Hi,

Thanks for posting this question.
The script shuts the database down by itself, so there's no need to close every CommitCRM client, or the Commit Services. The backup script should run without any problems, while the clients are still on.

However, we do suggest that the best practice is to enforce a procedure that involves shutting down any CommitCRM clients used at the end of the day; just to be safe.

Let me know if you have anymore questions on this topic.

Reno Breen

Are image based backups like Acronis not sufficient for backing up Commit live using VSS?

Hi,

This is an excellent question.
In order to backup CommitCRM, the CommitCRM database must be shut down, in order to make sure that no tables are being edited at the time of the backup.

Specifically regarding Acronis, there are 2 types of this software that can be run.

1) If you're using the installed version that can back the system up while the operating system that houses CommitCRM is still running, then you will need to create your own script that automatically closes the database to network communications, and then reinstates the system; you can take a few sections from the script above and customize them to your businesses needs.

2) If the Acronis software you are using is the type that you boot the server with, then the CommitCRM database has already been shut down during the windows shutdown sequence, and you can perform the Acronis backup freely by backing the CommitCRM folder up.

Let us know if you have any questions on this topic.

Reno Breen

Can I just say, as welcome as this script is its pretty clunky to have to do this to run a backup. Surely you can incorporate into Commit the option for a user with sufficient privaleges to just run a backup with one click? Have a backup section where you list the desired location of the backup so you dont need to do it every time, the task also kicks everyone out and runs the backup at a scheduled time or at your press of a button. Its pretty surprising that this still isn't in Commit. I cant tell you how painful and clunky it is to do a manual backup at the moment, when it should be able to be launched with a single click and some background settings. Be great to see this on the roadmap.

Hi Natrat,

Thanks for the comments; i'll pass them on.

Regards,
Sherry

Does the SQL version have the same limitations in regards to not being able to be backed up live using software like Acronis or Storagecraft?

Hi Ascendnet,

Thanks for sending this in.
Yes; the Commit SQL Database also requires shutting down the database prior to the backup, so we suggest using this software to only backup the archive created by CommitCRM or this script.

Alternatively, you can create your own scripts that shut down and restart the database, and use them to automatically shut the database down prior to an Acronis Image backup. Once the backup is complete, run the second script to enable the database.

Hope this helps.
Regards,
Sherry

Please add my vote to make Commit "VSS aware"

+1 ^ to the VSS aware

again on backup -

Implies that in order to successfully backup the entire Commit server all you need to do is shutdown the services and copy the commit folders. Thats pretty much all the above script does yes? If that the case you can just use something like the free Cobian Backup which allows you to shutdown services and restart them again on completion. Fire it off in the middle of the night and you have a working backup. Or am I missing something?

natrat, once nothing accesses the CommitCRM database you should be safe to backup the files in any way you want.
The nice thing with the script is that it instructs everyone to stop connecting to the database.


Make sure that -
All CommitCRM clients are stopped
CommitWebInterface service is stopped
CommitServer service is stopped
ANY third party system that uses Commit's APIs to integrate is stopped

Back the files.

Done!

Hope this helps.
Dina

I thought I'd create a small backup tool to perform CommitCRM backups a little easier.

Features include:
- backup the program directory, data directory or docs directory
- upload backups to an ftp server
- sends an email on success/failure
- can be scheduled to run automatically

download the program from http://www.smartitsupport.com/softwa...tcrm/setup.exe and let me know of any problems

very cool, but when i tried to install, i get the following error.

http://i.imgur.com/A2d4d.png

thanks for reporting the error. I think I have identified the problem and have made a change to the program, this should automatically update the next time you start the program.

update fixed issue, works great on windows server 2008 r2, and windows 7

Webpage for Backup for CommitCRM:
http://www.smartitsupport.com/misc-p...-for-commitcrm

I don't think that SmartIT's backup software is getting enough attention. We have been using it for about a month now, and recently setup the scheduled task with amazing results. This is ESSENTIAL in any business - with commit not having an automated backup system built-in, these guys have released their internally developed software, no doubt they developed for internal use, but to release it for FREE to other businesses, that is just a very classy and MUCH APPRECIATED move guys. I want to give a virtual hand-shake to all of you guys at Smart I.T. Support in Earls House, Earlsway all the way from the other side of the globe in Missouri, USA.

Commit, in my opinion, this software is something I would put in the wiki, with its own area for automated backup, this is something we use every day with very successful results. Again, thanks guys, and thanks Commit for everything else, including allowing the SmartIT guys and others to develop around Commit.

http://www.smartitsupport.com/misc-p...-for-commitcrm

Thanks for the feedback and I'm glad you find it useful.

I agree, I think it would be a great idea if Commit gave this program more attention and mentioned it in the wiki or recommended it to other users. We have put a lot of work into the program and have offered it free to other users - we'd like other users to be made aware of it so they can benefit from it too.

Commit if anyone's reading this, how about a mention in the wiki, newsletter, link on the main website, or any other suggestions?

SmartIT,

I'm not really seeing any specifics on what your program does. I see that it backs up the specified Commit folders but is it also shutting down the Commit services on the server and if so is it also starting them back up again when the backup completes?

Yes the program shuts down the commitCRM before backing up, when run both manually and as scheduled, as advised by Commit. Once the backup has finished CommitCRM services are started again.

The program can also notify you by email on error or successful backup.

Backups can also be uploaded to a remote server via FTP and, as mentioned, scheduled - just set it and forget it and if there's a problem it will email you.

You can see more info at http://www.smartitsoftware.co.uk/pro...-for-commitcrm

Thanks for your feedback and comments. We've also heard that this is a very nice and useful program. We are thinking about opening a third-party section on our main Web site that will list different programs that integrate with or enhance CommitCRM.

Thanks,
Dina

We currently use it to backup all out data, (we run SQL) as a scheduled task, and works prefect. The program goes through and disables all commit services, runs through the backup script, and then notifies myself and 1 other of success or failure, so far everything has been successful.

...but since Commit (as of 5.6) can backup their database without stopping the services, it seems like this utility and the original open source script listed on this thread is a little out dated, right? I have Commits backup utility running with the windows task scheduler, and it is working great. ...but maybe that's just for SQL versions?

I'm not sure about the sql version of CommitCRM as we do not use that, but it was built and tested against CommitCRM 5.6 so I wouldn't say it was outdated.

Obviously you can choose to use whatever backup solution you wish, this is just another option with a few additional features and easy to use gui we thought we'd make available to other commit users. If you'd rather use the other solution that's your choice.

Oh, don't take it personally, just asking... Just looked @ the wiki, and it looks like the Live Backup is only for SQL... anyway, looks like a nice utility for anyone that isn't on the SQL version!

Just wanted to say thanks to SmartIT for sharing this with the community. Very useful and user friendly tool ;)

I need to fully understand the backup process... we do have the commitsql version so I have set up a scheduled task to backup the database 10 minutes before out server backup systems kick off. My question is that the "CommitBackup.exe CommitCRMLiveSQLBackup" command only backs up the database... when we run the full server backup it then grabs this zip file along with all the other files (documents, emails, etc.). The problem seems to be that in a restore situation, the backup could be corrupt becuase the "CommitBackup.exe CommitCRMLiveSQLBackup" zip file may not account for activity (files/emails etc.) that happen inbetween... basically an inconsistant state.

thoughts?

//ray

oh, and 8+ on the VSS integration... having commit fully VSS integrated would solve ALL of these backup problems!

thanks --

//ray

You mean like no option to "replay log files" or something into the database?

If the current Commit SQL version is not VSS aware how about making it a true MS SQL database so that it would be?

Thanks Ascendnet. Whether VSS will or will not be supported is something that will be determined by Sybase, not us directly. If you're using SQL use the new Online Backup to freely backup your system while the system is in use. We do not have any plans to use other database engines as we believe that the one we use is an absolutely great product, reliable, stable, easy to manage, install and configure. If this will change we will probably revisit this.

Thanks.

Didn't really get an answer to the inconsistant state issue... one fix would be to add a switch to the backup exe... something like "CommitBackup.exe CommitCRMLiveSQLBackup FullBackup" which would backup the database AND all the other folders (documents, emails, etc.) -- this would then give us a big zip file that contained everything needed to restore from scratch in a consistant state.

//ray

Hi Ray, document files can be stored anywhere - on your server and sometimes even on local PCs (still being link-to from CommitCRM). In any case, the way we see it is that unlike database backups, standard server files backups should be a part of your standard backup procedure and not a part of the CommitCRM backup. We do not have any advantage in offering standard files backup (the opposite may be true). Having said that, you may want to look at the backup utility offered by SmartIT, as far as we understand it does backup the files and offers some additional nice features. Maybe you can use this program in addition to using our live SQL database backup option.

Thanks,
Dina

we have super robust backup systems in place which grab images and file level backups. The issue is that when the commit database backup runs, it's not grabbing everything (files, etc.) so that the backup is never really in a consistant state. For instance, if we kick off the backups at 5:50PM and they take 5 minutes to complet then our systems backup run at 6:00PM, there is still 5 minutes where files can be added to the system (e.g. automated tickets, eml files, etc.) OR delete OR modified which means that the database backup does not account for things being processed in that time... the only way to really get this to be 100% is one of two options:

1) Commit's backup system takes a backup of everything including the documents, logs, whatever all at the same time

2) The database engine becomes VSS aware... yes, sybase may be a good system but it's not VSS aware... why not use Microsoft's database and be done with it (we all know how to work with with the database and can make backups happen).

Until one of those two things happens, there is really no way to confirm that the backups of commit are in a consistant state.

//ray

Files are not considered, by us, as part of the CommitCRM backup. Just like backing up any other file you should backup your files in the standard way using the "super robust system" you have. I'm sure that the files will be backed up this way more safely than any of our attempt to write such a backup program. We will never beat your files backup program...

We do have advantages when it comes to database backup, especially when you're using SQL. We now support a live backup which is a very powerful feature.
Restoring CommitCRM from a database backup is very easy even if the entire CommitCRM folder is gone (install, restore data, done!).

Files restore should be done, in such cases, from your files back system. And if you work with VSS then you already have a backup to all files.

We currently do not have any plans to change this and backup external files ourselves simply because we will never do a better job than your dedicated backup program and we prefer to spent our resources on other items...

Thanks.

we use the commit sybase module and the live backup but because sybase is not vss aware, we are not getting clean in-state backups... the suggestion would be to have commit change the database back end to something that is vss compliant which would mean all image based backup systems would work 100%.

//ray

The procedure entry point GetNativeSystemInfo could not be located in the dynamic link library KERNEL32.DLL

What? No Windows 2000 support? Sheesh!

We have a number of in-house utility VM's running Windows 2000 (low RAM and DISK requirements), and one of these holds our Commit installation. Just thought I would try this solution instead of pausing and backing up the entire (very small) VM.

I think my new workstation has got more RAM than this VM has disk space. :)

Yes, many times new PCs today have better on-paper specs than the server...
Do you refer to the CommitCRM backup utility or to the third party's one offered by SmartIT?

This was the SmartIT backup program I was trying.

CommitCRM continues to work flawlessly on the Windows 2000 Workstation VM that is dedicated to it, including the Web and Email modules. Day to day operations including backups are naturally done on more modern machines.