This post is to assist in the setup of a BackupPC system able to backup in use files by using MS volume snapshots. It also has the benefit of only having the RSync daemon running during the backup which increases security.

This method is based on some other posts I have seen using VShadow, Backing Up Open Files on Windows with Rsync, and some suggestions on DiskShadow but goes further in using RSync as a system service giving full access to files, and removing the need to use winexe.

General Outline

1. Create a new user backuppc (try to match case of the user on Linux). Very limited rights
2. Install Cygwin with RSync, OpenSSH and configure them
3. Create a scheduled task to run as SYSTEM when triggered by a certain event
4. Set BackupPC server for passwordless login to host and modify Pre/Post Dump Cmds

Overview

In Windows 7 and Server 2008 R2 elevation is required to create snapshots. As a remote SSH connection cannot bypass UAC a way is needed to create the snapshot, bypassing the elevation prompt. I also wished to run the Rsync Daemon as SYSTEM user so it has rights to view all files.

To do this I moved all the elevated tasks into a Task Scheduler item that is set to run as SYSTEM, and is triggered by an Event log event.

1. BackupPC performs a password-less key SSH logon to client
2. It runs a script which creates an event log entry and then waits for RSync to start before returning to BackupPC and starting the backup.
3. Task scheduler is triggered by Event Log and starts DiskShadow as SYSTEM.
4. Disk shadow creates a shadow of any chosen volumes, mounts them and then starts RSync.
   It then waits for a file to be created by BackupPC at the end of the backup telling it to stop RSync and delete the snaphots.

Flow and triggers

Host PC

Create a new user, backuppc, you can limit this account further in Security policy after everything is configured.

Create a BackupPC folder and add the following scripts to it. These are also available in a zip file. Note: There is a file embedded within this post, please visit this post to download the file. You will need to modify paths depending on where you create the folder. I used C:\cygwin\BackupPC but a better location may be C:\cygwin\usr\share\BackupPC

Most of these scripts are just modified versions of the ones written for VShadow, changed to work with DiskShadow and Task Scheduler.

pre-cmd.vbs

' This file starts the commands
' It will start the snapshot process and quite once RSync is running

Const Rsync = "C:\cygwin\var\run\rsyncd.pid"
Const Flag = "C:\cygwin\var\run\wake.up"
Set fso = CreateObject("Scripting.FileSystemObject")
'
' Pid file shouldn't be there already
' Check /stop service , still there delete
'
If DoesFileExist(Rsync)=0 Then
	fso.DeleteFile(Rsync)
End If
'
' Nor should "wake.up"
'
If DoesFileExist(Flag)=0 Then
   fso.DeleteFile(Flag)
End If

Set objShell = CreateObject("WScript.Shell")
' objShell.Exec "C:\BackupPC\backuppc.cmd > " & Log

' This writes event log entry that triggers task scheduler to start system process
' that takes snapshot and starts RSync
objShell.Exec "Logevent.exe -r ""BackupPC"" -e 10 -s S ""Backup Start"" "
Wscript.Echo "Sent BackupPC Event Log Trigger"

'
' Just sleep until the file "rsyncd.pid" appears
'

While DoesFileExist(Rsync)
   wscript.sleep 10000
Wend

' functions

function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing

end function

backuppc.cmd

c:
cd C:\cygwin\BackupPC
diskshadow /s DiskShadowScript.txt /l C:\cygwin\var\log\diskshadow.log
del C:\cygwin\var\tmp\*.cab /q
c:cd C:\cygwin\BackupPCdiskshadow /s DiskShadowScript.txt /l C:\cygwin\var\log\diskshadow.log
del C:\cygwin\var\tmp\*.cab /q

DiskShadowScript.txt

#DiskShadow script file

#Make shadows persistent, No writers as data volume

# If backing up C: and any app files (ntds, database, etc) use writers

#SET CONTEXT PERSISTENT NOWRITERS

SET CONTEXT PERSISTENT

#Cab location for process

SET METADATA C:\cygwin\var\tmp\backup.cab

SET VERBOSE ON

BEGIN BACKUP

#Alias volume with alias

ADD VOLUME C: ALIAS SystemData

ADD VOLUME F: ALIAS UserData

#Create Snapshot

CREATE

#Expose the volume and run command file then unexpose

EXPOSE %UserData% B:

EXPOSE %SystemData% T:

EXEC C:\cygwin\BackupPC\Serverbackup.cmd

UNEXPOSE B:

UNEXPOSE T:

END BACKUP

#Delete the shadow copy

DELETE SHADOWS SET %VSS_SHADOW_SET%

#End of script

Serverbackup.cmd

REM Start RSync now that Snapshots are created

net start rsyncd

REM Need to wait until backup completed

cscript "C:\cygwin\BackupPC\sleep.vbs"

Logevent.exe -r "BackupPC" -e 20 -s S "Backup Completed"

sleep.vbs

Const Rsync = "C:\cygwin\var\run\rsyncd.pid"
Const Flag = "C:\cygwin\var\run\wake.up"
Set fso = CreateObject("Scripting.FileSystemObject")

' Just sleep until the file "rsyncd.pid" appears
While DoesFileExist(Rsync)
   wscript.sleep 10000
Wend

' Now sleep until the file "wake.up" appears
While DoesFileExist(Flag)
   wscript.sleep 10000
Wend

fso.DeleteFile(Flag)

' It's time to kill Rsync
'Stop Service
strServiceName = "rsyncd"
Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")
Set colListOfServices = objWMIService.ExecQuery("Select * from Win32_Service Where Name ='" & strServiceName & "'")
For Each objService in colListOfServices
    objService.StopService()
	Wscript.Echo "RSyncD Stopped"
Next

' Wait for Rsync to let go
wscript.sleep 5000

' Delete PID file
If DoesFileExist(Rsync)=0 Then
   fso.DeleteFile(Rsync)
End If

' functions
function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing

end function

Cygwin

Install Cygwin and choose to install packages RSync and OpenSSH

Start an elevated Cygwin (Run as an Administrator )

Config

Cygwin 1.7 changes the way to ignore NT Security from the nontsec environment to modifying fstab file.

In your prefered editor modify /etc/fstab and uncomment the bottom line and add noacl as below, this tells it to ignore security :

none /cygdrive cygdrive binary,noacl,posix=0,user 0 0

Run following to update group and user lists (if on a domain only add the users you want)

[bashlight=1]mkpasswd –l >/etc/passwd
mkgroup –l /etc/group[/bash]

OpenSSH

Configure Open SSH using ssh-host-config, create both the accounts it suggests for privilege isolation and running the service (cyg_service & sshd).
This should also automatically add the Firewall Exceptions (SSHD).

Setting up Keyless

Login as backuppc user.
Use

runas /user:backuppc cmd

Or

runas /user:domain\backuppc cmd

Run c:\cygwin\Cygwin.bat to start Cygwin as BackupPC user

You need to add the id_rsa.pub file from the BackupPC user on BackupPC server to C:\cygwin\home\backuppc\.ssh\authorized_keys If you have not already created one follow the instructions below in BackupPC – SSH section.

RSync

To install RSync as a system service use:

C:\cygwin\bin\cygrunsrv.exe -I rsyncd -d "RSync Daemon" -O --type manual -p /bin/rsync.exe -f "Used by BackupPC to remotely access files for backup" -a " --config=/etc/rsyncd.conf --daemon --no-detach"

Modify the /etc/rsyncd.conf file to reflect what drives you want to backup. It is better to edit this file later after you have modified the DiskShadowScript.txt and changed which drive letter it exposes shadows as.

Add a RSync exception to the firewall.

Open Windows Firewall with Advanced Security and choose Inbound Rules, New Rule, Program,  Next

Browse to C:\cygwin\bin\rsync.exe and then choose your options and name the rule

When finished open the rule, click Scope tab and add a Remote IP, that of the BackupPC server. This restricts RSync to only be accessible from BackupPC. You may also wish to similarly modify SSHD to only allow SSH access from BackupPC server.

DiskShadow

If you are backing up a windows Server 2008 or 2008 R2 host you already have DiskShadow installed. However, if you are running Windows 7 or Vista you will need to grab a copy from an equivalent server (x86 or x64). It is located in the System32 directory and you will also need the language file from the en-US folder.

I have zipped the files needed for x86 and x64 if you do not have immediate access to a Server.Note: There is a file embedded within this post, please visit this post to download the file. (The x86 files are from Server 2008 and x64 are from 2008 R2)

Copy the files from your architecture to the system32 directory and en-US subdirectory.

LogEvent

To generate the custom event log entry a tool from the Windows 2000 Resource kit is used, LogEvent, (Download). This needs to be either placed in the Path (e.g. Windows dir) or scripts need to directly call it.

Task Scheduler

The easiest way to configure this is to manually run LogEvent once to generate an event in the log.

Logevent.exe -r "BackupPC" -e 10 -s S "Backup Start"

Then open Event Viewer, Select the new BackupPC event and choose Attach Task to this Event, and in the wizard click next until it asks for the program, then give it C:\cygwin\BackupPC\backuppc.cmd

On Final page choose to Open Properties when you click Finish

Click the Change User or Group button and type in System as the user  and click OK. Also tick the Run with highest privileges box

BackupPC Server

SSH

This section is only needed if you do not already have keys generated for the backuppc user.

Login as backuppc user, either with password or simply “su – backuppc” from root

Generate SSH Keys, ssh-keygen –t rsa, do not set a password. Copy id_rsa.pub into C:\cygwin\home\backuppc\.ssh \authorized_keys

Test by running ssh -v backuppc@host to test the connection

BackupPC Host File

In the web interface change the following for the host.

DumpPreUserCmd:  $sshPath -q -x -l backuppc $host cscript "C:\cygwin\BackupPC\pre-cmd.vbs"
 DumpPostUserCmd: $sshPath -q -x -l backuppc $host echo "Complete: $xferOK" > /var/run/wake.up

Test Run

If you have existing backups from a previous RSync config, it is a good idea to run a full backup to ensure any in use files are part of the base. In my case I also changed the noacl flag which affected the file attributes RSync sees.

Tips

Set your antivirus to exclude the exposed drives. As they are read only it just slows down the reading of files by RSync.

Downloads

Updates

• While writing this up I found a good alternative way of using VShadow with SSH to backup in use files, that uses the AT command instead of task scheduler to get around UAC, BackupPC with ssh/rsync/VSS on Windows Server. The only down side to this method is the SSH connection requires an Admin account, but if configured securely this should be fine. It also a lot simpler to configure as it does not require configuring Task Scheduler.

Recently I discovered our Moodle gradebook was playing up and not correctly recording marks from assignments as well as been unable to assign categories from quizes.

I tracked the problem down to the Moodle database losing all its default entries for its tables. The cause is unknown but is quite likely the UTF conversion utility I ran previously to make the system compatible with Asian languages for LOTE.

To fix this problem I needed to recreate the Moodle DB structure and then reimport the data.

Backup

First make a backup. I made a mistake the first time I tried repairing this and the backup I made saved me.

mysqldump -u backupacc -p -Q --opt moodle > moodleBackup.sql

I created a duplicate site with a clean DB (copy the moodle folder and create a new DB) and went through the install. This needs to have all your modules and blocks to ensure all tables are created.

I then dumped the structure from this new DB and the old DB and compared them using Meld (or any other diffing program will work).

 mysqldump -u backupacc -p --no-data --skip-add-drop-table moodletest2 > moodleGoodStruc.sql
mysqldump -u backupacc -p -d --skip-add-drop-table moodle > moodleOldStruc.sql

Using Meld determine if there are any extra tables or fields in your current system that don’t exist in the clean one.

Then you need to determine if it is safe to delete those tables / field in your main DB or if they need to be added to the clean system. e.g. grade_items_history.decimals and grade_items_history.display were in the main DB but not the new one. After searching for it discovered they can be removed (MDL-15985).

After getting them to match you need to dump your Moodle data from your main site.

You can speed up the next steps by reducing the size of backup_log if you wish. Just do a

Delete
FROM `mdl_backup_log`
WHERE `time` <1242777600

Where the time code is calculated as a unix time code from about a month before todays date. This can reduce alot of space of your DB, and improve import/export times.

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

You need the –complete-insert option to ensure each insert is labelled with field name, in case field order is different. I didn’t do this the first time and needed to restore the backup.

Testing Merge

Now you need to test it. Create a new empty DB and import the Structure then import your data.

mysql -u root -p moodleTest3 < moodleStruc.sql
mysql -u root -p moodleTest3 < moodleData200906171559.sql

Check that no errors occurred during import, this is when I discovered most of my problems with extra non existent field.

Prep for Applying

Put your site in Maintenance mode so no users can login while you are testing this. (i.e. when you restore data site will already be in maintenance mode)

When everything is OK do another backup, as after this you are going to delete it all. (I recommend taking your site completely offline for this, not just maintenance mode. Edit your config.php and temporarily change either the username, password or db name or your database settings.)

Now re-export the Data again (assuming your site has been running while you have been doing tests)

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

Applying to Site

WARNING: HAVE A GOOD RELIABLE BACKUP BEFORE THIS STEP.  THIS WILL DESTROY YOUR MOODLE SITE.

Now use phpMyAdmin to drop every table in the Moodle main DB. Just use Check All at the bottom and select Drop.

Now you need to redo your test restore above but to your main DB.

Once complete reconnect your config.php file and check everything on your site looks the same. Open some courses, check some gradebooks, have a look around.

When its OK take out of Maintenance mode and you should be good to go.