Papercut can run on a Windows Server (2003, 2008 or 2008 R2) using windows print queues or a Novell Linux OES server with iPrint (CUPS & SAMBA), it is written in Java with some wrappers for the OS in use.

We originally ran it on a Server 2003 R2 (x86) but have recently updated to a Windows 2008 R2 (x64)print server. The main reason for the upgrade was to provide up to date drivers for Windows 7 32 & 64bit. We also run the Papercut MF version which costs A LOT more, but allows you to use it with external devices, such as photocopiers.

Papercut has some really useful features, Card recharge allows you to generate top up cards and print them out yourself. Then you just cut them up and given them to the office (or canteen) to sell to the students. Makes recharging much easier and no need for office staff to login to an admin console to top up accounts.

We recently had a student laptop roll-out (NSW DER), these laptops are not on our domain so needed an alternative way to print. Papercut has a feature called WebPrint which allows students to upload MS office or PDF files to a website and have them printed to the chosen printer. You need to run software for it on a windows pc (server or client) which automatically opens the document and prints it to the chosen printer. As our Print server is running Windows and is a virtual machine we just configured it to auto-login as a limited user we created and installed Office, and Acrobat.

We use the reports function for auditing staff accounts and charging printer usage back to faculties. Staff are automatically added to different charge accounts based on their AD group membership.

We have just started using Advanced Scripting to charge staff different prices than students on shared copiers and printers.
The following script is what we use, as it is based on how faculties are invoiced at the end of each term. It is based on the Papercut recipe “Discount for staff” with a few changes.

/*
* Change Pricing for staff
*
* Staff are charged based on cost to school.
*
*/
function printJobHook(inputs, actions) {

var DISCOUNT_GROUP   = "Staff-All";
var COST_COLOUR  = 0.07; // Click Rate of Colour
var COST_BW  = 0.02;  // Click Rate of B/W
var COST_A4  = 0.01;  // Price per page of A4
var COST_A3  = 0.03;  // Price per page of A3

/*
* This print hook will need access to all job details
* so return if full job analysis is not yet complete.
* The only job details that are available before analysis
* are metadata such as username, printer name, and date.
*
* See reference documentation for full explanation.
*/

if (!inputs.job.isAnalysisComplete) {
	// No job details yet so return.
	return;
}

if (inputs.user.isInGroup(DISCOUNT_GROUP)) {
	// Debug messages are written to [install-path]/server/logs/server.log
	actions.log.debug("Cost before discount: " + inputs.job.cost);

	// Initial 0
	var newCost = 0;

	// Includes Duplex
	if (inputs.job.paperSizeName == "A4") {
		newCost = inputs.job.totalSheets * COST_A4;
	}
	else {
		newCost = inputs.job.totalSheets * COST_A3 ;
	}

	newCost += inputs.job.totalGrayscalePages * COST_BW;
	newCost += inputs.job.totalColorPages * COST_COLOUR;

	 actions.job.setCost(newCost);

	actions.log.debug("Staff Pricing on " + inputs.job.printerName + " For " + inputs.job.totalSheets +
		" sheets had " + inputs.job.totalGrayscalePages + " B/W & " +
		inputs.job.totalColorPages + " Colour, Cost after discount: " + newCost);

	// Record that as discount was applied in the job comment.
	actions.job.addComment("Staff Pricing applied. Had " + inputs.job.totalGrayscalePages + " B/W & " +
	inputs.job.totalColorPages + " Colour");
	}
}

Other useful features:

• Students and staff see a summary of how many pages they are going to print, which helps reduce accidental printing of entire chapters of text instead of just a few pages.
• Logs and statistics show which printers are getting used, how much is getting printed, and who printed the document but didn’t collect it.
• Great support. I have contacted them over a number of issues and they have always gotten back quickly with good ideas, or have added features in later versions.

Downsides:

• Although the price for education version of Papercut NG is not too bad, to get the MF version which supports copiers it costs around twice as much. Also you then need cost recovery devices for the copiers ~AUD$1,800 each, and there are annual support and maintenance costs ~AUD$1,000/year.

If you want to get an idea of how much printing your staff and students or just want to keep a record of who is printing what (like you could do easily with iPrint)  try the free Papercut Print Logger.

Here are some simple VBScripts that can be added to a log on script or similar to pre-fill these for the user. Once you know the registry location where the identity information is stored it is quite easy to fill those values as part of a login script.

Microsoft Office

' Original MS Office script written by David Isaacs
Set oShell = CreateObject("WScript.Shell")

On Error Resume Next

strUsername = oShell.ExpandEnvironmentStrings("%USERNAME%")
strUserdomain = oShell.ExpandEnvironmentStrings("%USERDOMAIN%")

Set oUser = GetObject("WinNT://" & strUserdomain & "/" & strUsername & ",user")

oShell.RegWrite "HKCU\Software\Microsoft\Office\Common\UserInfo\UserInitials", strUsername
oShell.RegWrite "HKCU\Software\Microsoft\Office\Common\UserInfo\UserName", oUser.Fullname

Adobe Acrobat Pro and Reader

This will set the full name, office, email address and your company for multiple versions of Acrobat and Acrobat reader. You can add even more versions by adding extra lines to the array.

Also if you modified Acrobat to install with Acrobat.com disabled, but now wish to enable it this will enable it. We originally had it disabled, but found email and network form submission did not work properly until it was enabled.

The method for accessing the AD User object was posted by Mike Walker in this thread.

' Configure Adobe Acrobat default settings
' Written by James Rudd
Set oShell = CreateObject("WScript.Shell")
Set oFso = CreateObject("Scripting.FileSystemObject")

' Set the different registry paths for Acrobat
Dim regPaths(2)
regPaths(0) = "HKEY_CURRENT_USER\Software\Adobe\Adobe Acrobat\9.0\"  'For Acrobat Pro 9
regPaths(1) = "HKEY_CURRENT_USER\Software\Adobe\Acrobat Reader\9.0\"  'For Acrobat Reader 9
regPaths(2) = "HKEY_CURRENT_USER\Software\Adobe\Acrobat Reader\8.0\"  'Same for Acrobat Reader 8

On Error Resume Next

' Create the ADSystem Information Object
Set objADSystemInfo = CreateObject("ADSystemInfo")
' Get the current information into a new Object
Set objUser = GetObject("LDAP://" & objADSystemInfo.UserName)

For Each regPath In regPaths
 'Enable Acrobat.com by deleting key that contains disabling entries.
 oShell.regdelete regPath & "Workflows\"

 'Set Acrobat Identity Info
 oShell.RegWrite regPath & "Identity\tEMail", objUser.mail, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tName", objUser.givenName & " " & objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tFirstName", objUser.givenName, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tLastName", objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tCorporation", "Your Company Name", "REG_SZ"
 oShell.RegWrite regPath & "Identity\tDepartment", objUser.physicalDeliveryOfficeName, "REG_SZ"

 'Set Default Acrobat Collaboration details
 oShell.RegWrite regPath & "ShareIdentity\tEMail", objUser.mail, "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tFullName", objUser.givenName & " " & objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tCorporation", "Your Company Name", "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tDepartment", objUser.physicalDeliveryOfficeName, "REG_SZ"
Next

Combined

The following script combines both Office and Acrobat data in to one, and reuses the same data objects rather than use two different connection techniques.

' Configure Adobe Acrobat and MS Office user settings
' Written by James Rudd

Const strCompanyName = "Your School Name"

Set oShell = CreateObject("WScript.Shell")
Set oFso = CreateObject("Scripting.FileSystemObject")

' Create the ADSystem Information Object
Set objADSystemInfo = CreateObject("ADSystemInfo")
' Get the current information into a new Object
Set objUser = GetObject("LDAP://" & objADSystemInfo.UserName)

On Error Resume Next

'Office Details
oShell.RegWrite "HKCU\Software\Microsoft\Office\Common\UserInfo\UserInitials", objUser.sAMAccountName, "REG_SZ"
oShell.RegWrite "HKCU\Software\Microsoft\Office\Common\UserInfo\UserName", objUser.givenName & " " & objUser.sn, "REG_SZ"
' If set by installer Company Name is overidden on load.
oShell.RegWrite "HKCU\Software\Microsoft\Office\Common\UserInfo\Company", strCompanyName, "REG_SZ"

' Set the different registry paths for Acrobat
Dim regPaths(2)
regPaths(0) = "HKEY_CURRENT_USER\Software\Adobe\Adobe Acrobat\9.0\"  'For Acrobat Pro 9
regPaths(1) = "HKEY_CURRENT_USER\Software\Adobe\Acrobat Reader\9.0\"  'For Acrobat Reader 9
regPaths(2) = "HKEY_CURRENT_USER\Software\Adobe\Acrobat Reader\8.0\"  'Same for Acrobat Reader 8

For Each regPath In regPaths
 'Enable Acrobat.com by deleting key that contains disabling entries.
 oShell.regdelete regPath & "Workflows\"

 'Set Acrobat Identity Info
 oShell.RegWrite regPath & "Identity\tEMail", objUser.mail, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tName", objUser.givenName & " " & objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tFirstName", objUser.givenName, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tLastName", objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "Identity\tCorporation", "Your Company Name", "REG_SZ"
 oShell.RegWrite regPath & "Identity\tDepartment", objUser.physicalDeliveryOfficeName, "REG_SZ"

 'Set Default Acrobat Collaboration details
 oShell.RegWrite regPath & "ShareIdentity\tEMail", objUser.mail, "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tFullName", objUser.givenName & " " & objUser.sn, "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tCorporation", strCompanyName, "REG_SZ"
 oShell.RegWrite regPath & "ShareIdentity\tDepartment", objUser.physicalDeliveryOfficeName, "REG_SZ"
Next

This post is to assist in the setup of a BackupPC system able to backup in use files by using MS volume snapshots. It also has the benefit of only having the RSync daemon running during the backup which increases security.

This method is based on some other posts I have seen using VShadow, Backing Up Open Files on Windows with Rsync, and some suggestions on DiskShadow but goes further in using RSync as a system service giving full access to files, and removing the need to use winexe.

General Outline

1. Create a new user backuppc (try to match case of the user on Linux). Very limited rights
2. Install Cygwin with RSync, OpenSSH and configure them
3. Create a scheduled task to run as SYSTEM when triggered by a certain event
4. Set BackupPC server for passwordless login to host and modify Pre/Post Dump Cmds

Overview

In Windows 7 and Server 2008 R2 elevation is required to create snapshots. As a remote SSH connection cannot bypass UAC a way is needed to create the snapshot, bypassing the elevation prompt. I also wished to run the Rsync Daemon as SYSTEM user so it has rights to view all files.

To do this I moved all the elevated tasks into a Task Scheduler item that is set to run as SYSTEM, and is triggered by an Event log event.

1. BackupPC performs a password-less key SSH logon to client
2. It runs a script which creates an event log entry and then waits for RSync to start before returning to BackupPC and starting the backup.
3. Task scheduler is triggered by Event Log and starts DiskShadow as SYSTEM.
4. Disk shadow creates a shadow of any chosen volumes, mounts them and then starts RSync.
   It then waits for a file to be created by BackupPC at the end of the backup telling it to stop RSync and delete the snaphots.

Flow and triggers

Host PC

Create a new user, backuppc, you can limit this account further in Security policy after everything is configured.

Create a BackupPC folder and add the following scripts to it. These are also available in a zip file. Note: There is a file embedded within this post, please visit this post to download the file. You will need to modify paths depending on where you create the folder. I used C:\cygwin\BackupPC but a better location may be C:\cygwin\usr\share\BackupPC

Most of these scripts are just modified versions of the ones written for VShadow, changed to work with DiskShadow and Task Scheduler.

pre-cmd.vbs

' This file starts the commands
' It will start the snapshot process and quite once RSync is running

Const Rsync = "C:\cygwin\var\run\rsyncd.pid"
Const Flag = "C:\cygwin\var\run\wake.up"
Set fso = CreateObject("Scripting.FileSystemObject")
'
' Pid file shouldn't be there already
' Check /stop service , still there delete
'
If DoesFileExist(Rsync)=0 Then
	fso.DeleteFile(Rsync)
End If
'
' Nor should "wake.up"
'
If DoesFileExist(Flag)=0 Then
   fso.DeleteFile(Flag)
End If

Set objShell = CreateObject("WScript.Shell")
' objShell.Exec "C:\BackupPC\backuppc.cmd > " & Log

' This writes event log entry that triggers task scheduler to start system process
' that takes snapshot and starts RSync
objShell.Exec "Logevent.exe -r ""BackupPC"" -e 10 -s S ""Backup Start"" "
Wscript.Echo "Sent BackupPC Event Log Trigger"

'
' Just sleep until the file "rsyncd.pid" appears
'

While DoesFileExist(Rsync)
   wscript.sleep 10000
Wend

' functions

function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing

end function

backuppc.cmd

c:
cd C:\cygwin\BackupPC
diskshadow /s DiskShadowScript.txt /l C:\cygwin\var\log\diskshadow.log
del C:\cygwin\var\tmp\*.cab /q
c:cd C:\cygwin\BackupPCdiskshadow /s DiskShadowScript.txt /l C:\cygwin\var\log\diskshadow.log
del C:\cygwin\var\tmp\*.cab /q

DiskShadowScript.txt

#DiskShadow script file

#Make shadows persistent, No writers as data volume

# If backing up C: and any app files (ntds, database, etc) use writers

#SET CONTEXT PERSISTENT NOWRITERS

SET CONTEXT PERSISTENT

#Cab location for process

SET METADATA C:\cygwin\var\tmp\backup.cab

SET VERBOSE ON

BEGIN BACKUP

#Alias volume with alias

ADD VOLUME C: ALIAS SystemData

ADD VOLUME F: ALIAS UserData

#Create Snapshot

CREATE

#Expose the volume and run command file then unexpose

EXPOSE %UserData% B:

EXPOSE %SystemData% T:

EXEC C:\cygwin\BackupPC\Serverbackup.cmd

UNEXPOSE B:

UNEXPOSE T:

END BACKUP

#Delete the shadow copy

DELETE SHADOWS SET %VSS_SHADOW_SET%

#End of script

Serverbackup.cmd

REM Start RSync now that Snapshots are created

net start rsyncd

REM Need to wait until backup completed

cscript "C:\cygwin\BackupPC\sleep.vbs"

Logevent.exe -r "BackupPC" -e 20 -s S "Backup Completed"

sleep.vbs

Const Rsync = "C:\cygwin\var\run\rsyncd.pid"
Const Flag = "C:\cygwin\var\run\wake.up"
Set fso = CreateObject("Scripting.FileSystemObject")

' Just sleep until the file "rsyncd.pid" appears
While DoesFileExist(Rsync)
   wscript.sleep 10000
Wend

' Now sleep until the file "wake.up" appears
While DoesFileExist(Flag)
   wscript.sleep 10000
Wend

fso.DeleteFile(Flag)

' It's time to kill Rsync
'Stop Service
strServiceName = "rsyncd"
Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")
Set colListOfServices = objWMIService.ExecQuery("Select * from Win32_Service Where Name ='" & strServiceName & "'")
For Each objService in colListOfServices
    objService.StopService()
	Wscript.Echo "RSyncD Stopped"
Next

' Wait for Rsync to let go
wscript.sleep 5000

' Delete PID file
If DoesFileExist(Rsync)=0 Then
   fso.DeleteFile(Rsync)
End If

' functions
function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing

end function

Cygwin

Install Cygwin and choose to install packages RSync and OpenSSH

Start an elevated Cygwin (Run as an Administrator )

Config

Cygwin 1.7 changes the way to ignore NT Security from the nontsec environment to modifying fstab file.

In your prefered editor modify /etc/fstab and uncomment the bottom line and add noacl as below, this tells it to ignore security :

none /cygdrive cygdrive binary,noacl,posix=0,user 0 0

Run following to update group and user lists (if on a domain only add the users you want)

[bashlight=1]mkpasswd –l >/etc/passwd
mkgroup –l /etc/group[/bash]

OpenSSH

Configure Open SSH using ssh-host-config, create both the accounts it suggests for privilege isolation and running the service (cyg_service & sshd).
This should also automatically add the Firewall Exceptions (SSHD).

Setting up Keyless

Login as backuppc user.
Use

runas /user:backuppc cmd

Or

runas /user:domain\backuppc cmd

Run c:\cygwin\Cygwin.bat to start Cygwin as BackupPC user

You need to add the id_rsa.pub file from the BackupPC user on BackupPC server to C:\cygwin\home\backuppc\.ssh\authorized_keys If you have not already created one follow the instructions below in BackupPC – SSH section.

RSync

To install RSync as a system service use:

C:\cygwin\bin\cygrunsrv.exe -I rsyncd -d "RSync Daemon" -O --type manual -p /bin/rsync.exe -f "Used by BackupPC to remotely access files for backup" -a " --config=/etc/rsyncd.conf --daemon --no-detach"

Modify the /etc/rsyncd.conf file to reflect what drives you want to backup. It is better to edit this file later after you have modified the DiskShadowScript.txt and changed which drive letter it exposes shadows as.

Add a RSync exception to the firewall.

Open Windows Firewall with Advanced Security and choose Inbound Rules, New Rule, Program,  Next

Browse to C:\cygwin\bin\rsync.exe and then choose your options and name the rule

When finished open the rule, click Scope tab and add a Remote IP, that of the BackupPC server. This restricts RSync to only be accessible from BackupPC. You may also wish to similarly modify SSHD to only allow SSH access from BackupPC server.

DiskShadow

If you are backing up a windows Server 2008 or 2008 R2 host you already have DiskShadow installed. However, if you are running Windows 7 or Vista you will need to grab a copy from an equivalent server (x86 or x64). It is located in the System32 directory and you will also need the language file from the en-US folder.

I have zipped the files needed for x86 and x64 if you do not have immediate access to a Server.Note: There is a file embedded within this post, please visit this post to download the file. (The x86 files are from Server 2008 and x64 are from 2008 R2)

Copy the files from your architecture to the system32 directory and en-US subdirectory.

LogEvent

To generate the custom event log entry a tool from the Windows 2000 Resource kit is used, LogEvent, (Download). This needs to be either placed in the Path (e.g. Windows dir) or scripts need to directly call it.

Task Scheduler

The easiest way to configure this is to manually run LogEvent once to generate an event in the log.

Logevent.exe -r "BackupPC" -e 10 -s S "Backup Start"

Then open Event Viewer, Select the new BackupPC event and choose Attach Task to this Event, and in the wizard click next until it asks for the program, then give it C:\cygwin\BackupPC\backuppc.cmd

On Final page choose to Open Properties when you click Finish

Click the Change User or Group button and type in System as the user  and click OK. Also tick the Run with highest privileges box

BackupPC Server

SSH

This section is only needed if you do not already have keys generated for the backuppc user.

Login as backuppc user, either with password or simply “su – backuppc” from root

Generate SSH Keys, ssh-keygen –t rsa, do not set a password. Copy id_rsa.pub into C:\cygwin\home\backuppc\.ssh \authorized_keys

Test by running ssh -v backuppc@host to test the connection

BackupPC Host File

In the web interface change the following for the host.

DumpPreUserCmd:  $sshPath -q -x -l backuppc $host cscript "C:\cygwin\BackupPC\pre-cmd.vbs"
 DumpPostUserCmd: $sshPath -q -x -l backuppc $host echo "Complete: $xferOK" > /var/run/wake.up

Test Run

If you have existing backups from a previous RSync config, it is a good idea to run a full backup to ensure any in use files are part of the base. In my case I also changed the noacl flag which affected the file attributes RSync sees.

Tips

Set your antivirus to exclude the exposed drives. As they are read only it just slows down the reading of files by RSync.

Downloads

Updates

• While writing this up I found a good alternative way of using VShadow with SSH to backup in use files, that uses the AT command instead of task scheduler to get around UAC, BackupPC with ssh/rsync/VSS on Windows Server. The only down side to this method is the SSH connection requires an Admin account, but if configured securely this should be fine. It also a lot simpler to configure as it does not require configuring Task Scheduler.

Finally I found a Technet discussion on the issue, which correctly identified an incorrect registry value HPTrayCount which is set to 0 when the printer driver is changed, but is not set back to a working value.

This value needs to be changed to 0×12 (decimal 18) for each HP printer for it to work correctly. However, on print servers with lots of HP printers this can be very time consuming. To speed this process I have written the vbscript below which will check if the value exists for each printer, and if it is set to 0 will set it to 0×12.

' For each Printer in Registry check if HPTrayCount exists and is set to 0, then set to 12

'http://social.technet.microsoft.com/Forums/en/winserverprint/thread/5101195b-3aca-4699-9a06-db4578614e2d

strKeyPath = "SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Printers"
strValueName = "HPTrayCount"
inHPValue = 18 ' Is equal to Hex 12, makesprinter driver work.

'http://www.activexperts.com/activmonitor/windowsmanagement/adminscripts/registry/#EnumRegVals.htm
const HKEY_LOCAL_MACHINE = &H80000002
strComputer = "."
Set StdOut = WScript.StdOut

Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & _
strComputer & "\root\default:StdRegProv")

' Get list of Printers
oReg.EnumKey HKEY_LOCAL_MACHINE, strKeyPath, arrSubKeys

For Each subkey In arrSubKeys
  ' Read in HP Tray value
  strPrintPath = strKeyPath & "\" & subkey & "\PrinterDriverData"
  oReg.GetDWORDValue HKEY_LOCAL_MACHINE,strPrintPath,strValueName,dwValue
  If Not IsNull (dwValue) And dwValue = 0 Then
    'If Exists and 0 then set it to a real value to make it work
    StdOut.WriteLine subkey & ": HP Tray: " & dwValue
    oReg.SetDWORDValue HKEY_LOCAL_MACHINE,strPrintPath,strValueName,inHPValue
  End If
Next

Windows mail seems to generate large amounts of ESENT messages in the log file as it tries to backup the mail database. It also create 5Mb of files for every user as they logon. As we don’t even use Windows Mail, just Outlook for staff I wanted to disable it.

There appears to be no easy way to remove it from the system, but you can prevent it from setting up the details for every user when the log on.

Download and runs the Microsoft Sysinternals program Autoruns as an Administrator and then under the Explorer tab untick “Microsoft Windows” under both the Active Setup\Installed Components and the WOW6432Node\Active Setup\Installed Components.

Now when a new user logs in it will no longer set up Windows Mail files or shortcuts.

Autoruns Windows Mail Disable

Recently I discovered our Moodle gradebook was playing up and not correctly recording marks from assignments as well as been unable to assign categories from quizes.

I tracked the problem down to the Moodle database losing all its default entries for its tables. The cause is unknown but is quite likely the UTF conversion utility I ran previously to make the system compatible with Asian languages for LOTE.

To fix this problem I needed to recreate the Moodle DB structure and then reimport the data.

Backup

First make a backup. I made a mistake the first time I tried repairing this and the backup I made saved me.

mysqldump -u backupacc -p -Q --opt moodle > moodleBackup.sql

I created a duplicate site with a clean DB (copy the moodle folder and create a new DB) and went through the install. This needs to have all your modules and blocks to ensure all tables are created.

I then dumped the structure from this new DB and the old DB and compared them using Meld (or any other diffing program will work).

 mysqldump -u backupacc -p --no-data --skip-add-drop-table moodletest2 > moodleGoodStruc.sql
mysqldump -u backupacc -p -d --skip-add-drop-table moodle > moodleOldStruc.sql

Using Meld determine if there are any extra tables or fields in your current system that don’t exist in the clean one.

Then you need to determine if it is safe to delete those tables / field in your main DB or if they need to be added to the clean system. e.g. grade_items_history.decimals and grade_items_history.display were in the main DB but not the new one. After searching for it discovered they can be removed (MDL-15985).

After getting them to match you need to dump your Moodle data from your main site.

You can speed up the next steps by reducing the size of backup_log if you wish. Just do a

Delete
FROM `mdl_backup_log`
WHERE `time` <1242777600

Where the time code is calculated as a unix time code from about a month before todays date. This can reduce alot of space of your DB, and improve import/export times.

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

You need the –complete-insert option to ensure each insert is labelled with field name, in case field order is different. I didn’t do this the first time and needed to restore the backup.

Testing Merge

Now you need to test it. Create a new empty DB and import the Structure then import your data.

mysql -u root -p moodleTest3 < moodleStruc.sql
mysql -u root -p moodleTest3 < moodleData200906171559.sql

Check that no errors occurred during import, this is when I discovered most of my problems with extra non existent field.

Prep for Applying

Put your site in Maintenance mode so no users can login while you are testing this. (i.e. when you restore data site will already be in maintenance mode)

When everything is OK do another backup, as after this you are going to delete it all. (I recommend taking your site completely offline for this, not just maintenance mode. Edit your config.php and temporarily change either the username, password or db name or your database settings.)

Now re-export the Data again (assuming your site has been running while you have been doing tests)

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

Applying to Site

WARNING: HAVE A GOOD RELIABLE BACKUP BEFORE THIS STEP.  THIS WILL DESTROY YOUR MOODLE SITE.

Now use phpMyAdmin to drop every table in the Moodle main DB. Just use Check All at the bottom and select Drop.

Now you need to redo your test restore above but to your main DB.

Once complete reconnect your config.php file and check everything on your site looks the same. Open some courses, check some gradebooks, have a look around.

When its OK take out of Maintenance mode and you should be good to go.

It used to be fairly easier to deploy on your network, but after version 4.2 it became significantly more difficult if you use a deployment program like Zenworks. A new MSI test was introduced to determine where to install Google Earth. If the current user is not an Admin User it would redirect to [LocalAppDataFolder]Google\Google Earth (usually C:\Documents and Settings\username\Local Settings\Application Data\Google\Google Earth) which made it inaccessible to other users.

As Zenworks uses the System account, which is not an admin user, this would happen when the installer was launched by a student.

To fix this problem there are a few different techniques. You can use either ORCA or Admin Studio Tuner (from Zenworks) to create a transform file that ignores these checks. Alternatively you can use ORCA to edit the msi file directly, I had to use this technique for V5 due to some exisiting problems in the Google Earth MSI.

First download the full version of Google Earth, (V5.0.11337). You should be able to find later versions through Google.

To find what you need to edit you can do a search in ORCA for AdminUser you should find around 3 entries.

The major entries are:

• InstallExecuteSequence: ChangeInstallDirForNonAdmin: NOT AdminUser (Delete this row)
• InstallExecuteSequence: setALLUSERS: AdminUser (remove the word AdminUser so it applies to all installs)
• InstallUISequence: ChangeInstallDirForNonAdmin: NOT AdminUser (Delete this row)

You may also wish to remove AdminUser as a Condition from Component: Plus_Registry_wavdest.ax

Once these are removed or modified it should install in C:\Program Files\Google\Google Earth regardless of which user is logged in.

First a comment, although Microsoft seems to have a much better DNS system than on Netware 6.5, their DHCP implementation leaves a lot to be desired.

On Novell I could create a scope, reservations and options and it is stored in eDirectory. I could then have multiple DHCP servers with different IP ranges to offer. I could use the java console (sometime slow) to add a reservation and it would sync to both servers.

On Windows Server 2003 and 2008 I need to separate create scopes, allocation ranges and reservations on each server. Their is no synchronisation or communication between these servers. This means to migrate my reservations from my current Netware DHCP servers to the target Windows servers the reservations need to be added to each server, as well as adding any future reservations to both servers.

Migration Process

First use the Novell DNS/DHCP console and export the DHCP scope / database you are migrating.

I have written the following Perl code to read in the DHCP3TAB file generated and output a netsh file that will add all your reservations to any number of Windows DHCP servers.

You will need to modify the DHCP servers list to your AD servers and change the scopeName to the correct IP settings.

After running your DHCP3TAB through the perl script, copy output to your server and run netsh exec outputFile.txt to add it to your servers.

#!C:\Perl\bin\perl.exe

# Designed to read in Rservations from a Novell DHCP Tab file and output a NetSH script file
# From http://technet.microsoft.com/en-us/library/cc787375.aspx
# On the destination server, the exec command is used to load and execute the converted reservations:
#  netsh exec AdReservations.txt
# After you use the exec command to load the file, you must reconcile all scopes.
# Use net stop dhcpserver to stop the DHCP Server service and net start dhcpserver to restart it. Once the service is restarted, DHCP database changes take effect.

use strict; use warnings;

my $dhcptabName = "DHCP3TAB.txt";
my $outFile = "AdReservations.txt";
my @dhcpServers = ('\\\\DHCPServer1.win.us.schools.nsw.edu.au',
    '\\\\DHCPServer2.win.us.schools.nsw.edu.au');
my $scopeName = "10.10.11.0";

open F, "< $dhcptabName" or die "Can't open $dhcptabName : $!";
open O, "> $outFile" or die "Can't open $outFile : $!";

# File parsing
my $ip;
my $host;
my $mac;
my $type;
my $comment="";

while (my $line = <F>){
  if ($line =~ /^\[IP Address Configuration /i) {
  # New entry, clear values
    $ip="", $host="", $mac="", $type="", $comment="";

    while ((my $entry = <F>) !~ /^$/){
    # Parse and fill in values
      if ($entry =~ /IP Address Number = ([\d.]+)/i){
        $ip=$1;
      }
      elsif ($entry =~ /Assignment Type = (\d+)/i){
        $type=$1;
      }
      elsif ($entry =~ /Host Name = ([\w\-_]+)/i){
        $host=$1;
      }
      elsif ($entry =~ /MAC Address = 1 (.+)/i){
        $mac=$1;
        $mac=~ s/\s+//g;
      }
      elsif ($entry =~ /Comment = (.+)/i){
        $comment=$1;
      }
    }

    # Following determines which type of entries to convert. Default is only reservation.
    # You can comment it out and uncomment the second line to include reservations and allocated IPs.
    # If  type != 8 (reservation) discard
    next unless ($type == 8); 

    # If  type != 8 (reservation) or  != 2 (Allocated) discard
    # next unless  ($type  == 8 or $type  == 2);

    next if ($mac eq "" or $ip eq "" or $host eq "");
    foreach my $server (@dhcpServers){
      print O "Dhcp Server $server Scope $scopeName Add reservedip $ip $mac \"$host\" \"$comment\" \"BOTH\"\n";
    }
  }
}

close F;
close O;

As part of this rename we needed to change their Novell eDirectory account and home directory. We also needed to ensure linked systems such as an LDAP authenticated Moodle site and an IDM linked Active Directory (AD) domain were properly updated.

To perform the user account and folder rename we used Mass User from HBWare. This is a great program we already used for creating and managing home directories and quotas. There was a small bug in the rename area, but it was quickly fixed when brought to Hans’ attention.
Note: Before beginning you need to have a list of the old usernames and the new usernames. A simple two column Excel or CSV file would be fine.

Novell

Create Mapping File: The mapping file is used by Mass User to know what to rename the existing username to.

Use NDS Report ( http://www.novell.com/coolsolutions/tools/13908.html ) to create a list of all the accounts. You only want the DN and CN fields. Save this as an Excel or CSV file.

NDS Report: Select Student OU

NDS Report: Choose only CN

Create a new MS Access file and import the account list
Import the text / excel file that lists the old and new account names.
You now need to create a query that maps the cn to the old account name, and then use this to generate a list of the full old dn and the new account name.

Access: Mapping Query

Access: Query Results

You can then export this as a text file.

Access: Export as text file

Access: Export Delimited

You will need to set the field separate as ‘=’ and set the test qualifier to none.

Access: = as Delimiter and no Text Qualifier

You should end up with each line having format:
OLDNAME.OU.C=NEWNAME
Once the mapping file is generated you can apply it either to individual OUs (e.g. year groups), or to the entire Users container. I would recommend applying to small containers initially to allow checking for errors. After verifying all renames were performed correctly you can then apply the rename to your entire users’ container.

Mass User: Rename

IDM 3.0 – Active Directory

Before doing the mass rename we did some simple tests of renaming eDirectory accounts to see how they replicated to Active Directory. We determined that the individual renames were successfully propagated through IDM to automatically change the pre-2000 and logon name fields to match the new eDirectory account.
When performing the mass rename of accounts, AD was checked after doing each OU to check propagation was successful. We experienced no problems with IDM and all accounts were successfully synchronised with Active Directory.

Moodle

For Moodle we wished for users to retain their own accounts including all their settings and course information. To do this the username stored in the Moodle Database (DB) would need to be changed to the new username.
Before making changes to Moodle I recommend putting the site in admin mode and temporarily disabling your LDAP authentication. I only put the site in Admin mode and then had problems during migration as some students had attempted to logon after I had renamed the eDirectory accounts, but before migrating Moodle accounts. This created new user accounts in Moodle that prevented the update queries running due to duplicate key name problems (ie the old account would not rename to new one if one has already been created with the new name)
First you need to import your list of old and new account names into the database. If you already have it in a separate DB on your Moodle server you can use that, otherwise it is best to just to create a new table in the Moodle DB.
I used this query to check the mapping fields. You will need to adjust the fields and collation types for your own setup.
I use the IDNumber field to store the full DN, if you use a different field you will need to adjust this.

Validation:

# Shows the current username, the new username and the new IDNumber field for validation check.
SELECT mdl.`username` , ern.`StudentId` , replace( `idnumber` , mdl.`username` , ern.`StudentId`
COLLATE latin1_swedish_ci )
FROM moodle.`mdl_user` mdl, sbhsdata.`oasisStudentIdMap` ern
WHERE mdl.`username`
COLLATE latin1_general_ci = ern.`OldStudentId`

Update:

# Replaces the IDNumber field with the new username.
UPDATE moodle.`mdl_user` mdl, sbhsdata.`oasisStudentIdMap` ern
SET `idnumber` = replace( `idnumber` , mdl.`username` , ern.`StudentId` COLLATE latin1_swedish_ci )
WHERE mdl.`username`
COLLATE latin1_general_ci = ern.`OldStudentId`

# Replaces the mdl_user field with the new username.
UPDATE `mdl_user` mdl, `rename` ren
SET mdl.`username` = ren.`newname`
WHERE mdl.`username`= ren.`oldname`
Note: As can be seen by the queries, during the migration I did two separate updates, one to update the IDNumber field and another to update the mdl_user field. When I had completed migration I realised I had only needed to update the mdl_user field as Moodle would automatically update the value in IDNumber at next log on.

Round Up

The migration to the new accounts went well with no problems from student passwords or accounts, although we had a few instances of students using the old username instead of the new one.

Since performing the change we have also implemented a Papercut system which would have required the additional step of renaming all of its accounts to ensure student balances were carried across.

A great tool is Wisesoft Password Control which allows you to just type in the username, it will display info about the account and give you the option to change the password, enable/disable the account and unlock the account.
It is ideal to make available to teachers for resetting the student passwords if you use the Delegate control option in AD Users and Computers for your student OU’s.

The other useful tool on the website is a pair of tools Bulk Password Control and Bulk Modify. These allow you to bulk reset the password for a large group of students, or modify the attributes for a large number of users. It can read these in from a CSV file, and match the CSV entires to either sAMaccountName or some other user attribute. You can also set the attribute based on their existing attributes.

After downloading it will ask you for a code that you can get with a free registration on their site.