Make your own
Automated Backup
System

BETA Release Caveat

This is WEB release 1.1 BETA. It is only a Beta release, and I hope you still download all the files and try it. I have used a version of these command procedures for over 7 years now, but they had all been written specifically for my personal environment. This WEB version 1.1 Beta is a modification of my original routines to make it more general and easy to transport and install onto any system. I have not been able to test it on another system at this time, so I'm hoping you will do that for me. Please send me email with any problems and suggestions, and I'll respond with fixes immediately, and when I feel it is ready, I will make a full release with all fixes incorporated.

Installation Instructions

Prior to using this command procedure you should create a new account, I recommend calling it BACKUP, and it should have SYSTEM privileges, and a UIC in the system area like [1,5]. Since your BACKUP will be running under this account it is important that you set up the account's process quotas and limits appropriate for doing backups on your system. Doing this for your system is not a trivial task. Therefore, I have dedicated a separate web page dedicated to setting up your BACKUP account with the proper process quotas and limits. This account can have it's default directory on any disk (EXCEPT THE SYSTEM DISK), where the command procedures should reside as SYSTEM_BACKUP.COM, along with AUTO_DISK_REPAIR.COM, DEFINE_TAPE_DEVICE.COM, LOCATE.COM, DEFINE_CATALOG_DIR.COM, RESCHEDULE.COM, and DEFINE_BATCH_QUEUES.COM.

REMEMBER : It is very important that the BACKUP account's default root directory does not exist on the system disk. This is to prevent problems in the AUTO_DISK_REPAIR.COM command procedure.

After you have created the account, the backup root directory, and placed the above files in the root directory using their proper names, you then need to create the catalog directory. The catalog directory should go on a fairly large disk somewhere on your system. This directory will contain the catalog files,which later help you to determine what backup tapes and save sets specific files have been saved on, when you need to restore files. I have a specific disk on my system that is 4GB in size called SYS$CATALOG, and it has only one directory on it called SYS$CATALOG:[BACKUP_CATALOGS].

Now edit the file in the [BACKUP] root directory called DEFINE_CATALOG_DIR.COM. In this file, change the $DEFINE CAT_DIR command to point to your catalog device and directory.

This BACKUP utility runs in one of two modes. Single Job or Default Instance, or Multiple jobs or Dual Instance. The Single Job mode performs the backup of all disks in one batch job. The Multiple job version is if you have two tape drives, and you wish to backup some of your disks to one drive, and the other disks to the other tape drive. If you run in the multiple job mode, the first job is called INSTANCE A, and the second INSTANCE B. This can be helpful if you have a large disk farm that takes more than a day to perform a full image backup.

Now it's time to define our batch queue(s) and tape drive(s). First edit the file DEFINE_TAPE_DRIVES.COM. In this file you will find three symbol definitions currently set as :

$ BACKUP_TAPE_DEVICE   == "_$2$mkc400"
$ BACKUP_TAPE_DEVICE_A == "_$2$mkc400"
$ BACKUP_TAPE_DEVICE_B == "_$2$mka500"

Edit these definitions to reflect the tape drives you wish to use. The symbol BACKUP_TAPE_DEVICE is used for the single job/default instance mode. Likewise BACKUP_TAPE_DEVICE_A is for the multiple job mode, Instance A, and BACKUP_TAPE_DEVICE_B is for Instance B. If you run the single job/default instance, then the last two symbols are ignored, and likewise if you run in the multiple job/dual instance mode, the first symbol is ignored.

Just like the tape drives, the batch queues that the jobs run in are defined in the file DEFINE_BATCH_QUEUES.COM. Editing it, you will find these symbol definitions :

$ BACKUP_QUEUE   == "SPOCK$SYSMAN"
$ BACKUP_QUEUE_A == "SPOCK$SYSMAN"
$ BACKUP_QUEUE_B == "KIRK$SYSMAN"

As in the symbol definitions for tape, the first is for single job/default instance, and the last two are for multiple job/dual instance.

If you have already not done so, create a LOGIN.COM file for your BACKUP account. (You may want to use my LOGIN.COM example from my WEB pages as a baseline) In the INTERACTIVE section of the LOGIN.COM file, enter in these two symbol definitions :

$ LOCATE == "@SYS$LOGIN:LOCATE.COM"
$ RESCH*EDULE == "@SYS$LOGIN:RESCHEDULE.COM"

Now, you must create two more text files. The first is the file called MAILING_LIST.DAT. Using the text editor, create this file, and on the first line compose a list of usernames separated by commas (no spaces). This is the list of people who would receive daily mail messages at the end of each backup job. Only the first line of this file is read. All other lines are ignored.

Now using the editor, create the file DISK_SCHEDULE.DAT. This file specifies the disks to be backed up, and the baseline names of the backup savesets. If you are going to be operating in the multiple job/dual instance mode, then create two files, one for each instance; DISK_SCHEDULE_A.DAT and DISK_SCHEDULE_B.DAT. In this/these file(s), each line specifies the logical or physical name of the disk to be backed up, and it's associated baseline saveset name. Any line with an exclamation point in the first collum is treated as a comment. For example :

!**************************************
!***
!*** Example DISK_SCHEDULE.DAT file
!***
!**************************************
!
!*** First do the system disk
_$4$DKA0:,SYSDEV
!
!*** Now the user disks
SYS$USER1,USER1
SYS$USER2,USER2
SYS$USER3,USER3
!
!*** Then the data disks
SYS$DATA1,DATA1
SYS$STARS,STARS
SYS$PLANETS,PLANETS
[EOF]

Log out, and back in again, and you are now ready to begin your automated backup procedure.

Theory of Operation

Automatic backups work on a weekly cycle, performing a complete image backup of all specified drives on SATURDAY, beginning at 1:00 in the morning. Then each subsequent day, incremental backups of only new files or files modified since the last backup, will be recorded.

Starting with SATURDAY's image backup, backups will be started on a new tape volume. Subsequent incremental backups will be appended to the existing tape. The BACKUP saveset names are constructed using the baseline name from the second field in DISK_SCHEDULE file, and the first two characters of the day of the week that the backup is being performed on. For example, if the line in the DISK_SCHEDULE file is "SYS$USER1,USER1" then Saturday's image backup saveset is called USER1_SA.BCK. Likewise, the following incremental backup savesets will be called USER1_SU.BCK, USER1_MO.BCK, USER1_TU.BCK, USER1_WE.BCK, USER1_TH.BCK and USER1_FR.BCK.

For each backup of each disk, each day, there is a text file called the catalog file which is generated by the BACKUP/LIST qualifier. These files arestored in the catalog directory, and are named as in this example :

CAT_1999JUN26_USER1_SA.CATALOG

In the above example, the catalog file was generated on June 26, 1999 and contains the list of files backed up on the USER1 disk. The _SA indicates that this was an image backup performed on Saturday. The name of the saveset you need to restore from is USER1_SA.BCK.

After the completion of each backup job, the users specified by the file MAILING_LIST.DAT will receive a mail message concerning the results of the backup job. A typical backup mail message will contain a message for each disk processed in the job. Here is an example of a message indicating a successful backup of a disk :

24-JUN-1999 23:43:15.79:
$BACKU/VERIFY/MEDIA=COMP/IGN=(LAB,INTE)/REC/LIS=CAT_DIR:CAT_1999JUN24_USER1_TH.CATALOG -
/SINCE=BACKUP SYS$USER1:[*...]*.*;* _$4$mka5:USER1_TH.BCK/SAVE/BLOCK=65535/LABEL=990602
+++ Successful backup completed at 25-JUN-1999 00:05:13.65
Catalog file : $4$DKA300:[BACKUP_CATALOG]CAT_1999JUN24_USER1_TH.CATALOG;1
Device       : SYS$USER1
Saveset name : USER1_TH
Volume Label : 990602
Instance     : DEFAULT

Notice the "+++" prior to the words "Successful backup". If for some reason, the backup of this device failed, this line would be prefixed with "---" indicating a failure. If there is an error then the VMS Error message will also be displayed.

Sometimes a problem can keep an entire backup job from running at all. In this the mail message might contain the message :

%%% Backup failed because the backup tape failed to mount.
None was found in tape drive _$4$mka5.
Please initialize a new tape with the label "FREE", and place it in the drive.
Then resubmit the backup batch job with P1 = "SATURDAY".
Logfile closed at 22-JUN-1999 22:00:15.72

In most cases, the backup will resubmit itself for the processing to be done the following day. If for some reason, as in the above message, the job is not submitted, you are warned of this, as well as instructions for recovery. Sometimes the batch job aborts without being able to mail a message. If this occurs, then the job did not resubmit itself. If this is the case, you would not get a mail message as you might expect. If you suspect this has happened, first enter in the command

$SHOW ENTRY/USER=BACKUP

to see if the backup job(s) is/are still running. If they are not, go see the file SYSTEM_BACKUP.LOG (or SYSTEM_BACKUP_A.LOG/SYSTEM_BACKUP_B.LOG) in the [BACKUP] root directory to see what has happened, and correct the problem.

On Saturday's, to prevent the image backup operation from overwriting an existing backup tape, the procedure first mounts the tape to check the tape's volume name. The image backup will only continue if the tape volume name is set to the four character string "FREE". This means that you must pre initialize the tape with the label "FREE". If you have a multiple tape loader which switches tapes when a new one is requested, you would want to pre-initialize all tapes with the volume name FREE. This will insure that all new weekly image backups will begin on a new tape volume, and subsequent incremental volumes will be renamed by the backup utility.

The first volume of a weekly tape set will be given the six character label yymmdd, indicating the last two digits of the year, two digit month, and two digit day of the image backups, for example 990626, indicating June 26, 1999. Subsequent volumes will be labeled the same except the last two digits will always be 02. This is because of the designed behavior of the BACKUP command.

Additional Operations

 In a Open VMS system, BACKUPS of your disks are just one aspect of disk maintenance that should be done. The others are disk repair, and volume rebuilding. Both of these operations can also be done by the automated backup procedure.

Volume rebuilding concerns the maintenance of the disk bit allocation map which describes which blocks on a disk are being used or not. Because this information is cached into memory while a volume is mounted, if a system crashes, or a disk goes offline without being properly dismounted, the allocation map becomes stale. This means that some blocks may be improperly marked as allocated even though they are not. Normally, when the system reboots and disks are mounted, this bit map is rebuilt, however this operation takes time, and many system managers use the /NOREBUILD qualifier on the mount command when the system starts up, to speed up boot time. If this is done, then it is important to rebuild the disk volumes on a regular basis. By default, the backup procedure performs a complete volume rebuild of each disk prior to backing it up. Even if you don't use the /NOREBUILD qualifier on Startup mounts, I recommend you still let the backup procedure perform the rebuild. At worst case, it only adds a few minutes to the backing up of each disk volume. However, if you want to prevent backups from rebuilding each volume prior to backing up the volume, you can specify "NOREBUILD" as an option when starting or rescheduling backup jobs.

Disk repair is much more involved than volume rebuilding. In the normal course of operation under Open VMS, various portions of the Files-11 ODS (On Disk Structure) may become corrupt. Because of this, the automated backup procedure performs a complete disk check and repair on all volumes to be backed up, just prior to beginning the Saturday Image backups. The results of this operation, and errors corrected is also mailed out to the designated recipients in a separate mail message. Whenever an image backup is being performed, even if one is forced on a non-Saturday, the procedure will perform a disk check/repair on each volume, unless the "NOREPAIR" option is specified when starting or rescheduling backup jobs.

If the "NOREBUILD" option is specified, then it will remain in effect for all subsequent jobs that are automatically rescheduled. To cancel the "NOREBUILD" option, you must abort the current jobs, and reschedule them without the option. However the "NOREPAIR" option is only valid when rescheduling an image backup, and subsequent image backups will perform a complete disk check/repairs.

Starting and Rescheduling
BACKUP Jobs

The starting of the backup jobs can be complicated if just using the DCL SUBMIT command. However it can be done, but you must submit it to the proper queue as defined in the DEFINE_BATCH_QUEUE.COM file, and you must specify the three parameters properly. For information on the parameters, see the "DOCUMENTATION SECTION" in the command procedure SYSTEM_BACKUP.COM.

There is an easier way to do it. Use the command procedure called RESCHEDULE.COM. If you properly defined the symbol in the LOGIN.COM file, you can just enter in the command :

$ RESCHEDULE

and you will be hand walked through the procedure, complete with specifying options. 

After Backup ...
Locating Files to be Restored

The LOCATE command can be used (as long as you have it properly defined in your LOGIN.COM) to determine where a specific file or set of files is located in your archive of backup tapes. The basic form of the LOCATE command is as follows :

$LOCATE filestring [begindate] [enddate]

Where :

• filestring Is the name of the file you are looking for. It may be any substring of a complete file specification from directory name to version number. (Wildcards and disk devices not allowed).
• begindate Is the optional begin date to start the search. It must be a valid VMS date specification. If omitted, all backup catalogs are searched.
• enddate Is the optional ending date to end search. Must be a valid VMS date specification. If omitted, searches continue to last backup.

If any portion of the "filestring" is located in a complete file specification in any of the catalog files, it will be shown as well as the name of the catalog file(s) it was found in. So if the LOCATE command yields the catalog file name of :

CAT_1999JUN26_USER1_SA.CATALOG

then go get your June 26, 1999 backup tape, and restore the file from the save set USER1_SA.BCK. In a future revision, you will be able to optionally specify diskname and/or day of the week.