Tuning the BACKUP Account
for the

Automated Backup
System

Revised February 15, 2000


Selecting the proper process quotas and limits for the BACKUP account for your Automated Backup System is very important to quick backup operations without errors. Normal process quotas and limits can cause your automated backup job to take longer than necessary, however just raising values in the authorization record for the backup account can cause very strange errors, or even hang-ups when backing up. This section will give you a little background on how backup attempts to optimize operations and how you can adjust the authorized process quotas and limits for the backup account to get the most out of backup. You may already have a backup system established, or you may want to use the Automated Backup System provided at this site. Either way, you should concern yourself with the proper setting of the process quotas and limits for doing backups.



Why BACKUP has
special needs

Digital Equipment, now Compaq, has made many improvements to the BACKUP utility in the 20 some years of it's existence. Between version 5.x and 6.x of VMS they paid particular attention to optimizing disk reads when backing up from disk to another source. In an absolute ideal environment, the most efficient method would be to copy the entire disk to memory, then randomly access the memory in the order that you need to write out the saveset. But in the real world (even with virtual memory), this is not a realistic approach, so a balance must be struck. Backup therefore will optimize it's disk reads based on the amount of memory and other resources that are available to it.

To do this, backup first determines which blocks from the disk need to be read into it's available memory. It then sorts this list so that all the reads needed to get the data are sequential as to minimize disk seeks. When this data is written to memory, it is written in the order that it is to be written to the output file or device. Then the data is written to the output device in large blocks. The read and write operations are done in two execution threads (via AST's, or Asynchronous System Traps) within BACKUP, such that when the first set of blocks is being written to the output, the read thread is determining which are the next blocks to be read from the input.

So, the more memory that is available to backup, the larger the blocks of reads and writes can be. Also, the more IO operations that can be queued up at a time the faster data is moved to and from memory. So the best thing to do is to maximize these process quotas and limits as to get the most out of backing up, without adversely impacting your system's performance.



Evaluating your System

First you must evaluate your system by looking closely to the particular node(s) your backups will be running on. The first question you must ask yourself is "When I do my backups, what else might I expect to be going on at the same time?" In other words if you backup at night, and typically nobody works at night, and there are no overnight jobs running, then you effectively have the entire system to yourself, and you can set up backup to use all available resources. However if your system is often or sometimes busy at night, and you do not want to interfere with overnight processing, then you must look at what resources are available when typical overnight work is being done. In this case, you may have to strike a balance between overnight job performance, and getting your backups done before your users come into work and start loading your system.

If your backup runs on more than one node, you must evaluate all the involved nodes, and use the minimum value of all the nodes involved.

The first thing to determine is the amount of free memory that is typically available. To determine this you must evaluate the system when it is experiencing it's maximum or typical processing load during the time that backups would normally be run. To determine the amount of free memory, enter in the command :

$SHOW MEMORY/FULL

And look for the sections …

Physical Memory Usage (pages): Total  Free   In Use  Modified
Main Memory (1024.00Mb) 131072
115670 13735 1667
                              

Dynamic Memory Usage (bytes): Total Free   In Use Largest
Nonpaged Dynamic Memory 12034048
5756416 6277632 2029824
Paged Dynamic Memory 3866624 1769504 2097120 1761680

                              …

The important thing to observe here is the number of free pages of physical memory, and the number of free bytes in Nonpaged dynamic memory during typical loading of your system. Now it's time to look at some SYSGEN parameters. Determine the values for the two SYSGEN parameters WSMAX, VIRTUALPAGECNT, and CHANNELCNT by entering in these three DCL commands :

$MCR SYSMAN PARAMETER SHOW WSMAX
$MCR SYSMAN PARAMETER SHOW VIRTUALPAGECNT
$MCR SYSMAN PARAMETER SHOW CHANNELCNT

First make sure that the value for WSMAX is at least as big a the number of free pages of main memory that you determined from your system during typical loading. If WSMAX is less than the number of free pages, then you will want to increase WSMAX to a round number near the number of free pages.

Follow these steps only if you need to increase the value for WSMAX


  1. Edit the file SYS$SYSTEM:MODPARAMS.DAT.
  2. Insert the line MIN_WSMAX = nnnnn where "nnnnn" is at least the value determined as the typical number of free pages during typical loading, but never greater than the total number of pages. If there is already a line for WSMAX, replace it with this new one.
  3. Exit the editor.
  4. When convenient, AUTOGEN your system, and reboot. To AUTOGEN and reboot, enter in the command :

    $@SYS$UPDATE:AUTOGEN SAVPARAMS REBOOT


Adjusting BACKUP's
Process Quotas and Limits

To adjust the process quotas and limits for the special BACKUP account, you must use the VMS AUTHORIZE utility. How to use the AUTHORIZE utility to change these values will not be discussed here. I make the assumption that you already know how to use the utility and are capable of modifying parameters for the special BACKUP account.

PGFLQUO : Pagefile quota.

Set the process pagefile quota equal to the SYSGEN parameter VIRTUALPAGECNT divided by the number of concurrent backup jobs you plan to run. If either of these values, VIRTUALPAGECNT or PGFLQUOTA, are too low, you can get an error stating that there is insufficient virtual memory.

WSQUO & WSEXT : Working set quota and Working set extent.

Set both of these values to the lower of the two SYSGEN parameters WSMAX and VIRTUALPAGECNT-3000. That is, subtract 3000 from VIRTUALPAGECNT and compare it to WSMAX, and choose the lowest of the two values as your value for WSQUO and WSEXT.

FILLM : Open file limit.

Take the value of the SYSGEN parameter CHANNELCNT and subtract 16. Set the FILLM to this value. FILLM should never go below the value of 128.

ASTLM : Asynchronous System Trap Limit.

If you expect your batch jobs to be the only process running at backup, set ASTLM to the maximum value allowed, ie: 32767. If other jobs may be running, you need to adjust ASTLM to a lower value only if you want the other jobs to continue getting work done in the area of I/O to the disk. Having ASTLM at the maximum won't lock out other jobs from doing I/O, it will just impact their throughput. If you want the other jobs to have some room to breath, perform the following command durring High periods of I/O processing :

$MONITOR MODE/AVERAGE

After you have monitored the system for a while under these high periods of I/O processing, look at the percentage under executive mode. This percentage is the value by which you want to decrease the maximum value of ASTLM 32767 by. For example, if you see the executive mode average at 15%, then take 15% of 32767 and subtract it from 32767. This should be your desired value for ASTLM.

DIOLM : Direct IO limit.

Whatever you choose for ASTLM, subtract 16 and set DIOLM to this value. If you ever change ASTLM, you should also change DIOLM accordingly. DIOLM should never go below the larger of 4096 or 3 times FILLM.

BYTLM : Byte limit.

This is a tricky one. Start out by setting it to the value that you determined to be the number of free bytes of non paged dynamic memory during typical loading divided by the maximum number of concurrent backup jobs you plan to run. The value for BYTLM should never go below the calculated value (8 * DIOLM) + (256 * FILLM).

BIOLM : Buffered IO limit

Set BIOLM equal to the value of FILLM. If you ever change FILLM, change BIOLM accordingly.

ENQLM : Lock End Queue Limit

Determine 2% of the value of FILLM. Take the lesser of this value and 256. Add this result to FILLM for the value of ENQLM. If you ever change FILLM, change ENQLM accordingly.


Setting up the BATCH Queue

Remember, changing the working set quota and extent in the authorization file is not enough. Make sure that the batch queue that you run your automated backup job in must also allow for those working set values. Make sure the queue's Working set quota and Working set extent are the same as in the authorization file. Because of this requirement it is advised that you have a special batch queue tailored for your backup and set at a desired base priority.


Troubleshooting Problems

After following the above steps, you may still run into problems with your automated BACKUP. It is important to look at the statistics at the bottom of the BACKUP log file, or monitor your BACKUP process, as well as check for any errors. If you do run into some problems, check out these troubleshooting tips below.

  1. If you get the error message :
    1. %BACKUP-F-ALLOCMEM, error allocating virtual memory
      -LIB-F-INSVIRMEM, insufficient virtual memory.

    then you need to increase your pagefile quota. If this value is already equal to the SYSGEN parameter VIRTUALPAGECNT, then you need to explore increasing VIRTUALPAGECNT. IF you need to increase VIRTUALPAGECNT, see my page on Tuning your VMS system.

  2. If you get the error message :
    1. %BACKUP-?-NOMSG, message number

    then the problem is most likely FILLM. If so you need to verify that FILLM is 16 less than the SYSGEN parameter CHANNELCNT. If you continue to get theis message, try lowering FILLM by 5% at a time never letting it go below 128. If you do change FILLM, be sure to change DIOLM, BYTLM, BIOLM, and ENQLM according to the methods above.

  3. If you get the error message :
    1. %BACKUP-F-NOMSG, Message number 00A386E4
      -SYSTEM-F-EXQUOTA, exceeded quota

    and BYTLM is not larger than the amount of free bytes in non-paged dynamic memory divided by the maximum number of concurrent backup jobs, then you need to decrease FILLM step by step (5% at a time), until the problem disappears. Make sure FILLM does not go below 128.

  4. If the BACKUP process consumes too much physical memory, reduce the value of WSQUOTA in steps, but leave WSEXTENT as determined above.
  5. If BACKUP seems to overload the IO capacity of any of the disks to the point that other users notice performance reduction accessing the disk, then reduce the value for ASTLM and DIOLM accordingly.
  6. If BACKUP seems to use up too much non paged dynamic memory such that other users notice that all other IO operations are degraded, then reduce BYTLM in steps, paying particular attention to the dependencies of FILLM and BIOLM above.

My Home Page | VMS Home

DCL | Utilities | Management | Tips

FORTRAN | Pascal

eMail Questions

Quiz?