Adding a new
Node to a
VMS Cluster

Here are some instructions for adding a new satellite node to a VMS Cluster. Some of these instructions are not reflected in the normal documentation, however I believe they make the addition of new nodes as well as the management of the new nodes much easier. Before you continue to add a node or nodes, first sit down and decide how each new node is to be configured into your cluster. For each node you should decide on these items :

1. The node name.
2. Which disk/system it is going to boot from.
3. The boot root number.
4. The DECNet address
5. It's disk & Tape allocation class (if necessary or desired*).

* Note : I recommend always choosing a tape and disk allocation class, regardless of the fact that your workstation will not be serving disks or tapes. Also, for clarity sake and ease of management, try and keep all of the numbers the same. For example : DECNet number 10.33, boot root [sys33], and disk and allocation class of 33.

Now before you start hooking up your new node to the network, you should take any license PAKs associated with this node, and add them to the license database. Be sure to follow the instructions on adding licenses. For every license that yo add that is specifically for this node, you should modify the license after entry to use the /INCLUDE=node name option, to make sure no other node grabs it during a reboot.

Once the hardware of the new VAXStation has been set up and attached to the appropriate network interface, you must first gather some information from the workstation. At the three carrot prompt enter in the command :

>>> SHOW ETHER

This will show you the Ethernet address. Write this address down. It will be a set of six , two-digit hexadecimal numbers separated by commas. For example : 08-00-2B-F8-7D-C1. Then Enter in the command :

>>> SHOW DEV

This will show you all the devices on the station including the Ethernet controller which is usually ESA0:, the tape drives (if any) starting with MK, and the disk drives, starting with DK. Typically there is at least one disk drive on workstations, which is to be used as a local page and swap disk. Sometimes the workstations we order have a secondary disk which is much larger for other storage requirements. Typically, the smaller of the two drives is designed for a local page and swap disk, and the larger is for other storage requirements.

Write down both disk addresses, and the address of any tape drive local to the workstation. Now it's time to go to the Cluster server node to add the new workstation. Before you do anything here you should have chosen the root number, the allocation classes, the DECNET address and name of your new workstation. The DECNET address consists of the area number and the node number. The area number must be the same as the area number of the boot node. Although it's not documented, as mentioned above, for clarity's sake it is best to assign an allocation class, root number and node number the same for any given workstation. Due to the restrictions on root numbers for workstations these numbers should all be two digits between 10 and 99. For example, on my cluster I number all 3000 class workstations from 30 to 39, and 4000 class workstations from 40 to 49, etc.

So if I give a workstation the DECNET address of 10.33, then it's allocation class for disk and tape are both 30, and it's root number is [sys30.].

Once you have determined the DECNET address you must also determine the SCS ID number. To do this take the area number (number before the decimal point), multiply it by 1024, then add the node number. So for example, with a DECNET address of 10.33, the SCS ID number would be (10 x 1024) + 33 or 10273.

Now it's time to add the new station to the cluster. On the boot server node (the one that has also booted off the same disk that you have chosen to be your boot disk), from the SYSTEM manager's account, run the cluster config command procedure as per the documentation. If you haven't read the documentation, do so now. The name of the appropriate manual is "Open VMS Cluster Systems" Sections 8.1 and 8.2 specifically. It will prompt you for all of the information you have just determined.

*** VERY IMPORTANT ***

When the procedure is executing, it will ask you : "Will a local disk be used for paging and swapping?" To this response, you must answer "NO", even if you are planning to use a local disk for paging and swapping. The reason is that if you answer "YES" to this, the disk and page and swap files are out of your control, and is determined by defaults for a workstation with minimal memory, and not what your workstation has.

Instead, you must always answer this question with a "NO". This will cause a temporary page and swap file to be created on the boot disk, which is fine for the time being.

At the end of the cluster config command procedure, you will be asked to boot the workstation. Go ahead and do this. The firs time the workstation boots, it will automatically generate a new MODPARAMS.DAT file in the specific root on the boot disk, and reboot using an autogen.

When the workstation comes up after the autogen, you still have work to do. First log on under the SYSTEM account from the server and initialize the the local page and swap disk with the following command :

$INIT/INDEX=BEG node name$diskaddr node name$PGSWP

substituting "node name" with the name you chose for the new workstation, and "diskaddr" with the device disk address such as DKA100. It is very important that you use the qualifier /INDEX=BEG or /INDEX=END so that the index file is placed at the beginning or end of the disk volume, because the default is to create it in the middle of the volume, which would fragment the page and/or swap file.

Now mount the disk using the command :

$MOUNT node name$diskaddr node name$PGSWP node name$PGSWP

Then do the command :

$SHOW DEVICE/FULL node name$PGSWP

and determine the number of free disk blocks on the device. Take this number and divide it by 3,and round down to the nearest 100 blocks. This number should be size of the SWAPFILE. Take the size of the SWAPFILE and multiply it by 2, and this should be the size of the PAGEFILE. Take the number of blocks in the PAGEFILE and divide it by 2000. This will yield the size in Megabytes. This number should be at least 8 times the size of main memory for the workstation. If it is not, then we should be using another larger disk for paging and swapping, or another disk in conjunction with this disk. If the size of the page file is greater than 8 times the memory, everything is fine.

Now create the page and swap files. First enter in the command :

$CREATE/DIR node name$PGSWP:[SYSEXE]/OWN=[SYSTEM]

Then activate SYSGEN and create the page and swap files :

$MCR SYSGEN
SYSGEN>CREATE/CONT/SIZE=pagefilesize node name$PGSWP:[SYSEXE]PAGEFILE.SYS
SYSGEN>CREATE/CONT/SIZE=swapfilesize node name$PGSWP:[SYSEXE]SWAPFILE.SYS
SYSGEN>EXIT

Now it's time to modify the command procedure that mounts the page and swap disks. To do this set your default to SYS$MANAGER and edit the command procedure that DEC provides called SYPGSWPFILES.COM. In that file, at the appropriate place, enter in these command lines :

$IF (F$GETSYI("NODENAME") .EQS. "node name")
$THEN
$    MOUNT/SYSTEM node name$diskaddr node name$PGSWP node name$PGSWP
$    MCR SYSGEN
INSTALL node name$PGSWP:[SYSEXE]PAGEFILE.SYS/PAGEFILE
INSTALL node name$PGSWP:[SYSEXE]SWAPFILE.SYS/SWAPFILE
EXIT
$ELSE
$ENDIF

Remember to substitute all lowercase letters with your "node name" and "diskaddr". Now go to the specific root on the system disk for this new node, ie:

$SET DEFAULT SYS$SYSDEVICE:[SYSnn.SYSEXE]

where "nn" is the root number for the node you are adding. Once there enter in the commands :

$RENAME PAGEFILE.SYS PAGEFILE.OLD
$RENAME SWAPFILE.SYS SWAPFILE.OLD

and then :

$EDIT MODPARAMS.DAT

and set up the desired SYSGEN parameters for this node. When this is completed, make all the changes needed for the system wide startup command procedure SYSTARTUP_VMS.COM for the new node, then autogen the new node by logging onto the SYSTEM account on the new node and entering the command :

$@SYS$UPDATE:AUTOGEN SAVPARAMS REBOOT

When the new node has rebooted, you can go to the specific [SYSEXE] directory and delete the temporary PAGEFILE.OLD and SWAPFILE.OLD files. Once this has been completed, you can configure DECNET as per the DECNET configuration instructions. (Remember : If you use NETCONFIG.COM never ever answer "YES" to the prompt "Do you want to run these commands now". It will screw up your network objects. Instead, read my pages on "The Perils of using NETCONFIG.COM")

If you have any questions or problems with the above procedure, please feel free to eMail me with the specifics of your problem or question.