SOFTWARE RELEASE DOCUMENT ---------------------------------------------------------------------------- Product Name Integrity XC System NonStop Clusters for SCO UnixWare Product Number T5649D10IPM20 Part Number 421040-001 Softdoc Date 1 December 1998 ---------------------------------------------------------------------------- Release Summary This release of NonStop Clusters® for SCO® UnixWare® is an update to the D10IPM10 release of the Tandem® single system image (SSI) UNIX® clustering technology. This release includes features such as: * Cluster-wide (SSI): o Access to storage devices o Access to other system resources o Swap/paging service o System V IPC, devices, pipes o STREAMS access o Parallel networking o Cluster time synchronization o Distributed processes * Journaled file system * Disk mirroring using the Compaq SMART-2DH controller * File system failover * Application restart and failover Product Table of Contents This document contains descriptions of the following product components. Comp T Number Product Component Comp T Number Product Component T5649D10IPM20 NonStop Clusters Superseded IPMs This release supersedes the D10IPM10 release of NonStop Clusters. Requisite IPMs None Required Hardware None Required Firmware Version TPN 131790 A03-03 of the Tandem ServerNet SAN Switch. Required Software T5649D10 NonStop Clusters for SCO UnixWare operating system. T5649D10IPM10 NonStop Clusters for SCO UnixWare operating system. T5649D10IPM20 NonStop Clusters for SCO UnixWare operating system. SCO UnixWare 2.1.2 (supplied with the original factory shipment) Required Disk Space Minimum 2 GB Manuals The manuals associated with this product are defined in the following table. Part Number Manual Title 420938-001 Integrity XC System NonStop Clusters for SCO UnixWare System Administrator's Guide 420941-001 Integrity XC System Site Preparation and Hardware Installation 421180-001 Documentation Notes for Integrity XC System NonStop Clusters for SCO UnixWare Administrators Guide Migration Considerations None Installation Considerations Refer to this softdoc for D10IPM20 installation instructions, and to the "Known Problems" section of this softdoc for installation considerations. General Notes 1. An on-line version of this softdoc will be updated as new information is available about the product. The current softdoc for this product is viewable on the World Wide Web and is accessed from a link on the softdoc index page: https://www.austx.tandem.com/softdocs/ These pages are password-protected. At the prompts, enter "Tandem" for the userID and "!ntegr1ty" for the password. The server is case-sensitive, so note the uppercase T in the userID and the exclamation point and number 1 in the password. Trademarks Tandem, Integrity, NonStop, NonStop-UX, and the Tandem logo are trademarks or registered trademarks of Tandem Computers Incorporated in the United States and/or other countries. Oracle is a registered trademark, and Oracle 7 is a trademark of Oracle Corporation. Compaq, ProLiant, and the Compaq logo are trademarks or registered trademarks of Compaq Computers Incorporated in the United States and/or other countries. SCO, The Santa Cruz Operation, UnixWare, and the SCO logo are trademarks or registered trademarks of Santa Cruz Operation Incorporated in the United States and/or other countries. INFORMIX is a registered trademark and ONLINE Dynamic Server is a trademark of Informix Software Incorporated. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Limited. NFS is a registered trademark of Sun Microsystems, Inc. Veritas, VxFS, and VxVA are registered trademarks of Veritas Software Corporation. UniSoft is a registered trademark of UniSoft Corporation. All other brand and product names are trademarks or registered trademarks of their respective companies. Copyright Notice Copyright (c) 1998, Tandem Computers Incorporated. All rights reserved. The computer program listings, specifications and documentation herein are the property of Tandem Computers Incorporated or a third party supplier and shall not be reproduced, copied, disclosed, or used in whole or in part for any reason without the prior express written permission of Tandem Computers Incorporated. ---------------------------------------------------------------------------- Product Component: NonStop Clusters - T5649D10IPM20 ---------------------------------------------------------------------------- Description This is the D10IPM20 release of the NonStop Clusters for SCO UnixWare operating system for Compaq ProLiant Intel based PCs of the single-system image, distributed operating system. Contents The following packages belong to this component: Package Version Package Description D10IPM20 D10IPM20 NonStop Clusters software nscman D10IPM20 NonStop Clusters manpages D10I20doc D10IPM20 NonStop Clusters software release document Notes 1. Included in this release is a document entitled Documentation Notes for the NonStop Clusters Administrator's Guide. This document contains information that was unavailable when the manual was printed. Refer to the Documentation Notes for the latest changes to the installation procedures in Chapter 3 of the manual and the recovery procedures in Appendix B of the manual. 2. D10IPM20 includes an Increased Reliable Disk Capacity Feature. This feature increases the maximum disk capacity of NonStop Clusters for UnixWare. Maximum disk capacity is increased by a factor of 3 by adding support for multiple reliable disk subsystems. Previous releases of NonStop Clusters supported a single reliable disk subsystem that provided disk and file system failover between the primary and backup root nodes. This feature allows the configuration and management of two additional reliable disk subsystems connected between pairs of dependent nodes. The additional reliable disk subsystems will provide the equivalent functionality for disk and file system failover between dependent nodes that was previously provided between primary and backup root nodes. More information on this new feature is described in the Integrity XC System Site Preparation and Hardware Installation manual in Appendix B, System Upgrades. The default system configuration provides sufficient memory resources to perform filesystem failover under most conditions. However, it is possible for filesystem failover to fail to complete due to insufficient memory and thus hang the cluster. This condition can be avoided by increasing the DESFREEBYTES and LOTSFREEBYTES tunables sufficiently to ensure the system has adequate memory to perform the filesystem failovers. A reasonable rule of thumb is to add 30KB per failover-capable filesystem to the current values of DESFREEBYTES and LOTSFREEBYTES. 3. Real time applications are not supported in this release. They may cause the cluster to hang. 4. Mounting a filesystem with the chard or charddup mount options may display the following message on the system console when there isn't a node available to provide the backup cache for that filesystem. Message: WARNING: Error 11 mounting backup cache vfsp 0xXXXXXXXX This message commonly appears when booting the root node with the tnc_rootopt bootstrap parameter set to 0x84000 (chard,intr) or 0x104000 (charddup,intr). This message also appears when mounting any filesystem with the chard or charddup options and there are no backup cache nodes available for the filesystem (either none were specified with the bkup mount option or those specified have not joined the cluster). As specified in the mount(1m) manual page, the lack of a backup cache for a chard or charddup-mounted filesystem may result in writes to that filesystem being performed synchronously to disk to provide the level of data integrity associated with the respective mount option. If/when a node joins the cluster to provide the backup cache for the chard or charddup-mounted filesystem, writes to that filesystem will no longer be forced to be synchronous to disk. For example, if the root node is mounted with chard or charddup, writes may be performed synchronously to disk until the secondary root node joins the cluster to provide the backup cache for the root filesystem. 5. The following INFORMIX OnPerf menu options are not supported in this release: o physical processor metrics o physical processor tool This is a result of differences between UnixWare and NonStop Clusters. The current workaround is to use the sar utility. Also, the OnPerf utility may give the following error: Couldn't open data file /var/adm/metreg.data This error is also a result of the differences between UnixWare and NonStop Clusters. The current workaround is to start OnPerf with standard error messages routed to "/dev/null". No fix is currently planned for the preceding problems. OnPerf will report proper performance information on INFORMIX metrics. To obtain machine specific performance information, use the sar utility. 6. Two tunable parameters that are associated with system panic dump generation are still applicable for the D10IPM20 release. SYSDUMP_SELECTIVE: Flag value that determines if the system will dump all pages (0), or selectively dump only pages mapped in the kernel (1). The default value is 1: to generate a selective dump, occupying much less space, rarely losing information. SYSDUMP_POLL: Flag value that determines if the system will dump relying upon the disk interrupt (0), or poll for I/O completion (1). The default value is 1: to poll with disk interrupt disabled, in case the interrupt is shared with other devices which might stop the dump. 7. Software patches, or PTFs, from SCO for UnixWare are not supported with NonStop Clusters. NonStop Clusters must be installed over UnixWare 2.1.2. No SCO PTFs are to be installed before or after NonStop Clusters is installed. 8. The Site Preparation and Hardware Installation manual describes a Japanese PDU that may not be available at D10IPM20. A North American low-voltage PDU will be substituted until the Japanese PDU is available for shipment. The following known problems have workarounds: 1. Boot loop resolution needed on base SCO UnixWare installation (PTS 35084). Symptom: System attempts to rebuild and reboot continuously after system installation. Workaround: The following method can be used to resolve this situation should it occur. The following message will repeatedly appear on the console screen immediately after a system installation or reconfiguration: The UNIX Operating System kernel will now be rebuilt to incorporate recent configuration changes. Press the ENTER key when ready or the ESC key to stop. You should press the ESC key. At the prompt, enter the command "# resmgr". Examine the output to find the following entries: 19 cpqsc 0 5 4 10 7000 70ff 40101000 401010ff - - 4 0x1000000F 0 - 1 21 ida 2 5 4 5 8000 80ff c8000 cbfff - - 4 0xE11AE10 - - 2 22 ide 2 5 3 14 1f0 1ff 10000 10001 -1 - 0 - - - - NOTE: The key numbers (the first field) may not match the preceding example. The third field (unit number) is conflicting. The unit numbers are incorrect and must be manually changed to the following correct values. KEY UNIT UNIT#. cpqsc 0 ... ida 2 ... ide 1... The first field is the "KEY". Using the appropriate KEY, modify the appropriate unit number entry to match the preceding values. In this example, key is 22 and ide is changed from unit 2 to unit 1. resmgr -k 22 -p "UNIT" -v "1" Commit the configuration changes by entering the command "/etc/conf/bin/idconfupdate -f". Reboot the system and allow it to rebuild the UNIX operating environment. 2. Unable to copy wild card files from a mounted file system on a CM-1300(PTS 34195). Symptom: When trying to copy several files using the wild card symbol, the files do not copy correctly or it seems to "hang" and not copy anything. This only occurs when using ksh while running on a file system NFS mounted from a CM-1300. It works correctly on all other platforms. Workaround: Do not attempt file locking (for example, use csh) when accessing files NFS mounted from a CM-1300. 3. keepalive has no interface for "shutdown" scripts. (PTS 36470). Symptom: Currently, keepalive's only interface to tell a process to do a controlled shutdown is via a SIGTERM. This does not work well with processes that should be shut down another way,(e.g., lpsched). Workaround: Unregister the process with keepalive. Then use whatever shutdown procedure is appropriate for the process. 4. /etc/rc2.d/S51nis file missing domain name when created by "pkgadd nis" from SCO CDROM (PTS 36525). Symptom: Domain name will not have a valid entry. Workaround: In the file "/etc/rc2.d/S51nis", change the line: domain to: domain where is the internet domain name for the machine. 5. Installing NIS after NonStop Clusters overwrites "/etc/init.d/nis". This results in the NIS daemons no longer being registered with keepalive (PTS 36767). Symptom: NonStop Clusters does not install NIS by default. When pkgadd is used to install NIS from the SCO 2.1.2 CD, it replaces the Tandem version of the file "/etc/init.d/nis". As a result, all the changes that are required for spawndaemon are lost and NIS does not failover. Workaround: 1. Copy the file "/etc/init.d/nis" to "/tmp/nis" before installing NIS. 2. Install NIS from the SCO CD on each cluster. Do not configure NIS when it is installed, as that will be overwritten by step 3. 3. Copy the file saved in "/tmp/nis" to "/etc/init.d/nis" to each cluster. 4. Run "ypinit" with the correct option to configure each cluster. 6. System will not boot when one RAID cabinet is powered off (PTS 36770). Symptom: When one of the disk cabinets is powered off at boot time, and the F1 prompt is disabled, the entire disk array is disabled causing the system boot to fail. Workaround: The following configuration changes will enable the F1 prompt, change the recovery server option status, and prevent this problem. If you make the following configuration changes, it will be necessary to type F1 during boot to allow the cluster to boot when one RAID cabinet is powered off. However, in many cases, this is preferable to failing to boot at all. 1. Shutdown the cluster. 2. Power off all nodes. 3. Power on node 1, select F10 during the boot sequence to execute utilities from the System Partition. 4. At the "Main Menu", type to select "Advanced Mode". 5. At the "Main Menu", select System Configuration. 6. From the "System Configuration" menu, select Configure Hardware. 7. From the Configure Hardware menu, select Review or Modify Hardware Settings. 8. From the next menu, select "View and Edit Details". 9. Under "Slot? - Compaq SMART-?? Array Controller", highlight "Online Recovery Server Status". If "Primary" is selected, press , select "Recovery", and press . Note: If you get the "Write Cache Disable Warning" window, then ignore it and press . 10. Under "Advance Feature", find the entry "F1 boot prompt", if disabled, press , and select , then press . 11. Press F10 to continue. 12. Back at the previous menu, select "Save and exit". 13. Select "Save the configuration and restart the computer". 14. Press to reboot. 15. Power on the other nodes in your cluster. 7. Token deadlock in forced unmount (PTS 37167). Symptom: Under rare conditions, if you do a "umount -o force" on a running file system, it can hang indefinitely. Workaround: Do not forcibly unmount running file systems. 8. force unmount of NFS FS with yet-to-be-noticed downed server loops forever in cfs_hide_cnodes (PTS 37238). Symptom: When trying to unmount a file system using "umount -o force", a downed server may hang. Workaround: Do not force an unmount with a downed NFS server. 9. Closes done from process exit on downed hardmounts are non-interruptible (PTS 38167). Symptom: Process with files on a downed hardmount cannot be killed. Workaround: Remount or forcibly unmount the file system. 10. Unshare of downed file system hangs system shutdown (PTS 38283). Symptom: Shutdown hangs on a system that has a downed, shared, hardmount file system. Workaround: Use "umount -o force" to bring the file system down. 11. NonStop Clusters OS: unexpected defaults while running: console keyboard and mouse lockup on UnixWare 2.1.1 upgrade. Customer reported defect. (PTS 38293). Symptom: On some platforms, after adding the UnixWare 2.1.1 update package (update211), the local keyboard and mouse will not respond after rebooting the new kernel. This defect corrects itself after installing the UnixWare 2.1.2 package. Workaround: The user can telnet into the system, continue to pkgadd the update212 package, and reboot the system. Alternatively, if networking is not setup or otherwise unavailable, it may be possible to boot a 2.1.0 kernel (saved as unix.old during the 2.1.1 upgrade) and perform the 2.1.2 pkgadd using the old kernel. 12. new_rem_swapfile() can deadlock with tnc_anon_resv_adjust() over registrar locks (PTS 38390). Symptom: If the cluster has been running under a significant memory load, the root node appears "hung". Workaround: Ensure there is always sufficient swap space on each node to satisfy the peak per-node memory load. 13. UX:init: ERROR: Command is respawning too rapidly. Check for possible errors. id: co "/usr/lib/saf/ttymon -g -v -p "Console Login: " -d /dev/console.3 -l console" (PTS 38407). Symptom: After adding the D10IPM20 package, the following message may appear on the primary node's console prior to rebooting the node: UX:init: ERROR: Command is respawning too rapidly. Check for possible errors. id: kp "/sbin/keepalive -t 10 -P 1 -S 2" Workaround: If this message should occur, remove any core files in the / (root) directory which may have been created during the time the preceding messages was displayed. The message itself can be ignored. 14. cnet Adapter Check causes the whole cluster to hang (PTS 38666). Symptom: When installing the Oracle package, mounting the Oracle distribution CD on one node and the Oracle install scripts are executed from a different node, the following error may occur: WARNING: cnet: unit 0: Adapter Check interrupt 100002! Interface 0 in slot 16 has failed. Attempting to restart. When this happens, the cluster may hang. Workaround: Ensure the Oracle CD is mounted on the node 1 CD-ROM drive and that the install scripts are executed on node 1. To ensure the Oracle install scripts are executed on node 1, use the following onnode commands: onnode -p 1 ./start.sh onnode -p 1 ./orainst /c This may happen with other applications that access the CD-ROM drive heavily. 15. Running out of kernel virtual can cause poolrefresh to spin on cpu (hangs UP systems) (PTS 39941). Symptom: Backup cache node is a uniprocessor system and is hung after heavy activity on a chard or charddup-mounted filesystem. Workaround: Increase the SEGKMEM_PERCENT tunable. Default is 55. Try increasing by about 50%, e.g. try 80. 16. Receive error message when booting the dependent node from the boot floppy. Currently not in the PTS database. Symptom: When booting from the Secondary Boot Diskette, you receive the error message, "/sbin/metreg: getksym error = 154". Workaround: This is normal behavior. The message only occurs when you are installing the node from the boot floppy. Just ignore the message. New Features The following new features are supported with the NonStop Clusters D10IPM20 release: * ProLiant 1850R/400 support. * ProLiant 5500R/400 support. * ServerNet SAN online diagnostics. * Additional Disk Capacity Problems Corrected 1. LCCrq05483:auditrpt cannot read rfork/migrate/rexec entries into the log file. (PTS 31775, No TPR, Severity 2) 2. Standalone build is broken cause uts/io/tdmpci/pci.h overwrites uts/io/autoconf/ca/pci/pci.h. (PTS 32887, No Tpr, Severity 2) 3. Interface direct routes are not handled correctly. (PTS 33464, No TPr, Severity 2) 4. CLMS needs node transition vector support. (PTS 35621, No TPR, Severity 3) 5. TCP hangs on inp_rwlck while running VSTS test. (PTS 36081, No TPR, Severity 2) 6. pdimkdev makes devices with controller number 65534. (PTS 36203, No TPR, Severity 2) 7. Spurious "too many nodes" reboot from licensing support code. (PTS 36204, No TPR, Severity 3) 8. pkgadd of tcpip_nis.so fails. i.VUF script needs work? (PTS 36904, No TPR, Severity 2) 9. Panic in tcp_respond(). (PTS 37117, No TPR, Severity 2) 10. ics_msg_ack_done() kernel panic after a node goes down. (PTS 37170, No TPR, Severity 3) 11. Either "migrate(1)" is not moving processes, or "ps(1)" is not correctly reporting them with the -N option. (PTS 37319, No TPR, Severity 1) 12. inode table overflow on CD file system. (PTS 37509, No TPR, Severity 2) 13. Panic in bcopy() because of a bad length field in an LWP audit buffer. Customer reported defect. (PTS 37566, No TPR, Severity 3) 14. VPROC nodedown hung trying to remove pgrp from nonexistent session. (PTS 37711, No TPR, Severity 3) 15. Hang during nodedown processing after failover - semconv() spinning on lock. (PTS 37721, No TPR, Severity 3) 16. /etc/dfs/sharetab seems to get corrupted, causes mount access denial. (PTS 37785, No TPR, Severity 2) 17. Diskadd command is broken. (PTS 37993, No TPR, Severity 2) 18. ASSERTION CAUGHT: ioflag == CPFS_IO_NONE, file: cfs_vm_osi.c, line: 1139. (PTS 37995, No TPR, Severity 2) 19. Downgrading tokens at cli@svr doesn't clean backup cache anymore. (PTS 37997, No TPR, Severity 2) 20. Implement Parallel NeRD performance changes. (PTS 38001, No TPR, Severity 2) 21. fuser hangs in uts_fusers() on a down filesystem. (PTS 38013, No TPR, Severity 2) 22. cfsbc_sync_hplist() needs to handle trylock failure of svrcfstok's stateloc. (PTS 38075, No TPR, Severity 2) 23. Normal cfs pageouts should not set critical memory priority. (PTS 38099, No TPR, Severity 2) 24. HANFS: mountd can hang on down CFS HARD mount preventing new NFS mount/unmount/showmount. (PTS 38101, No TPR, Severity 2) 25. EXNFS DEBUG: ASSERTION CAUGHT: vfsp->vfs_count > 0, file: cfs_chard.c, line: 569. (PTS 38126, No TPR, Severity 3) 26. NFS mount can deadlock filesystem failover. (PTS 38129, No TPR, Severity 2) 27. Sleeping operations on down hardmounts now woken up when forcibly unmounted. (PTS 38165, No TPR, Severity 2) 28. NonStop Clusters OS: abending while running: open(2) causes cluster crash. Customer reported defect. (PTS 38247, TPR # 980626-1633-16640, Severity 2) 29. EXNFS client: ASSERTION CAUGHT: i < MAXIDS, file: cfs_hm.c, line: 1167. (PTS 38249, No TPR, Severity 3) 30. ASSERTION CAUGHT: !swappedin, file: rwsleep.c, line: 1257. (PTS 38253, No TPR, Severity 2) 31. Keepalive and Spawndaemon do not process partially read and written messages. (PTS 38304, No TPR, Severity 2) 32. EX NFS client/server: statd restart is broken. (PTS 38465, No TPR, Severity 2) 33. mountd needs to be added to the package for D20. (PTS 38468, No TPR, Severity 1) 34. Double failure of root server and root backup cache node deadlocks root failover. (PTS 38485, No TPR, Severity 2) 35. Cluster20 hung while serving as an IP server for netbench, cthon and a peer in the tcp_test. I was able to log in, but never received a prompt. All three nodes were effected. (PTS 38656, No TPR, Severity 3) 36. Clms can send rpc to itself while its taking over as the new clms master. (PTS 38679, No TPR, Severity 2) 37. EXNFS client: new CLIENT field ch_addr not set by clget() (no port info to CFS client). (PTS 38685, No TPR, Severity 2) 38. Assertion caught: getpl() == PLBASE, file ublock.c, line 276. (PTS 38696, No TPR, Severity 2) 39. PANIC: kernel-mode address fault on user address 0x4 (cfs_blkrsrv_giveback_svr). (PTS 38730, No TPR, Severity 2) 40. NFS panic in exportfs(). (PTS 38791, No TPR, Severity 2) 41. efork() hangs when fork() returns EAGAIN. (PTS 38794, No TPR, Severity 2) 42. Inconsistent return values from fsck after failover. (PTS 38801, No TPR, Severity 2) 43. cmount -p option doesn't wait for mounts to complete. (PTS 38880, No TPR, Severity 2) 44. svrcfstok_policy() can deadlock failover. (PTS 38887, No TPR, Severity 2) 45. NOTICE: ASSERTION CAUGHT: !(cp->c_flags & CHIDDEN), file: cfs_subr.c, line: 801. (PTS 38901, No TPR, Severity 2) 46. EX NFS client: Not enough XIDs allocated for writes when NFS code does read-to-do-write. (PTS 38904, No TPR, Severity 2) 47. Panic caused by failing over during mmap of MANDLOCK file. (PTS 38927, No TPR, Severity 2) 48. SIOCGIFALL doesn't return all interfaces because ifnet's not sorted by if_index properly. (PTS 38952, No TPR, Severity 2) 49. Intra-cluster TCP connections are not reset when one endpoint is lost and connection idle. (PTS 38953, No TPR, Severity 2) 50. crfs_open_0() has memory leak. (PTS 39015, No TPR) 51. Retry loop (added via TDM_BUG_FIX) in nfs_async_start() panics due to undone buf_t. (PTS 39018, No TPR, Severity 2) 52. Part of Accurate Idle facility omitted from Package Added kernels. (PTS 39025, No TPR, Severity 2) 53. kmem_alloc() panic during vproc nodedown. (PTS 39072, No TPR, Severity 2) 54. SPAD needs to be modified to support VIA emulator. (PTS 39079, No TPR, Severity 2) 55. Error handling when removing local disk from cluster needs to be improved. Customer reported defect. (PTS 39102, No TPR, Severity 3) 56. EXNFS client: PANIC: svrgivebacknode: Specified agent NOT found! (PTS 39115, No TPR, Severity 3) 57. SPAD should use the standard PCI interface and support autoconfig. (PTS 39130, No TPR, Severity 2) 58. NOTICE: ASSERTION CAUGHT: "%eax > $_A_PLBASE", file: intr_p.s, assertion #: 3. (PTS 39160, No TPR, Severity 2) 59. Top of tree pkgadd fails. (PTS 39168, No TPR, Severity 3) 60. Window in use of clms_get_cluster_num_nodes() following clms_get_tnc_nodelist(). (PTS 39206, No TPR, Severity 2) 61. Function aadReadConfigFile() must need configuration parameters for the resource acquisition program. (PTS 39221, No TPR, Severity 3) 62. tcp_output() fastpath logic needs to use loopback for local addresses. (PTS 39226, No TPR, Severity 2) 63. disksetup w/surface analysis on dependent node causes root node panic. (PTS 39227, No TPR, Severity 3) 64. Ifdef test around include files incorrect in spad.h - breaks build. (PTS 39228, No TPR, Severity 3) 65. Which node the cluster IP address is on is not guaranteed. Customer reported defect. (PTS 39239, No TPR, Severity 2) 66. The inet daemon does not restart after a failover. Customer reported defect. (PTS 39241, No TPR, Severity 2) 67. If route.d is running the node routing tables get messed up. Customer reported defect. (PTS 39244, No TPR, Severity 2) 68. swap -d /dev/swap. fails. (PTS 39258, No TPR, Severity 2) 69. Need to check whether the resource acquisition script is running before trying to kill it. (PTS 39278, No TPR, Severity 2) 70. Panic in strclose() when strelwp_findhash() returns NULL. (PTS 39331, No TPR, Severity 2) 71. Keepalive doesn't restart PPM when it is killed. (PTS 39334, No TPR, Severity 2) 72. Init must special-case its own node to avoid running generic nodeup there. (PTS 39343, No TPR, Severity 3) 73. When recovering a query session (through FDS) if the fd cannot be retrieved the PPM still tries to build a session around the bogus fd. (PTS 39345, No TPr, Severity 2) 74. The 'tnc' rc script wasn't checked in. (PTS 39347, No TPR, Severity 3) 75. Remove sequencing from backup cache support. (PTS 39349, No TPR, Severity 2) 76. rdev_size() local path broken. (PTS 39373, No TPr, Severity 2) 77. Cmount remounts file system incorrectly when failing over from an even to an odd node. (PTS 39385, No TPR, Severity 2) 78. Nodedown hung waiting for vproc slave daemon processing that was throttled due to low memory. (PTS 39405, No TPR, Severity 2) 79. CLMS doesn't handle CLUSTERNODE_SHUTDOWN state properly. (PTS 39417, No TPR, Severity 3) 80. Init respawns ttymon on down node. (PTS 39424, No TPR, Severity 3) 81. NonStop Clusters D10 - should cvip be used as default sending ip address in outgoing ippackets? Customer reported defect. (PTS 39457, TPR # 980929-754-16102, Severity 2) 82. Code-inspection-induced CLMS change breaks failover. (PTS 39474, No TPR, Severity 3) 83. Spawndaemon gets a 'Segmentation Fault - core dumped'. (PTS 39481, No TPR, Severity 2) 84. CLMS tells clients about key secondaries early, but doesn't tell the secondaries. (PTS 39482, No TPR, Severity 3) 85. Processes added to keepalive are not always registered permanently. (PTS 39483, No TPR, Severity 2) 86. Race in TIOCPGRP processing in strioctl(). (PTS 39485, No TPR, Severity 2) 87. clms_ready_key_nodes() skips secondaries. (PTS 39491, No TPR, Severity 3) 88. Race in nodeup of secondary key server node. (PTS 39506, No TPR, Severity 3) 89. Informing key nodes from master nodeup may race with key node join. (PTS 39523, No TPR, Severity 2) 90. After failover ipcs indicates that SHM segments on surviving node are gone. Customer reported defect. (PTS 39539, No TPR, Severity 2) 91. CLMS cluster statistics not maintained properly by API support. (PTS 39544, No TPR, Severity 2) 92. "arscli -o watch" does not register PPMs with Keepalive using a node set. (PTS 39553, No TPR, Severity 2) 93. ASSERTION CAUGHT: ATOMIC_INT_READ(&tp->t_inpcb->inp_qref) > 0, file: tcp_subr.c, line 258. (PTS 39581, No TPr, Severity 2) 94. Keepalive's mmapped file got corrupted. (PTS 39586, No TPR, Severity 2) 95. Panic in ip_control() SIOCSIFFLAGS route handling. (PTS 39590, No TPR, Severity 2) 96. NonStop Clusters OS: abending while running: panicstr: readmem error: user address for kernel process. Customer reported defect. (PTS 39594, PR # 981021-712-16640, Severity 3) 97. BASE EX NFS: Getting cli EDEADLK when svr lockd unavailable (problem for failover. (PTS 39603, No TPR, Severity 2) 98. Onnode fails in nodedown/node.generic when recovery node is still coming up. (PTS 39608, No TPR, Severity 2) 99. CLMS routines rclms_api_newstate() and rclms_send_nodeup_info() can deadlock. (PTS 39630, No TPR, Severity 2) 100. Keepalive: with -i option can fail because it mistakenly thinks another keepalive present. (PTS 39631, No TPR, Severity 2) 101. Keepalive calls map_ka_data() with data_fd variable uninitialized. (PTS 39632, No TPR, Severity 2) 102. KEEPALIVE: verify_ka_not_running() doesn't check current pid like find_other_ka(). (PTS 39633, No TPR, Severity 2) 103. Racing nodedowns can hang init. (PTS 39635, No TPR, Severity 3) 104. Make file in the PPM directory does not remake the PPM when a file gets changed. (PTS 39668, No TPR, Severity 2) 105. Keepalive doesn't reopen syslog after creating primary Keepalive. (PTS 39670, No TPR, Severity 2) 106. The CLMS master doesn't send nodedown messages to all the nodes it should. (PTS 39672, No TPR, Severity 3) 107. Long delay in ics_llnodeup() causes other nodeups to fail. (PTS 39673, No TPR, Severity 3) 108. rpcbind warmstart (-w) with restarting mountd gives errors to client. (PTS 39716, No TPR, Severity 2) 109. The ics_nodedown() in clms_client_startup_node() races with rclms_rgp_add_node(). (PTS 39725, No TPR, Severity 2) 110. clms_get_up_nodelist() needs to get HALF_DOWN nodes, too. (PTS 39726, No TPR, Severity 2) 111. PANIC: kernel-mode address fault on user address 0x0 -> in cfsbc_copytobuf(). (PTS 39730, No TPR, Severity 2) 112. Unisoft: Touch command failed to change(touch) date. (PTS 39745, No TPR, Severity 2) 113. Deadlock when spawning processes early in nodeup. (PTS 39749, No TPR, Severity 2) 114. BASE BUG: panic in do_ERROR() as called by free_zero() because of q->q_ptr == NULL. (PTS 39753, No TPR, Severity 2) 115. Init doesn't shoot coming-up nodes properly after failover. (PTS 39796, No TPR, Severity 2) 116. CLMS and the API's system call doesn't support changing state to KCLEANUP. (PTS 39797, No TPR, Severity 2) 117. Keepalive needs to support unregistration by daemon name and argument list. (PTS 39815, No TPR, Severity 2) 118. Need to make cfs caching policy tunable (cfs_delay_writes). (PTS 39839, No TPR, Severity 2) 119. CLMS client doesn't check for wrap of trans hist when getting it from the master. (PTS 39854, No TPR, Severity 3) 120. Multiple open/single write in charddup,bkup failure after writes complete results in error. (PTS 39875, No TPR, Severity 2) 121. Signal forwarding could result in NO_JOB_FORWARD ops stopping and hanging failover. (PTS 39908, No TPR, Severity 2) 122. Comparison takes precedence over bitwise-and near line 823 in spawndaemon.c. (PTS 39911, No TPR, Severity 2) 123. cspecvp() can loop forever on a down softmount. (PTS 39946, No TPR, Severity 3) 124. backup cache unmount needs to wait for in-progress file syncs. (PTS 39949, No TPR, Severity 2) 125. Window for race in job control handling in issig() vs. cancel_jobctrl(). (PTS 39968, No TPR, Severity 2) 126. loop/hang on dependent node failover test. (PTS 39990, No TPR, Severity 3) Known Problems Remaining 1. Stale RPC delivered after a nodedown. (PTS 36062, No TPR, Severity 2) 2. Races in forced unmount need to be examined. (PTS 36343, No TPR, Severity 2) Some issues are present where forced unmount of a file system can panic. Most likely, NFS would have problems. 3. select() supposed to time out in 5 seconds but does not return for over 1 minute. (PTS 36401, No TPR, Severity 2) 4. Stale cshandle after failover. (PTS 36457, No TPR, Severity 2) 5. /tmp, /var/tmp, /usr, and /var should all be hard mounted separate file systems. (PTS 36659, No TPR, Severity 2) 6. The regroup function rgp_select_cluster() should factor in the importance of root-fs nodes. (PTS 36830, No TPR, Severity 2) 7. cli_ipcname_getid() has failover race; CMS has a similar problem. (PTS 37157, No TPR, Severity 2) Currently under investigation. 8. Token deadlock in forced unmount. (PTS 37167, No TPR, Severity 2) 9. Force unmount of NFS FS with yet-to-be-noticed down server loops forever in cfs_hide_cnode. (PTS 37238, No TPR, Severity 2) 10. memfs may report out of space due to the way dswap usage is calculated. (PTS 37246, No TPR, Severity 2) There must be at least as much total swap in the cluster as there is total RAM in the cluster. Otherwise, it is possible that shared memory segments cannot be created and memfs file systems may report out-of-space. 11. Substantial memory leak found in ics/llics after performing nodedowns. (PTS 37262, No TPR, Severity 2) 12. PANIC in cfs_blkrsrv_svrnodedown(). (PTS 37292, No TPR, Severity 2) Currently under investigation. 13. Verify that if idbuild fails on shutdown that idcpunix does not propagate kernel. (PTS 37408, No TPR, Severity 2) 14. Panic in prreadobjdir after failover. (PTS 37528, No TPR, Severity 2) Under certain rare conditions during failover, the node taking over root/clms/init responsibility panics with the following: 00: PANIC: kernel-mode address fault Crash dump analysis will indicate the faulting thread is in the function prreadobjectdir(). Currently, there is no known workaround. 15. initprivs/filepriv are not distributed. (PTS 37533, No TPR, Severity 2) 16. NFS client needs to present itself as CVIP for all client messages. (PTS 37642, No TPR, Severity 2) 17. Kernel must reboot automatically for unattended operation. (PTS 37795, No TPR, Severity 2) 18. XDM error log fills /tmp. (PTS 37797, No TPR, Severity 2) 19. PANIC: kernel-mode address fault on user address 0x100. (PTS 37918, No TPR, Severity 3) 20. DEADLOCK: CMS locking must be fixed to prevent nodedown deadlock. (PTS 37930, No TPR, Severity 2) Possible deadlocks exist in the CMS code during nodedown. There is nothing to distinguish these from other possible deadlocks in nodedown. 21. UX:nfs mount: ERROR: get_addr: (host cluster20, program 100003, version 2) is not found on any transport. (PTS 37966, No TPR, Severity 2) Currently under investigation. 22. Need to fix sorting of routes in rtrequest(). (PTS 37985, No TPR, Severity 2) 23. NFS code is cfs_rb_dolocks() creates a bogus cnode. (PTS 37990, No TPR, Severity 2) Currently under investigation. 24. EXNFS: A dirty page from CFS as NFS client causes cfs_data_downgrade loop. (PTS 38002, No TPR, Severity 2) This defect will be seen as "hung" processes. Currently, wait for the NFS server to come back up. The processes will no longer be "hung". Even if NFS is not mounted with "intr", it is rare that the processes will not be interruptible. 25. Failover clusters need CVIP interfaces. (PTS 38016, No TPR, Severity 2) 26. HANFS: Backup locks not rebuilt when backup lock node joins cluster. (PTS 38025, No TPR, Severity 2) The user will not be able to detect this defect. This occurs if node 1 fails after node 2 has failed and rejoined the cluster, and if file locks are being backed up. 27. PANIC: Range tokens do not handle 2 GB or higher values. (PTS 38177, No TPR, Severity 2) This occurs because no single file or disk partition can be larger that 2 GB. Do not use the "dd" command to copy a disk partition larger than 2 GB. 28. ASSERTION CAUGHT: !swappedin, file: rwsleep.c, line: 1257. (PTS 38253, No TPR, Severity 2) This defect is corrected in the source base, but is not part of the D10 IPM20 release. 29. Unshare of down filesystem hangs system shutdown. (PTS 38283, No TPR, Severity 2) 30. EX NFS client: Node discovery of NFS mount with down NFS server can loop. (PTS 38359, No TPR, Severity 2) 31. new_rem_swapfile() can deadlock with tnc_anon_resv_adjust() over registrar locks. (PTS 38390, No TPR, Severity 3) This can occur on the root node when it exhausts local swap and attempts to access swap on other nodes. Non-root nodes will not encounter this deadlock. 32. EX NFS server: sharing of chard mount on non-lockd node will ASSERT on remote lockf. (PTS 38435, No TPR, Severity 2) 33. EX NFS server: lockd can be hung on unavailable chard cluster mount. (PTS 38436, No TPR, Severity 3) 34. Page table manipulation in kdb has bugs. (PTS 38523, No TPR, Severity 2) 35. EX NFS client: Cluster must deal with size changes at NFS server correctly. (PTS 38530, No TPR, Severity 2) 36. Nodedown needs to be run at higher priority. (PTS 38649, No TPR, Severity 2) 37. Cnet Adapter Check causes the whole cluster to hang. (PTS 38666, No TPR, Severity 2) 38. VXFS inode has a different generation number than CFS file handle. (PTS 38697, No TPR, Severity 2) 39. wtmpx file has incomplete records after failover when tnc_rootopt=0x104000. (PTS 38748, No TPR, Severity 2) 40. Get rid of tnc_rootopt in /stand/boot. (PTS 39134, No TPR, Severity 2) Currently, there is no way to specify mount options for the root filesystem, except through the tnc_rootopt bootstrap parameter which only allow the specification of the chard, charddup, and intr mount options. This means that there is no way to specify backup cache nodes for the root filesystem. As a workaround, the system will automatically use the secondary root node as the backup cache for the root filesystem. Additionally, if the secondary root node is not available, the system will automatically select another node in the cluster to provide the backup cache for the root filesystem. Please note that this auto-selection capability applies only to the root filesystem, i.e. only nodes specified via the bkup option to the mount command may become a backup cache for non-root filesystems. In the future, support will be added to use /etc/vfstab to determine the mount options for the root filesystem, e.g. to allow the explicit specification of its backup cache nodes. 41. Spawndaemon does not validate data provided by daemon configuration files. If fields such as the name of the restart script are invalid, Keepalive would be unable to restart the daemon. (PTS 39574, No TPR, Severity 2) 42. disksetup and diskadd man pages need changes for D10IPM20. (PTS 39829, No TPR, Severity 1) The disksetup and diskadd commands must be run on the node to which the disk is attached. 43. pdiconfig man page needs changes for D10IPM20. (PTS 39830, No TPR, Severity 1) The pdiconfig command has a new option. The -T flag can be specified with the -l option. This flag specifies the node about which to generate the configuration listing. 44. diskformat fails with error: V_REMOUNT failed: device busy. (PTS 39831, No TPR, Severity 1) When diskformat is used on NonStop Clusters, an error is reported and the disk is not formatted. It is not necessary to run diskformat on Compaq Disks. If desired, you may run disksetup with the surface analysis option. However, this may take up to 10 hours. 45. Hot add and hot remove of disks is not supported in NonStop Clusters. (PTS 39832, No TPR, Severity 1) NonStop Clusters does not support the Hot insertion or removal of disk devices feature of pdiadd. It is necessary to reboot the node for the new disks to be recognized. For RAID disks, the array must be configured offline using the Compaq Array Configuration Utility. pdi_hot.c is not clusterized 46. pditimeout is not clusterized for NonStop Clusters. (PTS 39833, No TPR, Severity 1) The pditimeout command is not clusterized for NonStop Clusters. This command is used only for the root node and cannot be used for any dependent nodes. Compaq HBA do not support timeouts; this functionality is currently not applicable to Compaq disk subsystems. 47. slow reboot/nodeup times when fully populated RAID on the node. (PTS 39932, No TPR, Severity 1) Slow nodeup occurs when nodeup processing begins before nodedown processing completes. If a node joins the cluster before nodedown processing is completed for that node, the node may take much longer to re-join the cluster. If you are experiencing this problem, the following actions may improve the time for a node to re-join. If the following appears in your /stand//boot files: AUTOBOOT=YES You may change this parameter in the /stand//boot file for each node to: AUTOBOOT=NO TIMEOUT=300 The node will delay 300 seconds before it attempts to rejoin. This will insure that nodedown processing has completed before nodeup processing begins. You may change the "TIMEOUT" value to increase or decrease the autoboot delay to optimize the node join time for your cluster. If your cluster is booting very slowly, you may improve the boot time by booting the nodes 1 at a time. End of Softdoc