First Edition (December 1999)
Part Number 161397-001
Compaq Computer Corporation
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Notice
The information in this publication is subject to change without notice.
COMPAQ COMPUTER CORPORATION SHALL NOT BE LIABLE FOR TECHNICAL OR
EDITORIAL ERRORS OR OMISSIONS CONTAINED HEREIN, NOR FOR INCIDENTAL OR
CONSEQUENTIAL DAMAGES RESULTING FROM THE FURNISHING, PERFORMANCE, OR
USE OF THIS MATERIAL. THIS INFORMATION IS PROVIDED "AS IS" AND COMPAQ
COMPUTER CORPORATION DISCLAIMS ANY WARRANTIES, EXPRESS, IMPLIED OR
STATUTORY AND EXPRESSLY DISCLAIMS THE IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR PARTICULAR PURPOSE, GOOD TITLE AND AGAINST
INFRINGEMENT.
This publication contains information protected by copyright. No part of this publication may be
photocopied or reproduced in any form without prior written consent from Compaq Computer
Corporation.
1999 Compaq Computer Corporation.
All rights reserved. Printed in the U.S.A.
The software described in this guide is furnished under a license agreement or nondisclosure agreement.
The software may be used or copied only in accordance with the terms of the agreement.
Compaq, Deskpro, Compaq Insight Manager, ServerNet, StorageWorks, Systempro, Systempro/LT,
ProLiant, ROMPaq, QVision, SmartStart, NetFlex, QuickFind, PaqFax, ProSignia, registered United
States Patent and Trademark Office.
Fastart, Netelligent, Systempro/XL, SoftPaq, QuickBlank, QuickLock and Neoserver are trademarks
and/or service marks of Compaq Information Technologies Group, L.P. in the U.S. and/or other
countries.
Microsoft, MS-DOS, Windows, and Windows NT are registered trademarks of Microsoft Corporation.
Pentium is a registered trademark and Xeon is a trademark of Intel Corporation.
Oracle is a registered trademark. Oracle8 and Oracle8i are trademarks of Oracle Corporation.
Other product names mentioned herein may be trademarks and/or registered trademarks of their
respective companies.
Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8 Release 8.0.5 and Oracle8i
Release 8.1.5 Administrator Guide
First Edition (December 1999)
Part Number 161397-001
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Contents
About This Guide
Purpose .................................................................................................................... xiii
Audience .................................................................................................................. xiii
Scope ........................................................................................................................ xiv
Referenced Manuals .................................................................................................. xv
Supplemental Documents ......................................................................................... xvi
Text Conventions.....................................................................................................xvii
Symbols in Text.......................................................................................................xvii
Symbols on Equipment.......................................................................................... xviii
Rack Stability ........................................................................................................... xix
Getting Help ............................................................................................................. xix
Compaq Technical Support ............................................................................... xix
Compaq Website.................................................................................................xx
Compaq Authorized Reseller..............................................................................xx
Chapter 1
Clustering Overview
Clusters Defined ...................................................................................................... 1-2
Availability .............................................................................................................. 1-3
Scalability ................................................................................................................ 1-3
Compaq Parallel Database Cluster Overview.......................................................... 1-4
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
iv Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Chapter 2
Architecture
Compaq ProLiant Servers ........................................................................................ 2-2
High-Availability Features of ProLiant Servers ............................................... 2-3
Shared Storage Subsystem Components .................................................................. 2-3
RA4000 Array................................................................................................... 2-3
RA4000 Array Controllers................................................................................ 2-4
Storage Hubs..................................................................................................... 2-5
Fibre Host Adapters .......................................................................................... 2-5
Gigabit Interface Converter-Shortwave ............................................................ 2-6
Fibre Channel Cables........................................................................................ 2-6
Shared Storage Subsystem Features......................................................................... 2-6
Maximum Distances Between Cluster Nodes and Shared Storage
Subsystem Components .................................................................................... 2-7
Redundant Fibre Channel Arbitrated Loop....................................................... 2-7
Multiple Redundant Fibre Channel Arbitrated Loops....................................... 2-9
I/O Data Paths................................................................................................. 2-11
SCSI Disks...................................................................................................... 2-13
I/O Path Configuration Guidelines......................................................................... 2-13
I/O Path Configuration Rules.......................................................................... 2-16
Active/Standby Configuration Examples........................................................ 2-17
Active/Active Configuration Examples .......................................................... 2-23
Summary of I/O Path Failure and Failover Scenarios..................................... 2-28
Cluster Interconnect Options.................................................................................. 2-33
Redundant Ethernet Cluster Interconnect ....................................................... 2-34
Redundant ServerNet Cluster Interconnect..................................................... 2-37
Local Area Network........................................................................................ 2-39
Chapter 3
Cluster Software Components
Overview of the Cluster Software............................................................................ 3-1
Microsoft Windows NT Server 4.0 .......................................................................... 3-2
Compaq Software..................................................................................................... 3-2
Compaq SmartStart and Support Software ....................................................... 3-2
Compaq System Configuration Utility ............................................................. 3-3
Compaq Array Configuration Utility................................................................ 3-3
Fibre Channel Fault Isolation Utility ................................................................ 3-3
Compaq Insight Manager.................................................................................. 3-4
Compaq Insight Manager XE ........................................................................... 3-4
Compaq Options ROMPaq ............................................................................... 3-4
Compaq Redundancy Manager......................................................................... 3-5
Compaq Operating System Dependent Modules .............................................. 3-5
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Contents v
Cluster Software Components
continued
Oracle Software ....................................................................................................... 3-6
Oracle8 Server Enterprise Edition Release 8.0.5.............................................. 3-7
Oracle8i Server Enterprise Edition Release 8.1.5............................................. 3-9
Application Failover and Reconnection Software .......................................... 3-11
Chapter 4
Planning
Site Planning............................................................................................................ 4-2
Capacity Planning for Cluster Hardware ................................................................. 4-3
Compaq ProLiant Servers................................................................................. 4-3
Planning Shared Storage Subsystem Components ........................................... 4-4
Planning Cluster Interconnect and Client LAN Components ........................... 4-5
Reference Material for Hardware Sizing .......................................................... 4-7
Planning the Cluster Configuration ......................................................................... 4-7
Sample Small Configuration for the PDC/O2000 ............................................ 4-8
Sample Large Configuration for the PDC/O2000 ............................................ 4-9
RAID Planning ...................................................................................................... 4-10
Supported RAID Levels ................................................................................. 4-12
Raw Data Storage and Database Size ............................................................. 4-13
Selecting RAID Levels................................................................................... 4-14
Planning the Grouping of Physical Disk Storage Space ........................................ 4-15
Disk Drive Planning .............................................................................................. 4-16
Nonshared Disk Drives................................................................................... 4-16
Shared Disk Drives......................................................................................... 4-16
Network Planning .................................................................................................. 4-17
Windows NT Server Hosts Files for the Ethernet Cluster Interconnect ......... 4-17
Windows NT Server Hosts Files for the ServerNet Cluster Interconnect....... 4-18
Client LAN ..................................................................................................... 4-18
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
vi Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Chapter 5
Installation and Configuration for Oracle8 Release 8.0.5
Installation Overview............................................................................................... 5-2
Installing the Hardware............................................................................................ 5-3
Setting Up the Nodes ........................................................................................ 5-3
Installing the Fibre Host Adapters .................................................................... 5-4
Installing GBIC-SW Modules for the Fibre Host Adapters .............................. 5-4
Cabling the Fibre Host Adapters to the Storage Hubs ...................................... 5-4
Installing the Ethernet Cluster Interconnect Adapters ...................................... 5-5
Installing the Client LAN Adapters .................................................................. 5-6
Setting Up the RA4000 Arrays ......................................................................... 5-6
Installing GBIC-SW Modules for the RA4000 Array Controllers.................... 5-8
Cabling the Storage Hubs to the RA4000 Array Controllers............................ 5-8
Installing Additional Redundant FC-ALs ....................................................... 5-12
Cabling the Ethernet Cluster Interconnect ...................................................... 5-13
Cabling the Client LAN .................................................................................. 5-14
Installing Operating System Software and Configuring the RA4000 Arrays ........ 5-15
Guidelines for Clusters ................................................................................... 5-15
Automated Installation Using SmartStart ....................................................... 5-16
Installing Compaq Redundancy Manager .............................................................. 5-20
Verifying Shared Storage Using Redundancy Manager ................................. 5-21
Defining Active Array Controllers ................................................................. 5-22
Installing Oracle Software ..................................................................................... 5-24
Verifying Cluster Communications ....................................................................... 5-24
Installing the Compaq OSDs.................................................................................. 5-25
OSD Installation Steps.................................................................................... 5-25
Running the NodeList Configurator ............................................................... 5-27
Installing Object Link Manager ............................................................................. 5-28
Configuring Oracle Software ................................................................................. 5-29
Additional Notes on Configuring Oracle Software......................................... 5-29
Verifying the Hardware and Software Installation................................................. 5-31
Cluster Communications................................................................................. 5-31
Access to Shared Storage from All Nodes...................................................... 5-32
OSDs ............................................................................................................... 5-32
Power Distribution and Power Sequencing Guidelines.......................................... 5-32
Server Power Distribution............................................................................... 5-33
RA4000 Array Power Distribution ................................................................. 5-33
Power Sequencing........................................................................................... 5-34
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Contents vii
Chapter 6
Installation and Configuration for Oracle8i Release 8.1.5
Installation Overview............................................................................................... 6-2
Installing the Hardware............................................................................................ 6-3
Setting Up the Nodes........................................................................................ 6-3
Installing the Fibre Host Adapters .................................................................... 6-4
Installing GBIC-SW Modules for the Fibre Host Adapters.............................. 6-4
Cabling the Fibre Host Adapters to the Storage Hubs ...................................... 6-5
Installing the Cluster Interconnect Adapters..................................................... 6-6
Installing the Client LAN Adapters .................................................................. 6-7
Setting Up the RA4000 Arrays......................................................................... 6-7
Installing GBIC-SW Modules for the RA4000 Array Controllers ................... 6-9
Cabling the Storage Hubs to the RA4000 Array Controllers............................ 6-9
Installing Additional Redundant FC-ALs....................................................... 6-13
Cabling the Cluster Interconnect .................................................................... 6-14
Cabling the Client LAN.................................................................................. 6-17
Installing Operating System Software and Configuring the RA4000 Arrays ........ 6-18
Guidelines for Clusters ................................................................................... 6-18
Automated Installation Using SmartStart ....................................................... 6-19
Installing Compaq Redundancy Manager.............................................................. 6-23
Verifying Shared Storage Using Redundancy Manager ................................. 6-24
Defining Active Array Controllers ................................................................. 6-25
Installing Compaq OSDs ....................................................................................... 6-27
Verifying Installation of the SNMP Service................................................... 6-28
Verifying Cluster Communications ................................................................ 6-29
Mounting Remote Drives and Verifying Administrator Privileges ................ 6-30
Installing Ethernet OSDs ................................................................................ 6-31
Installing ServerNet OSDs, Drivers, and SNMP Agents................................ 6-40
Verifying the ServerNet Cluster Interconnect ................................................ 6-52
Installing Oracle Software ..................................................................................... 6-52
Configuring Oracle Software................................................................................. 6-53
Additional Notes on Configuring Oracle Software ........................................ 6-53
Installing Object Link Manager ............................................................................. 6-54
Verifying the Hardware and Software Installation ................................................ 6-55
Cluster Communications ................................................................................ 6-55
Access to Shared Storage from All Nodes...................................................... 6-55
OSDs .............................................................................................................. 6-55
Power Distribution and Power Sequencing Guidelines ......................................... 6-56
Server Power Distribution .............................................................................. 6-56
RA4000 Array Power Distribution ................................................................. 6-57
Power Sequencing .......................................................................................... 6-58
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
viii Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Chapter 7
Cluster Management
Cluster Management Concepts ................................................................................ 7-2
Powering Off a Node Without Interrupting Cluster Services ........................... 7-2
Managing a Cluster in a Degraded Condition................................................... 7-2
Managing Network Clients Connected to a Cluster.......................................... 7-3
Cluster Events ................................................................................................... 7-3
Management Applications ....................................................................................... 7-4
Monitoring Server and Network Hardware....................................................... 7-4
Managing Shared Drives................................................................................... 7-5
Monitoring Redundant Fibre Channel Arbitrated Loops .................................. 7-5
Monitoring the Database................................................................................... 7-6
Remotely Managing a Cluster .......................................................................... 7-7
Software Maintenance for Oracle8 Release 8.0.5 .................................................... 7-7
Deinstalling the OSDs....................................................................................... 7-7
Upgrading Oracle8 Server ................................................................................ 7-8
Software Maintenance for Oracle8i Release 8.1.5 ................................................... 7-8
Deinstalling the OSDs....................................................................................... 7-8
Deinstalling a Partial OSD Installation ........................................................... 7-11
Upgrading Oracle8i Server ............................................................................. 7-11
Managing Changes to Shared Storage ................................................................... 7-12
Replacing a Failed Disk .................................................................................. 7-12
Adding Disk Drives to Increase Storage Capacity.......................................... 7-12
Adding an RA4000 Array............................................................................... 7-13
Replacing a Cluster Node ...................................................................................... 7-15
Removing the Node ........................................................................................ 7-15
Adding the Replacement Node ....................................................................... 7-16
Adding a Cluster Node........................................................................................... 7-19
Preparing the New Node................................................................................. 7-20
Preparing the Existing Cluster Nodes ............................................................. 7-21
Installing the Cluster Software for Oracle8 Release 8.0.5 .............................. 7-21
Installing the Cluster Software for Oracle8i Release 8.1.5 ............................. 7-22
Monitoring Cluster Operation ................................................................................ 7-23
Tools Overview............................................................................................... 7-23
Using Compaq Redundancy Manager ............................................................ 7-24
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Contents ix
Chapter 8
Troubleshooting
Basic Troubleshooting Tips ..................................................................................... 8-2
Power ................................................................................................................ 8-2
Physical Connections........................................................................................ 8-2
Accessibility of Resources................................................................................ 8-3
Software Revisions ........................................................................................... 8-4
Firmware Revisions.......................................................................................... 8-5
Troubleshooting Oracle8 Release 8.0.5 and OSD Installation Problems and
Error Messages ........................................................................................................ 8-6
While Running the NodeList Configurator, a Dialog Box Appears
Indicating Inability to Connect to Remote Nodes............................................. 8-6
Error Message: Unable to Start Cluster Manager (CMSRVR.EXE) ................ 8-6
Error Message: Unable to Start Oracle Service ................................................ 8-7
Error Message: "Dependent Service Has Not Started" Received While
Attempting to Start the Oracle Service ............................................................. 8-7
Error Message: Unable to Configure Oracle Using OPSCONF Utility............ 8-7
Error Message: Error In Creating Oracle Instance ........................................... 8-8
Error Message: Initialization of the Dynamic Link Library CM.DLL
Failed. The Process Is Terminating Abnormally .............................................. 8-8
Unable to Start the Database............................................................................. 8-8
Troubleshooting Oracle8i Release 8.1.5 and OSD Installation Problems and
Error Messages ........................................................................................................ 8-9
Potential Difficulties Installing the OSDs with the Oracle Universal
Installer ............................................................................................................. 8-9
Unable to Start OracleCMService .................................................................. 8-10
Unable to Start OracleNMService .................................................................. 8-11
Unable to Start the Database........................................................................... 8-11
Initialization of the Dynamic Link Library NM.DLL Failed.......................... 8-12
Troubleshooting Node-to-Node Connectivity Problems ....................................... 8-12
Nodes Are Unable to Communicate with Each Other .................................... 8-12
viping Does Not Complete Successfully ........................................................ 8-14
Unable to Ping the Cluster Interconnect or the Client LAN ........................... 8-15
Node or Nodes Unable to Rejoin the Cluster.................................................. 8-16
Ping Shows Cluster Interconnect IP Address Instead of
Client LAN IP Address................................................................................... 8-17
Troubleshooting Client-to-Cluster Connectivity Problems ................................... 8-18
A Network Client Cannot Communicate with the Cluster ............................. 8-18
Troubleshooting Shared Storage Problems............................................................ 8-19
Verifying Connectivity to the Redundant Fibre Channel Arbitrated Loop..... 8-19
Shared Disks in the RA4000 Arrays Are Not Recognized By One or More
Nodes .............................................................................................................. 8-20
Node Cannot Connect to the Shared Drives ................................................... 8-21
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
x Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Troubleshooting
continued
Troubleshooting Compaq Redundancy Manager Problems................................... 8-22
Event Logging................................................................................................. 8-22
Informational Messages .................................................................................. 8-22
Warning Message ........................................................................................... 8-25
Error Messages ............................................................................................... 8-25
Other Redundancy Manager Problems ........................................................... 8-27
Troubleshooting Other Potential Problems ............................................................ 8-28
Windows NT Blue Screen With AFD.SYS Failure Displayed ....................... 8-28
Appendix A
viping Utility
Syntax and Option Summary .................................................................................. A-1
Example ........................................................................................................... A-2
Error Diagnostics .................................................................................................... A-2
Appendix B
Diagnosing and Resolving Shared Disk Problems
Introduction ..............................................................................................................B-1
Run Object Link Manager On All Nodes.................................................................B-3
Restart All Affected Nodes in the Cluster................................................................B-4
Rerun and Validate Object Link Manager On All Affected Nodes .........................B-4
Run and Validate Compaq Redundancy Manager On All Nodes ............................B-5
Run Windows NT Server Disk Administrator On All Nodes ..................................B-6
Run and Validate the Array Configuration Utility On All Nodes............................B-6
Perform Cluster Software and Firmware Checks.....................................................B-7
Perform Cluster Hardware Checks...........................................................................B-7
Contact Your Compaq Support Representative .......................................................B-8
Glossary
Index
List of Figures
Figure 1-1. Example of a Compaq Parallel Database
Model PDC/O2000 cluster ................................................................................. 1-2
Figure 2-1. Maximum distances between PDC/O2000 cluster nodes and
shared storage subsystem components ............................................................... 2-7
Figure 2-2. Two-node PDC/O2000 cluster with one redundant Fibre
Channel Arbitrated Loop.................................................................................... 2-8
Figure 2-3. Two-node PDC/O2000 cluster with two redundant Fibre
Channel Arbitrated Loops ................................................................................ 2-10
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
Contents xi
Figure 2-4. Host adapter-to-Storage Hub data paths............................................. 2-11
Figure 2-5. Storage Hub-to-RA4000 Array data paths ......................................... 2-12
Figure 2-6. Active/standby configuration with one RA4000 Array...................... 2-18
Figure 2-7. Active/standby configuration with two RA4000 Arrays.................... 2-19
Figure 2-8. Active/standby configuration with three RA4000 Arrays.................. 2-20
Figure 2-9. Active/standby configuration with four RA4000 Arrays ................... 2-21
Figure 2-10. Active/standby configuration with five RA4000 Arrays.................. 2-22
Figure 2-11. Active/active configuration with two RA4000 Arrays..................... 2-24
Figure 2-12. Active/active configuration with three RA4000 Arrays................... 2-25
Figure 2-13. Active/active configuration with four RA4000 Arrays .................... 2-26
Figure 2-14. Active/active configuration with five RA4000 Arrays .................... 2-27
Figure 2-15. Redundant Ethernet cluster interconnect components in a
two-node PDC/O2000 cluster .......................................................................... 2-36
Figure 2-16. Redundant ServerNet cluster interconnect components in a
two-node PDC/O2000 cluster .......................................................................... 2-38
Figure 2-17. Redundant ServerNet cluster interconnect components in a
four-node PDC/O2000 cluster.......................................................................... 2-39
Figure 4-1. Two-node PDC/O2000 cluster with one RA4000 Array...................... 4-8
Figure 4-2. Six-node PDC/O2000 cluster with five RA4000 Arrays ..................... 4-9
Figure 4-3. RA4000 Array disk grouping for a PDC/O2000 cluster .................... 4-15
Figure 5-1. Connecting Fibre Host Adapters to Storage Hubs ............................... 5-5
Figure 5-2. RA4000 Arrays connected to clustered servers through one
redundant FC-AL ............................................................................................... 5-7
Figure 5-3. Cabling Storage Hubs to RA4000 Array Controllers in an
active/standby configuration .............................................................................. 5-9
Figure 5-4. Method 1 cabling in an active/active configuration with two
RA4000 Arrays ................................................................................................ 5-11
Figure 5-5. Method 2 cabling in an active/active configuration with two
RA4000 Arrays ................................................................................................ 5-11
Figure 5-6. Redundant client LAN and Ethernet cluster interconnect .................. 5-13
Figure 5-7. Server power distribution in a three-node cluster............................... 5-33
Figure 6-1. Connecting Fibre Host Adapters to Storage Hubs ............................... 6-5
Figure 6-2. RA4000 Arrays connected to clustered servers through one
redundant FC-AL ............................................................................................... 6-8
Figure 6-3. Cabling Storage Hubs to RA4000 Array Controllers in an
active/standby configuration ............................................................................ 6-10
Figure 6-4. Method 1 cabling in an active/active configuration with two
RA4000 Arrays ................................................................................................ 6-12
Figure 6-5. Method 2 cabling in an active/active configuration with two
RA4000 Arrays ................................................................................................ 6-12
Figure 6-6. Redundant client LAN and Ethernet cluster interconnect .................. 6-15
Figure 6-7. Redundant ServerNet cluster interconnect ......................................... 6-16
Figure 6-8. Server power distribution in a three-node cluster............................... 6-57
Figure B-1. Tasks for diagnosing and resolving shared storage problems ............. B-2
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
xii Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
List of Tables
Table 2-1 High-Availability Components of ProLiant Servers................................ 2-3
Table 2-2 Features of Active/Standby and Active/Active Configurations............. 2-14
Table 2-3 I/O Path Failure and Failover Scenarios for Active/Standby
Configurations With One RA4000 Array......................................................... 2-28
Table 2-4 I/O Path Failure and Failover Scenarios for Active/Standby
Configurations With Two or More RA4000 Arrays......................................... 2-29
Table 2-5 I/O Path Failure and Failover Scenarios for Active/Active
Configurations With Two or More RA4000 Arrays......................................... 2-31
Table 5-1 Active/Active Cabling Methods ............................................................ 5-10
Table 5-2 Active Array Controller Locations ........................................................ 5-22
Table 6-1 Active/Active Cabling Methods ............................................................ 6-11
Table 6-2 Active Array Controller Locations ........................................................ 6-26
Table 8-1 Compaq Redundancy Manager Informational Messages ...................... 8-22
Table 8-2 Compaq Redundancy Manager Warning Message ................................ 8-25
Table 8-3 Compaq Redundancy Manager Error Messages.................................... 8-25
Table 8-4 Troubleshooting Other Redundancy Manager Problems....................... 8-27
Table A-1 viping Errors .......................................................................................... A-2
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
About This Guide
Purpose
This administrator guide provides information about the planning, installation,
configuration, implementation, management, and troubleshooting of the
Compaq Parallel Database Cluster Model PDC/O2000 (PDC/O2000).
Audience
The expected audience of this guide consists primarily of MIS professionals
whose jobs include designing, installing, configuring, and maintaining
Compaq Parallel Database Clusters.
The audience of this guide must have a working knowledge of Microsoft
Windows NT Server and of Oracle databases or have the assistance of a
database administrator.
This guide contains information for network administrators, database
administrators, installation technicians, systems integrators, and other
technical personnel in the enterprise environment for the purpose of cluster
planning, installation, implementation, and maintenance.
IMPORTANT: This guide contains installation, configuration, and maintenance
information that can be valuable for a variety of users. If you are installing the PDC/O2000
but will not be administering the cluster on a daily basis, please make this guide available
to the person or persons who will be responsible for the clustered servers after you have
completed the installation.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
xiv Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Scope
This guide offers significant background information about clusters as well as
basic concepts associated with designing clusters. It also contains detailed
product descriptions and installation steps.
This administrator guide is designed to assist you in the following objectives:
Understanding basic concepts of clustering technology
s
Recognizing and using the high-availability features of a PDC/O2000
s
Planning and designing a PDC/O2000 cluster configuration to meet your
s
business needs
Installing and configuring PDC/O2000 hardware and software
s
Managing a PDC/O2000
s
Troubleshooting a PDC/O2000
s
The following summarizes the contents of this guide:
Chapter 1, "Clustering Overview," provides an introduction to
s
clustering technology features and benefits.
Chapter 2, "Architecture," describes the hardware components of a
s
PDC/O2000 and provides I/O path configuration information.
Chapter 3, "Cluster Software Components," describes software
s
components used with a PDC/O2000.
Chapter 4, "Planning," outlines an approach to planning and designing
s
cluster configurations that meet your business needs.
Chapter 5, "Installation and Configuration for Oracle8 Release 8.0.5,"
s
outlines the steps you will take to install and configure PDC/O2000
hardware and software for Oracle8 Release 8.0.5.
Chapter 6, "Installation and Configuration for Oracle8i Release 8.1.5,"
s
outlines the steps you will take to install and configure PDC/O2000
hardware and software for Oracle8i Release 8.1.5.
Chapter 7, "Cluster Management," includes techniques for managing
s
and maintaining a PDC/O2000.
Chapter 8, "Troubleshooting," contains troubleshooting information for
s
a PDC/O2000.
Appendix A, "viping Utility," documents the use of the viping utility to
s
test the ServerNet cluster interconnect.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
About This Guide xv
Appendix B, "Diagnosing and Resolving Shared Disk Problems,"
s
describes procedures to diagnose and resolve shared disk problems.
Glossary, contains definitions of many terms used in this guide.
s
Some clustering topics are mentioned, but not detailed, in this guide. For
example, this guide does not describe how to install and configure Oracle8 or
Oracle8i on a cluster. For information about these topics, see the referenced
and supplemental documents listed in subsequent sections.
Referenced Manuals
For additional information, refer to documentation related to the specific
hardware and software components of the Compaq Parallel Database Cluster.
These related manuals include, but are not limited to:
Documentation related to the ProLiant servers you are clustering (for
s
example, guides, posters, and performance and tuning guides)
Compaq ServerNet documentation (for clusters using Oracle8i)
s
G ServerNet PCI Adapter Installation Guide
G ServerNet Switch Installation Guide
Compaq StorageWorks documentation
s
G Compaq StorageWorks RAID Array 4000 User Guide
G Compaq StorageWorks Fibre Channel Host Bus Adapter Installation
Guide
G Compaq StorageWorks Fibre Channel Storage Hub 7 Installation
Guide
G Compaq StorageWorks Fibre Channel Storage Hub 12 Installation
Guide
Microsoft Windows NT Server documentation
s
G Microsoft Windows NT Server Administrator's Guide
G Microsoft Windows NT Server/Enterprise Edition Administrator's
Guide
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
xvi Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Oracle8 Release 8.0.5 documentation
s
G Oracle8 Enterprise Edition Getting Started Release 8.0.5 for
Windows NT
G Oracle Parallel Management User's Guide
G Oracle8 Enterprise Edition CD
Oracle8i Release 8.1.5 documentation
s
G Oracle8i Parallel Server Setup and Configuration Guide Release
8.1.5
G Oracle8i Enterprise Edition for Windows NT and Windows 95/98
Release Notes, Release 8.1.5
G Oracle8i Enterprise Edition CD
Supplemental Documents
The following technical documents contain important supplemental
information for the Compaq Parallel Database Cluster Model PDC/O2000:
Supported Ethernet Interconnects for Compaq Parallel Database
s
Clusters Using Oracle Parallel Server (ECG062/0299), at
www.compaq.com/highavailability
Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8
s
Parallel Server Release 8.0.5 Certification Matrix, at
www.compaq.com/highavailability
Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i
s
Parallel Server Release 8.1.5 Certification Matrix, at
www.compaq.com/highavailability
Configuring Compaq RAID Technology for Database Servers, technote
s
#1842061, at
www.compaq.com/highavailability
Various technical white papers on Oracle and cluster sizing, which are
s
available from Compaq ActiveAnswers website, at
www.compaq.com/activeanswers
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
About This Guide xvii
Text Conventions
This document uses the following conventions to distinguish elements of text:
User Input, GUI Text a user types or enters appears in boldface.
Items a user selects from a GUI, such as tabs,
Selections
buttons, or menu items, also appear in boldface.
User input and GUI selections can appear in
uppercase and lowercase letters.
File Names, Command These elements can appear in uppercase and
Names, Directory lowercase letters.
Names, Drive Names
Menu Options, Dialog These elements appear in initial capital letters.
Box Names
When you are instructed to type information, type
Type
the information without pressing the Enter key.
without Enter
When you are instructed to enter information, type
Enter
the information and then press the Enter key.
Enter
Symbols in Text
These symbols might be found in the text of this guide. They have the
following meanings.
WARNING: Text set off in this manner indicates that failure to follow directions
in the warning could result in bodily harm or loss of life.
CAUTION: Text set off in this manner indicates that failure to follow directions
could result in damage to equipment or loss of information.
IMPORTANT: Text set off in this manner presents clarifying information or specific
instructions.
NOTE: Text set off in this manner presents commentary, sidelights, or interesting points
of information.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
xviii Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Symbols on Equipment
These icons may be located on equipment in areas where hazardous conditions
may exist.
Any surface or area of the equipment marked with these symbols
indicates the presence of electrical shock hazards. Enclosed area
contains no operator serviceable parts.
WARNING: To reduce the risk of injury from electrical shock hazards,
do not open this enclosure.
Any RJ-45 receptacle marked with these symbols indicates a Network
Interface Connection.
WARNING: To reduce the risk of electrical shock, fire, or damage to
the equipment, do not plug telephone or telecommunications
connectors into this receptacle.
Any surface or area of the equipment marked with these symbols
indicates the presence of a hot surface or hot component. If this
surface is contacted, the potential for injury exists.
WARNING: To reduce the risk of injury from a hot component, allow
the surface to cool before touching.
Power Supplies or Systems marked with these symbols
indicate the equipment is supplied by multiple sources of
power.
WARNING: To reduce the risk of injury from electrical shock,
remove all power cords to completely disconnect power from
the system.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
About This Guide xix
Rack Stability
WARNING: To reduce the risk of personal injury or damage to the equipment,
be sure that:
s The leveling jacks are extended to the floor.
s The full weight of the rack rests on the leveling jacks.
s The stabilizing feet are attached to the rack if it is a single rack installation.
s The racks are coupled together in multiple rack installations.
s A rack can become unstable if more than one component is extended for
any reason. Extend only one component at a time.
Getting Help
If you have a problem and have exhausted the information in this guide, you
can get further information and other help in the following locations.
Compaq Technical Support
In North America, call the Compaq Technical Phone Support Center at
1
1-800-OK-COMPAQ . This service is available 24 hours a day, 7 days a week.
Outside North America, call the nearest Compaq Technical Support Phone
Center. Telephone numbers for worldwide Technical Support Centers are
listed on the Compaq website. Access the Compaq website by logging on to
the Internet at
www.compaq.com
1
For continuous quality improvement, calls may be recorded or monitored.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
xx Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Be sure to have the following information available before you call Compaq:
Technical support registration number (if applicable)
s
Product serial numbers
s
Product model names and numbers
s
Applicable error messages
s
Add-on boards or hardware
s
Third-party hardware or software
s
Operating system type and revision level
s
Detailed, specific questions
s
Compaq Website
The Compaq website has information on this product as well as the latest
drivers and Flash ROM images. You can access the Compaq website by
logging on to the Internet at
http://www.compaq.com
Compaq Authorized Reseller
For the name of your nearest Compaq authorized reseller:
In the United States, call 1-800-345-1518.
s
In Canada, call 1-800-263-5868.
s
Elsewhere, see the Compaq website for locations and telephone
s
numbers.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: a-frnt.doc Last Saved On: 11/3/99 10:41 AM
1
Chapter
Clustering Overview
For many years, companies have depended on clustered computer systems to
fulfill two key requirements: to ensure users can access and process
information that is critical to the ongoing operation of their business, and to
increase the performance and throughput of their computer systems at minimal
cost. These requirements are known as availability and scalability,
respectively.
Historically, these requirements have been fulfilled with clustered systems
built on proprietary technology. Over the years, open systems have
progressively and aggressively moved proprietary technologies into
industry-standard products. Clustering is no exception. Its primary features,
availability and scalability, have been moving into client/server products for
the last few years.
The absorption of clustering technologies into open systems products is
creating less expensive, non-proprietary solutions that deliver levels of
function commonly found in traditional clusters. While some uses of the
proprietary solutions will always exist, such as those controlling stock
exchange trading floors and aerospace mission controls, many critical
applications can reach the desired levels of availability and scalability with
non-proprietary client/server-based clustering.
These new clustering solutions use industry-standard hardware and software,
thereby providing key clustering features at a lower price than proprietary
clustering systems. Before examining the features and benefits of the Compaq
Parallel Database Cluster Model PDC/O2000 (PDC/O2000), it is helpful to
understand the concepts and terminology of clustered systems.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 10/29/99 11:47 AM
1-2 Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Clusters Defined
A cluster is an integration of software and hardware products that enables a set
of loosely coupled servers and shared storage subsystem components to
present a single system image to clients and to operate as a single system. As a
cluster, the group of servers and shared storage subsystem components offers a
level of availability and scalability far exceeding that obtained if each cluster
node operated as a stand-alone server.
The PDC/O2000 uses Oracle8 Parallel Server Release 8.0.5 or Oracle8i
Parallel Server Release 8.1.5, each of which is a parallel database that can
distribute its workload among the cluster nodes.
Figure 1-1 shows an example of a PDC/O2000, including two nodes (ProLiant
servers), two Compaq StorageWorks RAID Array 4000s (RA4000 Arrays),
two Compaq StorageWorks Fibre Channel Storage Hubs (Storage Hubs), a
redundant cluster interconnect, and a client local area network (LAN).
A PDC/O2000 can contain multiple redundant Fibre Channel Arbitrated Loops
(FC-ALs). In this example, the clustered nodes are connected to the database
on the RA4000 Arrays through one redundant FC-AL. Clients access the
database through the client LAN, and the cluster nodes communicate across
the cluster interconnect.
Client LAN
Cluster
Switch/Hub Interconnect
Fibre Host
Adapters (2)
Fibre Host
Adapters (2)
Node 1 Node 2
Storage
Storage
Hub #1
Hub #2
RA4000 Array #1 RA4000 Array #2
Figure 1-1. Example of a Compaq Parallel Database
Model PDC/O2000 cluster
A PDC/O2000 running Oracle8 Release 8.0.5 supports a redundant Ethernet
cluster interconnect. A PDC/O2000 running Oracle8i Release 8.1.5 supports a
redundant Ethernet or Compaq ServerNet cluster interconnect.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 10/29/99 11:47 AM
Clustering Overview 1-3
Availability
When computer systems experience outages, the amount of time the system is
unavailable is referred to as downtime. Downtime has several primary causes:
hardware faults, software faults, planned service, operator error, and
environmental factors. Minimizing downtime is a primary goal of a cluster.
Simply defined, availability is the measure of how well a computer system can
continuously deliver services to clients.
Availability is a system-wide endeavor. The hardware, operating system, and
applications must be designed for availability. Clustering requires stability in
these components, then couples them in such a way that failure of one item
does not render the system unusable. By using redundant components and
mechanisms that detect and recover from faults, clusters can greatly increase
the availability of applications critical to business operations.
Scalability
Simply defined, scalability is a computer system characteristic that enables
improved performance or throughput when supplementary hardware resources
are added. Scalable systems allow increased throughput by adding components
to an existing system without the expense of adding an entire new system.
In a stand-alone server configuration, scalable systems allow increased
throughput by adding processors or more memory. In a cluster configuration,
this result is usually obtained by adding cluster nodes.
Not only must the hardware benefit from additional components, but also
software must be constructed in such a way as to take advantage of the
additional processing power. Oracle8 Parallel Server and Oracle8i Parallel
Server distribute the workload among the cluster nodes. As more nodes are
added to the cluster, cluster-aware applications can use the parallel features of
Oracle8 or Oracle8i Parallel Server to distribute workload among more
servers, thereby obtaining greater throughput.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
Part Number: 161397-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 10/29/99 11:47 AM
1-4 Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide
Compaq Parallel Database
Cluster Overview
As traditional clustering technology has moved into the open systems of
client/server computing, Compaq has provided innovative, customer-focused
solutions. The PDC/O2000 moves client/server computing one step closer to
the capabilities found in expensive, proprietary cluster solutions, at a fraction
of the cost.
The PDC/O2000 combines the popular Microsoft Windows NT Server
operating system and the industry-leading Oracle8 Parallel Server or Oracle8i
Parallel Server with award-winning Compaq ProLiant servers and shared
storage subsystems.
Together, these hardware and software components provide improved
performance through a truly scalable parallel application and improved
availability using clustering software that rapidly recovers from detectable
faults. These components also provide improved availability through
concurrent multinode database access using Oracle8 or Oracle8i Parallel
Server.
Compaq Confidential Need to Know Required
Writer: Linda Waterhouse Project: Parallel Database Cluster Model PDC/O2000 for Oracle Releases 8.0.5 and 8.1.5 Administrator Guide Comments:
| 161397-000
161397-001
|
