First Edition (September 1999)
Part Number 153668-001
Compaq Computer Corporation
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Notice
The information in this publication is subject to change without notice.
COMPAQ COMPUTER CORPORATION SHALL NOT BE LIABLE FOR TECHNICAL OR
EDITORIAL ERRORS OR OMISSIONS CONTAINED HEREIN, NOR FOR INCIDENTAL OR
CONSEQUENTIAL DAMAGES RESULTING FROM THE FURNISHING, PERFORMANCE, OR
USE OF THIS MATERIAL. THIS INFORMATION IS PROVIDED "AS IS" AND COMPAQ
COMPUTER CORPORATION DISCLAIMS ANY WARRANTIES, EXPRESS, IMPLIED OR
STATUTORY AND EXPRESSLY DISCLAIMS THE IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR PARTICULAR PURPOSE, GOOD TITLE AND AGAINST
INFRINGEMENT.
This publication contains information protected by copyright. No part of this publication may be
photocopied or reproduced in any form without prior written consent from Compaq Computer
Corporation.
1999 Compaq Computer Corporation.
All rights reserved. Printed in the U.S.A.
The software described in this guide is furnished under a license agreement or nondisclosure agreement.
The software may be used or copied only in accordance with the terms of the agreement.
Compaq, Deskpro, Compaq Insight Manager, Systempro, Systempro/LT, ProLiant, ROMPaq, QVision,
SmartStart, NetFlex, QuickFind, PaqFax, ProSignia, registered United States Patent and Trademark
Office.
Fastart, Neoserver, Netelligent, Systempro/XL, SoftPaq, QuickBlank, QuickLock are trademarks and/or
service marks of Compaq Computer Corporation.
Microsoft, Windows, and Windows NT are registered trademarks to Microsoft Corporation.
Intel, Pentium, and Xeon are trademarks and/or registered trademarks of Intel Corporation.
Oracle is a registered trademark and Oracle8 is a trademark of Oracle Corporation.
Other product names mentioned herein may be trademarks and/or registered trademarks of their
respective companies.
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
First Edition (September 1999)
Part Number 153668-001
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Contents
About This Guide
Purpose ....................................................................................................................... xi
Audience ..................................................................................................................... xi
Scope .........................................................................................................................xii
Referenced Manuals ................................................................................................ xiii
Supplemental Documents ......................................................................................... xiv
Text Conventions.......................................................................................................xv
Symbols in Text........................................................................................................xvi
Symbols on Equipment.............................................................................................xvi
Rack Stability ..........................................................................................................xvii
Getting Help .......................................................................................................... xviii
Compaq Technical Support ............................................................................ xviii
Compaq Website................................................................................................xix
Compaq Authorized Reseller.............................................................................xix
Chapter 1
Clustering Overview
Clusters Defined ...................................................................................................... 1-2
Availability .............................................................................................................. 1-3
Scalability ................................................................................................................ 1-3
Compaq Parallel Database Cluster Overview.......................................................... 1-4
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
iv Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Chapter 2
Architecture
Compaq ProLiant Servers ........................................................................................ 2-2
High-Availability Features of ProLiant Servers ............................................... 2-2
Shared Storage Subsystem Features......................................................................... 2-3
Maximum Distance Between Cluster Nodes and Shared Storage
Subsystem Components .................................................................................... 2-3
Redundant Fibre Channel Arbitrated Loops ..................................................... 2-4
Multiple Redundant Fibre Channel Arbitrated Loops....................................... 2-6
Shared Storage Subsystem Components .................................................................. 2-8
RA8000/ESA12000 Storage Subsystem ........................................................... 2-8
HSG80 Array Controllers ................................................................................. 2-8
Storage Hubs..................................................................................................... 2-9
KGPSA-BC Host Adapters............................................................................. 2-10
Fibre Channel Arbitrated Loop Data Paths ............................................................ 2-11
Host Adapter-to-Storage Hub Data Paths ....................................................... 2-11
Storage Hub-to-RA8000/ESA12000 Storage Subsystem Data Path............... 2-12
SCSI Disks...................................................................................................... 2-12
Cluster Interconnect ........................................................................................ 2-13
Redundant Ethernet Cluster Interconnect ....................................................... 2-13
Local Area Network............................................................................................... 2-16
Chapter 3
Cluster Software Components
Microsoft Windows NT Server 4.0 .......................................................................... 3-2
Compaq Software..................................................................................................... 3-2
Compaq SmartStart and Support Software ....................................................... 3-3
Compaq System Configuration Utility ............................................................. 3-3
Compaq Insight Manager.................................................................................. 3-3
Compaq Options ROMPaq ............................................................................... 3-4
Compaq StorageWorks Command Console...................................................... 3-4
Compaq StorageWorks Secure Path for Windows NT ..................................... 3-4
Compaq Operating System Dependent Modules .............................................. 3-5
Oracle Software ....................................................................................................... 3-6
Oracle8 Server Enterprise Edition Release 8.0.5 .............................................. 3-6
Oracle8 Server Release 8.0.5 ............................................................................ 3-6
Oracle8 Parallel Server Option Release 8.0.5 ................................................... 3-6
Oracle8 Enterprise Manager Release 8.0.5 ....................................................... 3-7
Client Reconnect Software ............................................................................... 3-7
Oracle Certification........................................................................................... 3-8
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Contents v
Chapter 4
Planning
Site Planning............................................................................................................ 4-2
Capacity Planning for Cluster Hardware ................................................................. 4-3
Compaq ProLiant Servers................................................................................. 4-3
Planning Shared Storage Components.............................................................. 4-3
Planning Cluster Interconnect and Client LAN Components ........................... 4-5
Reference Material for Hardware Sizing .......................................................... 4-6
Planning the Cluster Configuration .................................................................. 4-6
Sample Small Configuration for the PDC/O5000 ............................................ 4-7
Sample Large Configuration for the PDC/O5000 ............................................ 4-8
RAID Planning ........................................................................................................ 4-9
Supported RAID Levels ................................................................................. 4-10
Raw Data Storage and Database Size ............................................................. 4-11
Selecting RAID Levels................................................................................... 4-11
Planning the Grouping of Physical Disk Storage Space ........................................ 4-12
Disk Drive Planning .............................................................................................. 4-13
NonShared Disk Drives .................................................................................. 4-13
Shared Disk Drives......................................................................................... 4-14
Network Planning .................................................................................................. 4-14
Windows NT Server Hosts Files for the Ethernet Cluster Interconnect ......... 4-14
Client LAN ..................................................................................................... 4-15
Chapter 5
Installation and Configuration
Installation Overview............................................................................................... 5-2
Installing the Hardware............................................................................................ 5-3
Setting Up the Nodes ............................................................................................... 5-3
Installing KGPSA-BC PCI-to-Optical Fibre Channel Host Adapters...................... 5-4
Cabling the KGPSA-BC Host Adapters to the Storage Hubs.................................. 5-4
Installing the Ethernet Cluster Interconnect Adapters ............................................. 5-5
Installing the Client LAN Adapters ......................................................................... 5-5
Cabling the Ethernet Cluster Interconnect ............................................................... 5-6
Cabling the Client LAN........................................................................................... 5-7
Redundant Client LAN..................................................................................... 5-8
Installing Hardware into an RA8000/ESA12000..................................................... 5-8
Installing the Power Supplies ........................................................................... 5-8
Installing the AC Input Power Controllers ....................................................... 5-9
Installing the Storage Building Blocks (SBBs) ................................................ 5-9
Installing the HSG80 Array Controllers ........................................................... 5-9
Installing the HSG80 Cache Memory Modules.............................................. 5-10
Installing the PCMCIA Program Cards .......................................................... 5-10
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
vi Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Installation and Configuration
continued
Cabling a Master Enclosure to Expansion Enclosures ........................................... 5-10
Cabling the SE I/O Modules ........................................................................... 5-11
Cabling the EMUs .......................................................................................... 5-14
Setting the PVA SCSI Addresses.................................................................... 5-15
Cabling Array Controllers to the Storage Hubs ..................................................... 5-15
Installing Operating System Software.................................................................... 5-15
Guidelines for Clusters ................................................................................... 5-15
Using SmartStart............................................................................................. 5-16
Cluster-Specific SmartStart Installation.......................................................... 5-17
Automated Installation Steps .......................................................................... 5-17
Setting Up and Configuring an RA8000/ESA12000 Storage Subsystem .............. 5-19
Designating a Server as a Maintenance Terminal........................................... 5-19
Turning On the Subsystem Power .................................................................. 5-20
Installing the StorageWorks Command Console (SWCC) Client................... 5-20
Using the SWCC to Configure the Storage Subsystem for Fibre Channel ..... 5-21
Discovering SBBs........................................................................................... 5-25
Installing the Adapter Drivers and Device Drivers......................................... 5-25
Running the HSZInstall FC Utility ................................................................. 5-26
Verifying HSG80 Array Controller Properties ............................................... 5-27
Configuring a Storageset................................................................................. 5-28
Verifying Access............................................................................................. 5-29
Installing Secure Path for Windows NT ................................................................ 5-31
Preparing an Existing RA8000/ESA12000 for Secure Path Operation........... 5-32
Installing the Secure Path Driver and Secure Path Agent ............................... 5-32
Installing Secure Path Manager ...................................................................... 5-33
Specifying Storage Unit I/O Paths .................................................................. 5-34
Power Up the Second Storage Hub ........................................................................ 5-34
Creating Partitions.................................................................................................. 5-35
Using Secure Path Manager ................................................................................... 5-36
Determining Disk Identity .............................................................................. 5-37
Balancing the I/O Load................................................................................... 5-37
Defining a Persistent Secure Path RA8000/ESA12000 Drive
Configuration .................................................................................................. 5-38
Installing Oracle Software ..................................................................................... 5-39
Verifying Cluster Communications ....................................................................... 5-39
Installing the Compaq OSDs.................................................................................. 5-40
OSD Installation Steps.................................................................................... 5-41
Run the NodeList Configurator....................................................................... 5-42
Installing Object Link Manager ............................................................................. 5-43
Configuring Oracle Software ................................................................................. 5-44
Additional Notes on Configuring Oracle Software......................................... 5-44
Start the Database in Parallel Mode................................................................ 5-46
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Contents vii
Installation and Configuration
continued
Verifying the Hardware and Software Installation ................................................ 5-46
Cluster Communications ................................................................................ 5-46
Access to Shared Storage from All Nodes...................................................... 5-46
OSDs .............................................................................................................. 5-47
Client Access to the OPS Database ................................................................ 5-47
Power Distribution Guidelines............................................................................... 5-47
Server Power Distribution .............................................................................. 5-47
Storage Subsystem Power Distribution .......................................................... 5-48
Chapter 6
Cluster Management
Management Applications ....................................................................................... 6-2
Monitoring Server and Network Hardware ...................................................... 6-2
Monitoring Storage Subsystem Hardware........................................................ 6-2
Managing Shared Storage................................................................................. 6-3
Monitoring Cluster Performance ...................................................................... 6-4
Monitoring the Database .................................................................................. 6-5
Managing a Remote Cluster ............................................................................. 6-5
Managing Network Clients Connected to a Cluster................................................. 6-6
Monitoring Cluster Events....................................................................................... 6-6
Managing a Cluster in a Degraded Condition.......................................................... 6-7
Secure Path and Drive Status Monitor.............................................................. 6-7
Automatic Failover Detection and Status Reporting ........................................ 6-8
Manual Failback and Status Reporting............................................................. 6-9
Power Sequencing Guidelines ............................................................................... 6-10
Powering Down a Node Without Interrupting Cluster Services..................... 6-10
Shutting Down a Node ................................................................................... 6-10
Powering Down a Node.................................................................................. 6-10
Powering Up a Node....................................................................................... 6-11
Software Maintenance ........................................................................................... 6-11
Uninstalling the OSDs .................................................................................... 6-11
Uninstalling Secure Path ................................................................................ 6-11
Upgrading Oracle8 Server .............................................................................. 6-12
Managing Changes to Shared Storage ................................................................... 6-13
Adding New Storagesets to Increase Storage Capacity .................................. 6-13
Expanding a Virtual Disk Within a Storageset ............................................... 6-14
Replacing a Failed Drive in a Storage Subsystem .......................................... 6-14
Replacing a KGPSA-BC Host Adapter .......................................................... 6-15
Adding a Shared Storage Subsystem .............................................................. 6-17
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
viii Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Cluster Management
continued
Replacing a Cluster Node ...................................................................................... 6-18
Removing the Node ........................................................................................ 6-18
Adding a New or Replacement Cluster Node ........................................................ 6-19
Preparing the New or Replacement Node....................................................... 6-19
Preparing the Existing Cluster Nodes for Adding a New Node...................... 6-21
Installing the Cluster Software........................................................................ 6-21
Chapter 7
Troubleshooting
Basic Troubleshooting Tips ..................................................................................... 7-2
Power ................................................................................................................ 7-2
Physical Connections ........................................................................................ 7-2
Accessibility of Resources................................................................................ 7-3
Software Revisions ........................................................................................... 7-3
Firmware Revisions .......................................................................................... 7-4
Secure Path Restrictions ................................................................................... 7-5
Troubleshooting Oracle and OSD Installation Problems and Error Messages......... 7-5
While Running the NodeList Configurator, a Dialog Box Appears
Indicating Inability to Connect to Remote Nodes............................................. 7-5
Error Message: Unable to Start Cluster Manager (CMSRVR.EXE) ................ 7-5
Error Message: Unable to Start Oracle Service ................................................ 7-6
Error Message: "Dependent Service Has Not Started" Received While
Attempting to Start the Oracle Service ............................................................. 7-6
Error Message: Unable to Configure Oracle Using OPSCONF Utility ............ 7-6
While Trying to Create an Oracle Instance, Unable to Start the Oracle
PGMSService.................................................................................................... 7-7
Initialization of the Dynamic Link Library CM.DLL Failed. The Process is
Terminating Abnormally .................................................................................. 7-7
Unable to Start OracleCMService..................................................................... 7-7
Unable to Start the Database............................................................................. 7-8
Troubleshooting Node-to-Node Connectivity Problems.......................................... 7-8
Nodes Are Unable to Communicate with Each Other ...................................... 7-8
Unable to Ping the Cluster Interconnect or the Client LAN ............................. 7-9
Node or Nodes Unable to Rejoin the Cluster.................................................... 7-9
Ping Shows Cluster Interconnect IP Address Instead of
Client LAN IP Address................................................................................... 7-10
Troubleshooting Client-to-Cluster Connectivity Problems.................................... 7-11
A Network Client Cannot Communicate with the Cluster.............................. 7-11
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Contents ix
Troubleshooting
continued
Troubleshooting Shared Storage Problems............................................................ 7-11
Verifying KGPSA-BC Device Driver Initialization ....................................... 7-12
A Cluster Node Cannot Connect to the Shared Drives................................... 7-14
An Attempt to "Drag-and-Drop" Storagesets Between Path Rails from
Secure Path Manager Generated a Windows NT Blue Screen ....................... 7-15
Windows NT Disk Administrator Shows Storagesets With the Same Label
(Dual Image)................................................................................................... 7-15
Device or Devices Were Not Found by KGPSA-BC Device Driver.............. 7-16
Devices on One FC-AL Path Cannot Be Seen by the Cluster Nodes ............. 7-17
Storagesets Are Not on Their Original Preferred_Paths................................. 7-18
Troubleshooting Other Potential Problems............................................................ 7-19
A New Node Repeatedly Displays a Windows NT Blue Screen During
Boot After the HSZdisk Driver Is Installed .................................................... 7-19
Windows NT Blue Screen With AFD.SYS Failure Displayed....................... 7-20
Appendix A
Hyperterminal Utility
Setting Up the Hyperterminal Utility................................................................A-1
Shortcut for Configuring a Storage Subsystem for Fibre Channel ...................A-2
Glossary
Index
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
About This Guide
Purpose
This administrator guide provides information about the planning, installation,
configuration, and implementation of the Compaq Parallel Database Cluster
Model PDC/O5000 (PDC/O5000).
Audience
The expected audience of this guide consists primarily of MIS professionals
whose jobs include designing, installing, configuring, and maintaining
Compaq Parallel Database Clusters.
The audience of this guide must have a working knowledge of Microsoft
Windows NT Server and of Oracle databases or have the assistance of a
database administrator.
This guide contains information for network administrators, database
administrators, installation technicians, systems integrators, and other
technical personnel in the enterprise environment for the purpose of cluster
planning, installation, implementation, and maintenance.
IMPORTANT: This guide contains installation, configuration, and maintenance
information that can be valuable for a variety of users. If you are installing the PDC/O5000
but will not be administering the cluster on a daily basis, please make this guide available
to the person or persons who will be responsible for the clustered servers after you have
completed the installation.
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
xii Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Scope
This guide offers significant background information about clusters as well as
basic concepts associated with designing clusters. It also contains detailed
product descriptions and installation steps.
This administrator guide is designed to assist you in the following objectives:
Understanding basic concepts of clustering technology
s
Recognizing and using the high-availability features of PDC/O5000
s
Planning and designing your PDC/O5000 cluster configuration to meet
s
your business needs
Installing and configuring your PDC/O5000 hardware and software
s
Managing your PDC/O5000 cluster
s
Troubleshooting the PDC/O5000 cluster
s
The contents of this guide are outlined below:
Chapter 1, "Clustering Overview," provides an introduction to
s
clustering technology features and benefits.
Chapter 2, "Architecture," describes the hardware components of the
s
PDC/O5000.
Chapter 3, "Cluster Software Components," describes software
s
components used with the PDC/O5000.
Chapter 4, "Planning," outlines an approach to planning and designing
s
cluster configurations that meet your business needs.
Chapter 5, "Installation and Configuration," outlines the steps you will
s
take to install and configure PDC/O5000 hardware and software.
Chapter 6, "Cluster Management," includes techniques for managing
s
and maintaining your PDC/O5000 cluster.
Chapter 7, "Troubleshooting," contains troubleshooting information for
s
the PDC/O5000.
Appendix A, "Hyperterminal Utility," discusses how to use the
s
Hyperterminal utility as a shortcut method to configure a storage
subsystem for a Fibre Channel instead of using the StorageWorks
Command Console (SWCC).
The Glossary contains definitions of important terms used in this guide.
s
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
About This Guide xiii
Some clustering topics are mentioned, but not detailed, in this guide. For
example, this guide does not describe how to install and configure Oracle8 on
a cluster. See "Referenced Manuals" and "Supplemental Documents."
Referenced Manuals
For additional information, refer to documentation related to the specific
hardware and software components of your PDC/O5000 cluster. These include
but are not limited to:
Documentation related to the ProLiant servers you are clustering (for
s
example, guides, posters, and performance and tuning guides)
Compaq StorageWorks documentation
s
G StorageWorks documentation provided with the RA8000/ESA12000
Storage Subsystem
G Secure Path documentation provided with the Secure Path CD
G Compaq StorageWorks Fibre Channel Storage Hub 7 Installation
Guide
G Compaq StorageWorks Fibre Channel Storage Hub 12 Installation
Guide
Microsoft Windows NT Server documentation
s
G Microsoft Windows NT Server Administrator's Guide
G Microsoft Windows NT Server/Enterprise Edition Administrator's
Guide
Oracle8 Release 8.0.5 documentation
s
G Oracle8 Parallel Management User's Guide
G Oracle8 Enterprise Edition Getting Started for Windows NT
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
xiv Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Supplemental Documents
The following technical documents contain important supplement information
for the Compaq Parallel Database Cluster Model PDC/O5000:
Supported Ethernet Interconnects for Compaq Parallel Database
s
Clusters Using Oracle Parallel Server (ECG062/0299), located at
www.compaq.com/highavailability/
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8
s
Parallel Server Release 8.0.5 Certification Matrix, located at
www.compaq.com/highavailability/
Configuring Compaq RAID Technology for Database Servers, technote
s
# 1842061, located at
www.compaq.com/highavailability/
Various technical whitepapers on Oracle8 and cluster sizing, which are
s
available from Compaq's ActiveAnswers website at
www.compaq.com/activeanswers/
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
About This Guide xv
Text Conventions
This document uses the following conventions to distinguish elements of text:
Keys appear in boldface. A plus sign (+) between
Keys
two keys indicates that they should be pressed
simultaneously.
USER INPUT User input appears in a different typeface and in
uppercase.
FILENAMES File names appear in uppercase italics.
Menu Options, These elements appear in initial capital letters.
Command Names,
Dialog Box Names
COMMANDS, These elements appear in uppercase.
DIRECTORY NAMES,
and DRIVE NAMES
When you are instructed to type information, type
Type
the information without pressing the Enter key.
without Enter
When you are instructed to enter information, type
Enter
the information and then press the Enter key.
Enter
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
xvi Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Symbols in Text
These symbols may be found in the text of this guide. They have the following
meanings.
WARNING: Text set off in this manner indicates that failure to follow directions
in the warning could result in bodily harm or loss of life.
CAUTION: Text set off in this manner indicates that failure to follow directions
could result in damage to equipment or loss of information.
IMPORTANT: Text set off in this manner presents clarifying information or specific
instructions.
NOTE: Text set off in this manner presents commentary, sidelights, or interesting points
of information.
Symbols on Equipment
These icons may be located on equipment in areas where hazardous conditions
may exist.
Any surface or area of the equipment marked with these symbols
indicates the presence of electrical shock hazards. Enclosed area
contains no operator serviceable parts.
WARNING: To reduce the risk of injury from electrical shock hazards,
do not open this enclosure.
Any RJ-45 receptacle marked with these symbols indicates a Network
Interface Connection.
WARNING: To reduce the risk of electrical shock, fire, or damage to
the equipment, do not plug telephone or telecommunications
connectors into this receptacle.
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
About This Guide xvii
Any surface or area of the equipment marked with these symbols
indicates the presence of a hot surface or hot component. If this
surface is contacted, the potential for injury exists.
WARNING: To reduce the risk of injury from a hot component, allow
the surface to cool before touching.
Power supplies or systems marked with these symbols indicate
the equipment is supplied by multiple sources of power.
WARNING: To reduce the risk of injury from electrical shock,
remove all power cords to completely disconnect power from
the system.
Rack Stability
WARNING: To reduce the risk of personal injury or damage to the equipment,
be sure that:
s The leveling jacks are extended to the floor.
s The full weight of the rack rests on the leveling jacks.
s The stabilizing feet are attached to the rack if it is a single rack installation.
s The racks are coupled together in multiple rack installations.
s Only one component is extended at a time. A rack may become unstable if
more than one component is extended for any reason.
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
xviii Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Getting Help
If you have a problem and have exhausted the information in this guide, you
can get further information and other help in the following locations.
Compaq Technical Support
You are entitled to free hardware technical telephone support for your product
for as long you own the product. A technical support specialist will help you
diagnose the problem or guide you to the next step in the warranty process.
In North America, call the Compaq Technical Phone Support Center at
1
1-800-OK-COMPAQ . This service is available 24 hours a day, 7 days a week.
Outside North America, call the nearest Compaq Technical Support Phone
Center. Telephone numbers for worldwide Technical Support Centers are
listed on the Compaq website. Access the Compaq website by logging on to
the Internet at
www.compaq.com
Be sure to have the following information available before you call Compaq:
Technical support registration number (if applicable)
s
Product serial numbers
s
Product model names and numbers
s
Applicable error messages
s
Add-on boards or hardware
s
Third-party hardware or software
s
Operating system type and revision level
s
Detailed, specific questions
s
1
For continuous quality improvement, calls may be recorded or monitored.
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
About This Guide xix
Compaq Website
The Compaq website has information on this product as well as the latest
drivers and Flash ROM images. You can access the Compaq website by
logging on to the Internet at
www.compaq.com
Compaq Authorized Reseller
For the name of your nearest Compaq authorized reseller:
In the United States, call 1-800-345-1518.
s
In Canada, call 1-800-263-5868.
s
Elsewhere, see the Compaq website for locations and telephone
s
numbers.
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
Compaq Confidential Need to Know Required
Writer: Elizabeth Gardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
File Name: a-frnt.doc Last Saved On: 9/8/99 3:12 PM
1
Chapter
Clustering Overview
Before examining the features and benefits of the Compaq Parallel Database
Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 (PDC/O5000), it is
helpful to understand the concepts and terminology of clustered systems. For
many years, companies have depended on clustered computer systems to
fulfill two key requirements: to ensure users can access and process
information that is critical to the ongoing operation of their business, and to
increase the performance and throughput of their computer systems at minimal
cost. These requirements are known as availability and scalability,
respectively.
Historically, these requirements have been fulfilled with clustered systems
built on proprietary technology. Over the years, open systems have
progressively and aggressively moved proprietary technologies into
industry-standard products. Clustering is no exception. Its primary features,
availability and scalability, have been moving into client/server products for
the last few years.
The absorption of clustering technologies into open systems products is
creating less expensive, nonproprietary solutions that deliver levels of
functionality commonly found in traditional clusters. While some uses of the
proprietary solutions will always exist, such as those controlling stock
exchange trading floors and aerospace mission controls, many critical
applications can reach the desired levels of availability and scalability with
nonproprietary client/server-based clustering.
These new clustering solutions use industry-standard hardware and software,
thereby providing key clustering features at a lower price than proprietary
clustering systems.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 9/8/99 2:58 PM
1-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Clusters Defined
A cluster is an integration of software and hardware products that enables a set
of loosely coupled servers and shared storage subsystem components to
present a single system image to clients and to operate as a single system. As a
cluster, the group of servers and shared storage subsystem components offers a
level of availability and scalability far exceeding that obtained if each cluster
node operated as a stand-alone server.
The PDC/O5000 uses Oracle8 Parallel Server Release 8.0.5, which is a
parallel database that can distribute its workload among the cluster nodes.
Figure 1-1 shows an example of a PDC/O5000, including two nodes (ProLiant
servers), one RA8000/ESA12000 Storage Subsystem, two Compaq
StorageWorks Storage Hubs, a redundant cluster interconnect, and a client
local area network (LAN).
Clients access the database through the client LAN, and the cluster nodes
communicate across the cluster interconnect.
Client LAN
Cluster
Switch/Hub
Interconnect
KGPSA-BC
KGPSA-BC
Host
Host
Adapters (2)
Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub Storage Hub
HSG80 Ctrl A
HSG80 Ctrl B
RA8000/
ESA12000
6-24 drives
Figure 1-1. Example of a Compaq Parallel Database
Model PDC/O5000 cluster
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 9/8/99 2:58 PM
Clustering Overview 1-3
Availability
When computer systems experience outages, the amount of time the system is
unavailable is referred to as downtime. Downtime has several primary causes:
hardware faults, software faults, planned service, operator error, and
environmental factors. Minimizing downtime is a primary goal of a cluster.
Simply defined, availability is the measure of how well a computer system can
continuously deliver services to clients.
Availability is a system-wide endeavor. The hardware, the operating system,
and the applications must be designed for availability. Clustering requires
stability in these components, then couples them in such a way that failure of
one item does not render the system unusable. By using redundant components
and mechanisms that detect and recover from faults, clusters can greatly
increase the availability of applications critical to business operations.
Scalability
Simply defined, scalability is a computer system characteristic that enables
improved performance or throughput when supplementary hardware resources
are added. Scalable systems allow increased throughput by adding components
to an existing system without the expense of adding a new system.
In a stand-alone server configuration, scalable systems allow increased
throughput by adding processors or more memory. In a cluster configuration,
this result is usually obtained by adding cluster nodes.
Not only must the hardware benefit from additional components, but also
software must be constructed in such a way as to take advantage of the
additional processing power. Oracle8 Parallel Server distributes its workload
among the cluster nodes. As more nodes are brought into the cluster,
cluster-aware applications can use the parallel features of Oracle8 Parallel
Server to distribute workload among more servers, thereby obtaining greater
throughput.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 9/8/99 2:58 PM
1-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Compaq Parallel Database
Cluster Overview
As traditional clustering technology has moved into the open systems of
client/server computing, Compaq has provided innovative, customer-focused
solutions. The PDC/O5000 moves client/server computing one step closer to
the capabilities found in expensive, proprietary cluster solutions, at a fraction
of the cost.
The PDC/O5000 combines the popular Microsoft Windows NT Server
operating system and the industry-leading Oracle8 Parallel Server with
award-winning Compaq ProLiant servers and shared storage subsystems.
Together, these hardware and software components provide improved
performance through a truly scalable parallel application and improved
availability using clustering software that rapidly recovers from detectable
faults. These components also provide improved availability through
concurrent multinode database access using Oracle8 Parallel Server.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: b-ch1 Clustering Overview.doc Last Saved On: 9/8/99 2:58 PM
2
Chapter
Architecture
The Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8
Release 8.0.5 (PDC/O5000) is an integration of a number of different
hardware and software products. This chapter discusses how these products
play a role in bringing a complete clustering solution to your computing
environment.
The hardware products include:
Compaq ProLiant servers
s
Shared storage subsystem components
s
G Compaq StorageWorks RAID Array 8000 Fibre Channel Storage
Subsystem or the Compaq StorageWorks Enterprise Storage Array
12000 Fibre Channel Storage Subsystem (RA8000/ESA12000
Storage Subsystem)
G HSG80 Array Controllers
G Compaq StorageWorks Storage Hubs (storage hubs)
G KGPSA-BC PCI-to-Optical Fibre Channel Host Adapters
(KGPSA-BC Host Adapters)
Cluster interconnect components
s
G Ethernet NIC adapters
G Ethernet cables
G Ethernet switches
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-2 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
The software products include:
Microsoft Windows NT Server 4.0 with Service Pack 3 or 4
s
Compaq drivers and utilities
s
Oracle8 Enterprise Edition Release 8.0.5 with the Oracle8 Parallel
s
Server Option Release 8.0.5
IMPORTANT: Compaq recommends Service Pack 4 of Windows NT Server for a
redundant Ethernet cluster interconnect or client LAN. However, because certain
applications may not work with Service Pack 4, consult with your software expert to
confirm that your applications can run with Service Pack 4. Using Service Pack 3 requires
installing the approved Microsoft patch (hot fix) article ID Q156655 (Title: Memory Leak
and STOP Screens Using Intermediate NDIS Drivers).
Refer to Chapter 3, "Cluster Software Components," for a description of the
software products used with the PDC/O5000.
Compaq ProLiant Servers
A primary component of any cluster is the server. Each PDC/O5000 consists
of cluster nodes, where each node is a Compaq ProLiant server.
All nodes in a PDC/O5000 cluster must be identical in model. In addition, all
components common to all nodes in a cluster, such as memory, number of
CPUs, and the interconnect adapters, must be identical and identically
configured.
NOTE: For an up-to-date list of Compaq Parallel Database Cluster-certified servers for the
PDC/O5000 and detailed information about minimum and maximum cluster
configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 for
Oracle8 Parallel Server Release 8.0.5 Certification Matrix. This document is available on
the Compaq website at www.compaq.com/highavailability/
High-Availability Features of ProLiant Servers
In addition to the increased application and data availability enabled by
clustering, ProLiant servers include many reliability features that provide a
solid foundation for effective clustered server solutions. The PDC/O5000
cluster is based on ProLiant servers, most of which offer excellent reliability
through redundant power supplies, redundant cooling fans, and Error
Checking and Correcting (ECC) memory. The high-availability features of
ProLiant servers are a critical foundation of Compaq clustering products.
Table 2-1 lists the high-availability features found in many ProLiant servers.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-3
Table 2-1
High-Availability Components of ProLiant Servers
Hot-pluggable hard drives Redundant power supplies
Digital Linear Tape (DLT) array (optional) ECC-protected processor-memory bus
Uninterruptible power supply (optional) Redundant processor power modules
ECC memory PCI Hot Plug slots (in some servers)
Offline backup processor Redundant cooling fans
Shared Storage Subsystem Features
Maximum Distance Between Cluster Nodes
and Shared Storage Subsystem Components
By using standard short-wave Fibre Channel cables, each RA8000/ESA12000
Storage Subsystem master enclosure can be placed up to 500 meters from the
storage hubs, and the storage hubs can be placed up to 500 meters from the
cluster nodes. See Figure 2-1.
500 meters maximum
KGPSA-BC KGPSA-BC
Host Adapters (2) Host Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub Storage Hub
500 meters
maximum
HSG80 Ctrl A
HSG80 Ctrl B
RA8000/
ESA12000
Storage Subsystem
(Master Enclosure)
6-24 drives
Figure 2-1. Maximum distance between PDC/O5000 cluster nodes and
shared storage components
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-4 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Redundant Fibre Channel Arbitrated Loops
The PDC/O5000 uses redundant Fibre Channel Arbitrated Loops (FC-ALs) to
provide two independent I/O communication paths between the cluster nodes
and the shared storage subsystems.
Figure 2-2 shows the redundant Fibre Channel Arbitrated Loops (FC-ALs) in a
two-node cluster.
KGPSA-BC KGPSA-BC
Host Adapters (2) Host Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub
Storage Hub
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
6-24 drives 6-24 drives
6-24 drives 6-24 drives
6-24 drives 6-24 drives
Figure 2-2. Two-node PDC/O5000 cluster with redundant Fibre
Channel Arbitrated Loops
NOTE: The dotted lines in Figure 2-2 indicate the maximum allowable configuration of
RA8000/ESA12000 Storage Subsystem enclosures for one redundant Fibre Channel
Arbitrated Loop.
As Figure 2-2 shows, each of the two FC-AL paths has exclusive use of the
following hardware components:
One KGPSA-BC Host Adapter in each cluster node
s
One storage hub
s
One HSG80 Array Controller (A or B) in each RA8000/ESA12000
s
Storage Subsystem master enclosure
This redundant path configuration gives cluster resources increased
availability with no single point of failure.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-5
As Figure 2-2 also shows, each of the two KGPSA-BC Host Adapters in a
cluster node connects to a different storage hub. The number of ports provided
on each storage hub (7-port or 12-port) dictates how many cluster nodes and
HSG80 Array Controllers you can install on each path of a redundant FC-AL.
Compaq Secure Path software must be installed on each cluster node in the
cluster. With Secure Path installed, data can flow simultaneously between the
KGPSA-BC Host Adapters in a node and the RA8000/ESA12000 Storage
Subsystems, and the customer can perform static load balancing of the two
FC-AL paths to help maximize throughput. Use the Secure Path Manager
graphical user interface to perform this load balancing and to obtain current
information about the state of the redundant FC-AL paths.
Secure Path will detect the failure of a component or connection along one
FC-AL path and automatically transfer all I/O operations to the second FC-AL
path. Some typical failures and responses are:
The failure of one KGPSA-BC Host Adapter in a node will cause
s
subsequent data requests intended for the failed adapter to be routed
over the other FC-AL path and the KGPSA-BC Host Adapter installed
on that path.
The failure of a storage hub or Fibre Channel cable will be detected as a
s
failure of the KGPSA-BC Host Adapter to which the cable is connected.
Failover to the second FC-AL path, which uses the other host adapter,
storage hub, and Fibre Channel cable, will occur.
The failure of an HSG80 Array Controller will cause the second array
s
controller in the RA8000/ESA12000 Storage Subsystem master
enclosure and its FC-AL path to take over for the failed controller.
Secure Path routes all data requests over the other FC-AL path to the
second array controller, which is able to access the logical disks
configured to the failed array controller.
In all of the above cases, interruptions to the end user are minimal and in some
cases may not even be noticeable.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-6 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Multiple Redundant Fibre Channel Arbitrated Loops
The PDC/O5000 supports the use of multiple redundant Fibre Channel
Arbitrated Loops within the same cluster. You would install additional
redundant FC-ALs in a PDC/O5000 cluster to:
Increase the amount of shared storage space available to the cluster's
s
nodes. Each redundant FC-AL path pair can accommodate a maximum
of two RA8000/ESA12000 Storage Subsystem master enclosures with
two expansion enclosures each. Each enclosure can hold up to 24 disk
drives, for a combined total of 144 disk drives.
Increase the PDC/O5000 cluster's I/O performance.
s
Adding a second redundant FC-AL pair involves duplicating the hardware
components used in the first redundant FC-AL pair. Note, however, that the
second redundant FC-AL pair can contain the same or fewer
RA8000/ESA12000 Storage Subsystem enclosures and disk drives than the
supported maximum of 144 disk drives in six enclosures.
The hardware for each redundant FC-AL pair consists of two KGPSA-BC
Host Adapters in each cluster node, two storage hubs, at least one
RA8000/ESA12000 Storage Subsystem with two HSG80 Array Controllers in
the master enclosure, and Fibre Channel cables.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-7
Figure 2-3 shows a two-node PDC/O5000 cluster with two redundant FC-ALs.
| 153668-001
|
