Application Notes
Data Replication Manager over an Internet Protocol Intersite Link
Par t Number: EK-DRMIP-AA. A01
First Edition (December 2001)
Product Version: ACS Version 8.6-1P
This document lists the third-party Fibre Channel-Internet Protocol (FC-IP) interface devices that are
qualified by Compaq for use in a Data Replication Manager (DRM) FC-IP solution. In addition, this
document overviews the DRM FC-IP solution including introductory material on data replication and the
devices and protocols used with the DRM FC-IP solution.
For the latest version of these Application Notes and other DRM documentation, visit the Compaq storage
website at: http://www.compaq.com/products/sanworks/drm/index.html
b
2001 Compaq Information Technologies Group, L.P.
Compaq, the Compaq logo, SANworks, StorageWorks, Tru64, and OpenVMS are trademarks of Compaq
Information Technologies Group, L.P. in the U.S. and/or other companies.
Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the U.S. and/or other
countries.
UNIX is a trademark of The Open Group in the U.S. and/or other countries.
All other product names mentioned herein may be trademarks of their respective companies.
Confidential computer software. Valid license from Compaq required for possession, use or copying. Consistent
with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and
Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial
license.
Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information is
provided "as is" without warranty of any kind and is subject to change without notice. The warranties for
Compaq products are set forth in the express limited warranty statements accompanying such products. Nothing
herein should be construed as constituting an additional warranty.
Compaq service tool software, including associated documentation, is the property of and contains confidential
technology of Compaq Computer Corporation or its affiliates. Service customer is hereby licensed to use the
software only for activities directly relating to the delivery of, and only during the term of, the applicable services
delivered by Compaq or its authorized service provider. Customer may not modify or reverse engineer, remove,
or transfer the software or make the software or any resultant diagnosis or system management data available to
other parties without Compaq's or its authorized service provider's consent. Upon termination of the services,
customer will, at Compaq's or its service provider's option, destroy or return the software and associated
documentation in its possession.
Printed in the U.S.A.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link
First Edition (December 2001)
Par t Number: EK-DRMIP-AA. A01
2 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Application Notes Contents
Application Notes Contents
These application notes cover the following major topics:
"Other Data Replication Manager Documentation," on page 3
"Compaq Technical Support," on page 4
"About This Document," on page 4
"Qualified Vendor Products for FC-IP Interface Devices," on page 5
"Introduction to Data Replication Manager over an IP Link," on page 12
-- Supported Configurations
-- Fibre Channel to Internet Protocol (FC-IP)
-- Dual and Shared Links
"Network Considerations," on page 15
"World Wide FC-IP Demonstration," on page 16
"Case Study," on page 17
-- DRM Performance in a WAN Environment
-- Distance Considerations
"Glossary," on page 20
Intended Audience
This document is intended for customers who are using the SANworksTM by Compaq Data Replication
Manager and plan to operate it with Fibre Channel over an Internet protocol intersite link.
This document references the principal manual for DRM, Compaq SANworks Data Replication
Manager HSG80 ACS Version 8.6-1P Configuration Guide, part number AA-RPHZB-TE, and it is
assumed that the reader has already viewed this material.
Other Data Replication Manager Documentation
Additional documentation, including white papers and best practices documents, are available via the
Compaq website at: http://www.compaq.com/products/sanworks/drm/index.html
The following documentation may also be helpful:
Compaq SANworks Data Replication Manager HSG80 ACS Version 8.6-1P Failover/Failback
Procedures Guide, part number AA-RPJOB-TE
Compaq SANworks Data Replication Manager HSG80 ACS Version 8.6-1P Release Notes, part
number AA-RPJ2B-TE
Compaq SANworks Data Replication Manager HSG80 ACS Version 8.6-1P Scripting User Guide,
part number EK-DRMSC-OA. BO1
Compaq StorageWorks Heterogeneous Open SAN Design Reference Guide, part number
AA-RMPNA-TE
Compaq StorageWorks SAN Switch Zoning Reference Guide, part number EK-P20ZG-GA
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 3
Compaq Technical Support
Compaq Technical Support
In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ. This
service is available 24 hours a day, 7 days a week.
Be sure to have the following information available before you call Compaq:
Technical support registration number (if applicable)
Product serial numbers
Product model names and numbers
Applicable error messages
Operating system type and revision level
Detailed, specific questions
Outside North America, call the nearest Compaq Technical Support Phone Center. Telephone numbers
for world wide Technical Support Centers are listed on the Compaq website at www.compaq.com.
Compaq Website
Check the Compaq website for more information on the complete line of Fibre Channel storage
products, product certification, technical information, updates, and documentation. This information
can be accessed through our website at:
http://www.compaq.com/storage
Compaq Authorized Reseller
Consultation on all aspects of networking from purchasing to setup and network management is
available through your Compaq Authorized Reseller.
For the name of your nearest Compaq Authorized Reseller:
In the United States, call 1-800-345-1518.
In Canada, call 1-800-263-5868.
Elsewhere, see the Compaq website for locations and telephone numbers.
About This Document
The primary purpose of this document is to list the third-party vendor products that are qualified as an
IP interface for the DRM Fibre Channel over Internet protocol (DRM FC-IP) solution, and to provide
an introduction of FC-IP technology. The vendor products section begins on page 5. Details for each
interface product follow, including a basic specification table and any special device settings that are
required for operation with DRM.
Beginning on page 12, the vendor products section is followed by a general overview of the DRM
FC-IP interface. The overview is intended to introduce uninitiated users to concepts and terminology
related to this Compaq solution. This section may help storage personnel understand the networking
concepts of this solution.
4 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Qualified Vendor Products for FC-IP Interface Devices
Qualified Vendor Products for FC-IP Interface Devices
The vendor products listed in Table 11 are qualified for use as FC-IP interface devices for the DRM
FC-IP storage solution. The vendors are listed in alphabetical order. Each qualified device has a
corresponding fact sheet. Compaq does not rate the devices or make recommendations about these
vendors. Users should make their own vendor product evaluations. This document will be revised as
additional vendor products are qualified.
Table 11: Vendor IP Devices Qualified for the DRM FC-IP Solution
See Fact
Company Name Device Sheet on
Computer Network Technology UltraNet Edge Storage Router Model page 6
Corporation (CNT) 1000
SANcastle Technologies, Inc. GFS-8 Global Data Fabric Switch page 8
SAN Valley Systems, Inc. SL1000 IP-SAN Gateway page 10
Installation and Configuration
For installation and configuration information, please refer to the manufacturer's product
documentation. Some DRM-specific configuration settings may be included with the vendor-specific
data that follows.
Warranty Information
Compaq does not warrant third-party products. Please consult the original equipment manufacturer for
warranty information.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 5
CNT FC-IP Interface Equipment Fact Sheet
CNT FC-IP Interface Equipment Fact Sheet
Vendor: Computer Network Technology Corporation (CNT) http://www.cnt.com
Model Number: UltraNet Edge Storage Router Model 1000 Cer tification Date*: December 3, 2001
Feature Specification
Suppor ted hardware revision 00910000 Revision F
Suppor ted firmware revision 1.2.1
Product function Gateway
Fibre Channel speed 1 Gb/s
Number of FC ports 1
FC buffer-to-buffer credits 7
FC optical interface Single mode, multi mode - SFF pluggable
FC optical connector LC class 1 optics
Ethernet speed (maximum) 100 Mb/s full duplex with compression
Ethernet interface 100 BaseT
Ethernet connectors RJ-45
Number of Ethernet ports 1
Ethernet port load balancing N/A
IP protocol T CP
Power supply Single standard, dual optional
Input voltage, current 90-240 VAC, 47-63 Hz, 2.5 Amps
Power dissipation 60W
Hot swap power supply Yes
Hot swap fans Yes
Field Replaceable Units Power supply, fans, entire unit
Out-of-band management ports RS-232 serial, 10/100 Ethernet
SNMP support, version SNMP v1, MIB-II, partial RFC 2233 (extended interface), partial
FCMGMT-MIB (Fibre Channel MIB)
Internet List Price Unavailable at time of print, contact CNT directly
Form factor 1 rack unit (1U): 1.75"x17.5"x14.5" (HxWxD)
Vendor Statement of Competitive Advantages:
Data compression: Maximum bandwidth utilization, saving on expensive WAN costs
Fabric to fabric CRC: End-to-end data checking for guaranteed data integrity
Incremental Session Management: Network level error recovery for more efficient data delivery
Data caching: Maximum throughput and flow control across the network, regardless of distance
FC and IP payload matching for maximum bandwidth utilization
100 Mb/s Ethernet support: Use existing network without costly Gb/s Ethernet upgrade
Management tools: Point and click GUI configuration tool and web-based monitoring tool, simplifying and
reducing management costs
*Information about this product is current as of this date.
6 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
CNT FC-IP Interface Equipment Fact Sheet
CNT Specific Setup Information
Fibre Channel Brocade Switch Firmware
This device has been qualified with Brocade Fibre Channel (FC) switch firmware versions 2.1.9m and
2.4.1f. Other firmware versions are not qualified for use with this solution.
IMPORTANT: At the time of this document release, firmware version 2.4.1f or later had not been
officially released. Please contact your Compaq Storage Representative to obtain version 2.4.1f Fibre
Channel Switch firmware as it becomes available.
Remote Switch License
A Remote Switch license is required for switches attached directly to CNT Edge devices.
IMPORTANT: When loading this license, do not change the following parameters from the default
values:
E_D_TOV
Data field size
Suppress Class F Traffic
SYNC IO mode
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 7
SANcastle FC-IP Interface Equipment Fact Sheet
SANcastle FC-IP Interface Equipment Fact Sheet
Vendor: SANcastle Technologies, Inc. http://www.sancastle.com
Model Number: GFS-8 Global Data Fabric Switch Cer tification Date*: December 3, 2001
Feature Specification
Suppor ted hardware revision GFS-8
Suppor ted firmware revision 1.1, 1.2
Product function Multifunction 8 port FC storage switch and IP Gateway
Fibre Channel speed 1 Gb/s full duplex totaling 200 Mb/s
Number of FC ports All ports can be configured as FC or IP
FC buffer-to-buffer credits 29
FC optical interface MM
FC optical connector SC
Ethernet speed (maximum) 1000 Mb/s (gigabit Ethernet)
Ethernet interface MM
Ethernet connectors SC
Number of Ethernet ports All ports can be configured as FC or IP
Ethernet port load balancing Yes
IP protocol UDP
Power supply Single
Input voltage, current 120/240 VAC 1.4 Amps
Power dissipation 135 watts
Hot swap power supply No
Hot swap fans No
Field Replaceable Units No
Out-of-band management ports 10baseT Ethernet and RS232 serial port
SNMP support, version VNMP v1, v2 MIB 2, FE-MIB (rfc2837), FCMGMT-MIB
Form factor 2.5U rack mount or shelf mount
Internet List Price $26,250 - $31,750 (as configured)
Other Features
Suppor ts:
Direct IP to Fibre Channel translation for IP/SAN integration, includes VLAN tags to FC zones.
Autonomous Regions for highly secured nets.
Soon to support dual hot swap power supplies and load balancing.
Vendor Statement of Competitive Advantages
The GFS-8 is a full-function FC switch that can be managed as part of the FC fabric.
The switch function allows multiple FC links to talk to more than one GbE port.
Best in class Buffer Credit supporting distance
E port interoperable with Brocade and InRange
*Information about this product is current as of this date.
8 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
SANcastle FC-IP Interface Equipment Fact Sheet
SANcastle Specific Setup Information
Fibre Channel Brocade Switch Firmware
This device has been qualified with Brocade FC switch firmware version 2.4.1f. Other firmware
versions are not qualified for use with this solution.
IMPORTANT: At the time of this document release, firmware version 2.4.1f or later had not been
officially released. Please contact your Compaq Storage Representative to obtain version 2.4.1f Fibre
Channel Switch firmware as it becomes available.
DRM Setup
No DRM-specific setup information is required. Please refer to SANcastle installation documentation.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 9
SAN Valley Systems FC-IP Interface Equipment Fact Sheet
SAN Valley Systems FC-IP Interface Equipment Fact Sheet
Vendor: San Valley Systems, Inc. http://www.sanvalley.com
Model Number: SL1000-AC & SL1000-DC IP-SAN Gateway Cer tification Date*: December 3, 2001
Feature Specification
Suppor ted hardware revision 03
Suppor ted firmware revision 1.2.0 R28
Product function Fibre Channel over IP Gateway
Fibre Channel speed 1 Gb/s full-duplex (200 Mb/s)
Number of FC ports 4
FC buffer-to-buffer credits Suppor ts 2 to 12 BB credits
FC optical interface 50-62.5 micron MM, 850nm
FC optical connector MT-RJ
Ethernet speed (maximum) 1 Gb/s full duplex gigabit Ethernet
Ethernet interface 50-62.5 micron MM, 850nm
Ethernet connectors MT-RJ
Number of Ethernet ports 4 gigabit Ethernet ports
Ethernet port load balancing No
IP protocol UDP
Power supply 2 power supplies per chassis
Input voltage, current SL1000-AC: Universal input 90-240 VAC, 50-60 HZ front power entry
SL1000-DC: DC input voltage 36 V to 75 V, DC input current 2 Amps
Power dissipation 80 W
Hot swap power supply No - but redundant power supplies
Hot swap fans No - but redundant fans
Field Replaceable Units Entire unit
Out-of-band management ports 10/100BaseT & RS232 serial port
SNMP support, version SNMP V1, V2, V3
Form factor 1U 19" Rack Mount 1.72"H x 17"W x 19"D
Internet List Price $29,950
Other features:
End-to-end flow control, IP Layer 2 & 4 Modes, traffic shaping, QoS via ToS (RFC 791)/DiffServ (RFC 2475),
integrated GUI-based management platform
Vendor Statement of Competitive Advantages:
Wire-speed gigabit performance up to 500 km, any wire-speed (8 Mb/s - 1000 Mb/s) over 4000 km
Traffic shaping: Optimized bandwidth utilization leveraging any MAN/WAN to fit every IT budget
Credit management: Low latency, high performance solution with full data path management
Easy to use management GUI for configuration/monitoring; integrates with CA, HP and Tivoli
E-port interoperability with all major FC switch vendors
*Information about this product is current as of this date.
10 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
SAN Valley Systems FC-IP Interface Equipment Fact Sheet
SAN Valley Specific Setup Information
Fibre Channel Brocade Switch Firmware
This device has been qualified with Brocade FC switch firmware versions 2.1.9m and 2.4.1f. Other
firmware versions are not qualified for use with this solution.
IMPORTANT: At the time of this document release, firmware version 2.4.1f or later had not been
officially released. Please contact your Compaq Storage Representative to obtain version 2.4.1f Fibre
Channel Switch firmware as it becomes available.
Buffer-To-Buffer (BB) Credits
If any link in the IP/GbE network between a pair of SL1000s is less than 1000 Mb/s (gigabit), or if the
traffic is shaped to lower data rates, the BB credits on every Brocade switch in the SAN must be set to
2. This issue will be resolved by SAN Valley in January, 2002. In the interim, refer to "SL1000 Release
Notes P/N 72-03011-01 for procedures on adjusting BB credits.
Novell NetWare Minimum Speed
Novell NetWare will not function with the San Valley SL1000 if the data rate of any link in the IP/GbE
network between a pair of SL1000s is less than 1000 Mb/s (gigabit) or if the traffic is shaped to lower
data rates. This issue will be resolved by SAN Valley in January, 2002.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 11
Introduction to Data Replication Manager over an IP Link
Introduction to Data Replication Manager over an IP Link
Supported Configurations
The configurations and restrictions allowed in the DRM FC-IP solution are found in Table 4-1 of the
Compaq SANworks Data Replication Manager HSG80 ACS Version 8.6-1P Configuration Guide, part
number AA-RPHZB-TE. The DRM manual is available via the Compaq website at:
http://www.compaq.com/products/sanworks/drm/index.html
Fibre Channel Brocade Switch Firmware Version
Please refer to the vendor specific setup information (Table 11 on page 5) for the Brocade firmware
version that is qualified for the specific IP device.
Operating Systems Tested
Table 12 below lists the operating systems that were tested and certified for the DRM FC-IP solution.
Table 12: DRM FC-IP Tested Operating Systems
OS manufacturer OS Version
Compaq OpenVMS 7.3
Compaq Tru64 UNIX 5.1a
HP UX 11.0
IBM AIX 4.3.3
Microsoft Windows 2000/NT Server 2000, NT
Advance Server,
Data Center
Novell NetWare 5.1 SP3 & 6.0
SUN Solaris 7.0 & 8.0
DRM Overview
DRM is a storage-based disk mirroring and workload migration solution that copies data online and in
real time to remote locations through an extended Storage Area Network (SAN). During normal data
processing, data is written to local and remote sites. Although copies of data reside at both sites, host
data access normally occurs through the local site unless a failure or catastrophe occurs that disables
processing at that site. When such a failure occurs, the remote site can continue processing data.
DRM provides rapid data access recovery and continued data processing after the loss of one or more
components. DRM uses the peer-to-peer remote copy function of the HSG80 controller to achieve data
replication. HSG80 controller pairs at the local site are connected to their partner HSG80 controller
pairs at the remote site.
DRM can replicate data between extended storage area networks (SANs) over unlimited distances
through a Fibre Channel over an IP link.
12 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Introduction to Data Replication Manager over an IP Link
Fibre Channel to Internet Protocol (FC-IP)
In this Compaq solution, DRM is configured to link FC SANs using an Internet protocol (IP) intersite
link.
Using Internet protocol over an IP-based network, DRM can link sites over any extended distance.
Local SANs are connected through an IP network to create an extended SAN. An FC-IP interface is
used at each end of the intersite link. Each FC-IP interface box encapsulates received FC frames into
IP packets for transmission over the IP network. Similarly, the FC-IP box extracts the original FC
frame from received IP packets and retransmits them to the destination FC node. The FC-IP boxes also
handle IP-level error recovery. Figure 11 shows an overview of a DRM FC-IP configuration.
Network Interconnect
Host A Host B Host W Host X
Switch A Switch Y
IP
FC-IP FC-IP
interface interface
Controller Controller Controller Controller
Pair A Pair B Pair Y Pair Z
Switch B Switch Z
IP
FC-IP FC-IP
interface interface
CXO7786A
Figure 11: DRM configured over an FC-IP interface
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 13
Introduction to Data Replication Manager over an IP Link
Dual Links
The connection between the two DRM sites is called an intersite link (ISL). The DRM configuration in
Figure 1-2 has two (dual) ISLs for redundancy. A dual link configuration is the benchmark for disaster
protection, because if one link fails, the other link will temporarily handle all data replication. A
simplified drawing of a dual link configuration is shown in Figure 12.
For enhanced fault tolerance, a customer may choose to contract with two different providers for the
two ISLs, including different entries into the data center.
FC-IP
FC-IP Switch Y
Switch A
equipment
equipment IP
FC-IP
FC-IP
equipment
equipment IP
Switch Z
Switch B
CXO7794A
Figure 12: Dual link IP configuration
In the dual link configuration, Compaq recommends that the maximum sustained I/O load be limited to
40% of the maximum available link bandwidth. This limit allows for both instantaneous bursts of I/O
activity as well as minimizing the performance effect of a single link failure, during which the entire
I/O workload is carried by the surviving link.
Shared Link
A shared link configuration shown in Figure 13 uses only one ISL between the fabrics. While it is
workable, this configuration is not recommended because it does not allow redundancy between SANs.
It also decreases DRM performance because the bandwidth available for storage is shared between the
two fabrics.
Switch Y
Switch A
IP
Switch Switch
or or
router router
Switch B Switch Z
CXO7795A
Figure 13: Shared link configuration
14 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Network Considerations
Network Considerations
Considerations Relevant to Using the Existing IP Network
The ability to use your existing network with DRM FC-IP depends on the type of storage replication
you plan to do and the traffic already existing on your current network. The key consideration is
whether you have enough unused/available bandwidth from your network to continue the current
network load, any growth, plus handle DRM replication load demands.
Replication Type Use Existing Network? Factors
Mirrored DRM FC-IP A separate network is For peak performance for your current network,
SAN recommended. and for peak DRM performance a separate
network is recommended. A dedicated network is
the benchmark for mirrored DRM FC-IP SAN
systems.
Data Migration The use of your existing Data migration is a one-time movement of data for
network may be upgrade or maintenance purposes. During a
possible. migration it is possible to use your existing
network, however the network performance may
be significantly affected.
Network Speeds
In general, the FC-IP equipment supports Ethernet speeds of 10, 100, and 1,000 Mb/s (gigabit
Ethernet). The network connection should be selected to match the amount of data to be replicated.
Please refer to the case study on page 17 to determine how to size the required bandwidth.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 15
World Wide FC-IP Demonstration
World Wide FC-IP Demonstration
In September 2001, as part of a technology demonstration, Compaq built a global multi-vendor SAN
by linking up three separate SANs on three continents--Australia, North America, and Europe--using
DRM FC-IP technology. With Nijmegan, The Netherlands as the local site, data was replicated to
remote sites deployed in the United States and Australia. At that time, it was the longest distance ever
spanned for a network using FC-IP technology.
Local SAN
Nijmegan,
Remote SAN #1 5,000 miles
The Netherlands
Colorado Springs, 8,000 kilometers 10,000 miles
Colorado, USA
16,000 kilometers
Remote SAN #2
Sidney, Australia
CXO7787A
Figure 14: The Global SAN Spanned Europe, Australia, and North America
This demonstration shows how data can be managed around the world in a unified network. The
advantages include significant improvements in IT operations and the replication of data within
long-distance enterprises. With the DRM FC-IP solution, heterogeneous (multiple operating systems)
global SANs are capable of operating as one integrated unit. The solution allows data to be replicated
in different places over any distance--in campus, metropolitan, and wide-area environments--while
retaining data integrity.
Features
Heterogeneous
Multiple SANs
The demonstration was performed over a variety of lines and speeds.
Location ISL
Nijmegan, The Netherlands E1 (2.0 Mb/s)
Colorado Springs, Colorado USA OC-3 (155 Mb/s)
Sydney, Australia T1 (1.5 Mb/s)
16 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Case Study
Case Study
The following case study illustrates many of the considerations involved in planning a DRM
configuration using FC-IP intersite links.
DRM Performance in a WAN Environment
When planning for replication across distances, the bandwidth and speed of the link may not be the
determining factor in performance of the DRM system.
IMPORTANT: Careful consideration must be given to the distances involved as it applies to DRM.
Distance Considerations
The HSG80 controller uses SCSI protocol to manage the storage devices. Before a SCSI I/O can be
transmitted, it must be encapsulated into Fibre Channel frames. Because of SCSI protocol, a minimum
of 4 trips over the long distance link is required.
These trips are conceptually:
Tell the remote site you want to transmit data.
Wait for the acknowledgment from the remote site.
Send the data to the remote site.
Wait for the acknowledgment from the remote site.
When sending data over fiber, the transmission time is approximately 5 microseconds per kilometer.
Since a minimum of four trips is required for each SCSI data transfer, this translates to a total
transmission delay per command of 20 microseconds per kilometer, or about 32.2 microseconds per
mile. For example, if a remote site is located 150 miles away from the local site, the total time will be
4,830 microseconds (4.83 milliseconds) for every data transfer. Since a typical I/O operation on a
non-DRM configuration with write-back cache takes approximately 500 microseconds, long distances
can have a significant effect on performance.
NOTE: The above calculations for a link of 150 miles do not include any latency induced by the FC to
IP conversions, or latency of the routers and switches in the network.
As an example, in a DRM transfer, an 8 KB synchronous host and replication data transfer over 150
miles will use a bandwidth of 1.6 MB/s. In practical terms, as the distance increases the bandwidth that
the DRM system requires decreases. This principle is illustrated in Figure 15.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 17
Case Study
Distance vs. Required Bandwidth
20
15
Bandwidth --->
10
5
0
0 200 400 600 800 1000
Distance --->
Figure 15: How distance affects bandwidth
Synchronous and Asynchronous Replication
The HSG80 Array Controller functions in two distinct and different operation modes, synchronous and
asynchronous replication.
Synchronous Replication
In the synchronous mode of operation, command completion is not returned to the host until the data
has been stored on both the local and remote sites. If the host computer issues multiple write requests
at the same time (asynchronous host requests), all of these requests are pipelined (multiple requests on
the connection at the same time). Even though multiple commands may be outstanding, completion for
each command is returned only when all data for that command has been received and acknowledged
by the remote site. Since the commands are pipelined, there is very little queuing, so the response time
of each request is determined by the total time for the four trips. This means that the length and type of
the communications path is the primary factor for the response time of each command.
The maximum number of host requests that may be outstanding simultaneously is controlled by the
OUTSTANDING_IO parameter on the controller. If the number of commands issued by the host
exceeds this value, those commands are queued internally by the local controller for later transmission.
Under almost all circumstances, Compaq recommends using synchronous replication for DRM.
Asynchronous Replication
Asynchronous operation reduces response time by returning I/O completion to the host as soon as the
data is sent to the local controller, but before it is received by the remote system. The result is that the
host computer sees only the response time associated with the local system. The time delays associated
with data replication are masked because the replication is accomplished in the background.
18 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Case Study
Because completion of the transfer is returned to the host, the host may send additional requests,
confident that the previous data is safely written. Because of this, the controller must ensure that all
data that was previously acknowledged as being complete, but that is currently "in transit" to the
remote system, is delivered in the correct order. This is accomplished by allowing only one data
transfer at a time (for a particular association set) to be sent to the remote site. Any additional data
transfer commands are queued internally within the controller and sent one at a time as the previous
transfers complete.
Because the host sees a very fast response time, it may issue I/O requests at a rapid rate. The
replication, on the other hand, proceeds at the rate dictated by the length of the intersite link. As a
result, a large queue of requests that are waiting for replication may build within the controller.
The number of queue requests is limited by the OUTSTANDING_IO parameter. The number of
requests that have been acknowledged to the host by the controller as being completed, but have not yet
been sent (acknowledged) by the remote site, is compared to the value of OUTSTANDING_IO. If the
number of requests is greater, then subsequent requests sent by the host are not acknowledged as
completed until they are sent to and acknowledged by the remote site.
Example
A simple example may help to clarify this:
Assume that OUTSTANDING_IO is set to a value of 5 and that a host issues a request, waits for
completion from the controller, then immediately issues another request. Assume also that the DRM
configuration is on IP over a very long distance, requiring a significant amount of time to perform
replication. In asynchronous mode, each request issued by the host is acknowledged as completed by
the controller very quickly. As a result, the host issues five requests before the remote site has
completed the first request. If the host then issues another (sixth) request, this exceeds the value of
OUTSTANDING_IO. The local controller does not return completion of this request to the host, but
queues it behind the previous requests that are awaiting replication. Completion of the sixth request is
not returned until all five preceding requests have been sent to and acknowledged by the remote site,
and the sixth request itself has been sent and acknowledged. The result is that the host sees a response
time for the first five requests equal to the time taken by the local controller, while the sixth request has
a response time equal to the sum of the transit times from local to remote controller for all preceding
requests.
Assume that the time for the local controller is 500 microseconds, while the total time for replication of
an I/O to the remote controller is 5 milliseconds. In this case, the host sees a response time of 500
microseconds for each of the first five requests, since completion is returned immediately. The sixth
request has a response time slightly less than 5.5 milliseconds because it must wait for the first requests
to be completed by the remote site.
Due to this extremely long response time, asynchronous replication is not recommended for DRM
solutions requiring an IP WAN. Please see the white paper part number 1426-0100A-WWEW, DRM
Inter-site Link Performance Analysis.
Conclusion
When DRM is used in a situation where the local and remote sites are located many miles apart, the
speed of light through fiber can cause unacceptable delays in the completion of an I/O transaction.
Increasing the amount of available bandwidth cannot solve this problem. Careful consideration must
be given to these factors when matching customers needs and wants to a particular application.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 19
Glossary
Glossary
This glossary defines terms relevant to the Data Replication Manager and the
Fibre-Channel-to-Internet-Protocol link.
array controller See controller.
asynchronous mode A mode of operation of the remote copy set where the write operation
reports command completion to the host after the data is on the
initiating controller, but before completion of the remote command.
Asynchronous mode can provide greater performance and faster
response time, but the data on all members at any one time cannot be
guaranteed identical.
See also synchronous mode.
controller A hardware device that uses software to facilitate communications
between a host and one or more storage devices organized in an array.
The HS-series StorageWorks family of controllers are all array
controllers.
data rate The bandwidth required without any protocol overhead (8-bit bytes).
default gateway The default path that a computer or router uses to forward and route
data between two or more networks that have different protocols.
dual-redundant A storage subsystem configuration that consists of two active
configuration controllers operating as a single controller. If one controller fails, the
other assumes control of the failing controller's devices.
fabric A network of switches containing a Fibre Channel Arbitrated Loop.
Fibre Channel A technology protocol standardized by ANSI that allows very
high-speed, switching-based serial transmissions.
Fibre Channel Arbitrated A serial data transfer protocol developed by storage device
Loop manufacturers and standardized by ANSI in which high bandwidth
transfer is accommodated between SANs. FC-AL supports optical
media and supports full-duplex transfer rates between SCSI storage
systems.
FC-IP A configuration in which Fibre Channel protocol data is transmitted via
an Internet protocol intersite link.
Fibre Channel Switch A device that provides high-speed, high-bandwidth routing of data (in
Fibre Channel protocol) via link-level addressing.
20 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
Glossary
frame The basic unit of communication using the Fibre Channel protocol.
Each frame consists of a payload encapsulated in control information.
The initiator breaks up the exchange into one or more sequences,
which in turn are broken into one or more frames. The responder
recombines the frames into sequences and exchanges.
IP address The numeric address that identifies a particular computer on the
Internet.
latency The amount of time required for a transmission to reach its destination.
link A connection between two adjacent Fibre Channel ports, consisting of
a transmit fiber and a receive fiber. An example is the connection
between the Fibre Channel switch port and the HSG80 controller.
local site For subsystems using the disaster tolerant Data Replication Manager
solution, the local site is the SAN that is the primary source of
information. When a system outage occurs, the database is recovered
from the remote site SAN.
mirroring The act of creating an exact copy or image of data.
network In data communication, a configuration in which two or more terminals
or devices are connected to enable information transfer.
redundancy The provision of multiple interchangeable components to perform a
single function to deal with failures and errors. A RAIDset is
considered to be redundant when user data is recorded directly to one
member, and all the other members and associated parity also are
recorded. If a member is missing from the RAIDset, its data can be
regenerated as needed, but the RAIDset is no longer redundant until the
missing member is replaced and reconstructed.
remote site For subsystems using the disaster tolerant Data Replication Manager,
data processing occurs at a local SAN site and the data is replicated or
mirrored to the remote site SAN. When local system outage occurs, the
database is recovered from the remote site system.
SCSI Small Computer System Interface. A processor-independent, standard
protocol for system-level interfacing between a computer and
intelligent devices, including hard drives, floppy disks, CD-ROMs,
printers, scanners, and others.
speed of light through Approximately 200,000 kilometers per second or 5 microseconds to
fiber traverse one kilometer.
Application Notes Data Replication Manager over an Internet Protocol Intersite Link 21
Glossary
subnet mask Also known as address mask. An IP network that can be reached
through a single IP address. All the members of the subnet share the
mask value. Members of the subnet can then be referenced more easily.
A subnetwork is a network that is part of another network, connected
through a gateway, bridge, or router.
synchronous mode A mode of operation of the remote copy set where the data is written
simultaneously to the cache of the local subsystem and the cache of the
remote subsystem. The I/O completion status is not sent until all
members of the remote copy set are updated.
See also asynchronous mode.
22 Application Notes Data Replication Manager over an Internet Protocol Intersite Link
| 26-0100A-WW
72-03011-01
AA-RMPNA-TE
AA-RPHZB-TE
AA-RPJ2B-TE
AA-RPJOB-TE
EK-DRMIP-AA
EK-DRMSC-OA
EK-P20ZG-GA
|
