Cisco ONS 15454 DWDM User manual

CHAPTER

1-1

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

General Troubleshooting

This chapter provides procedures for troubleshooting the most common problems encountered when

operating a Cisco ONS 15454 DWDM shelf in ANSI or ETSI platforms. To troubleshoot specific alarms,

see Chapter 2, “Alarm Troubleshooting.” If you cannot find what you are looking for, contact Cisco

Technical Support (1-800-553-2447).

Alarms can occur even in those cards that are not explicitly mentioned in the Alarm sections. When an

alarm is raised, refer to its clearing procedure.

Note In this chapter, “ONS 15454” refers to both ANSI and ETSI versions of the platform unless otherwise

noted.

This chapter includes the following sections on network problems:

Note For dense wavelength division multiplexing (DWDM) network acceptance tests, refer to the

Cisco ONS 15454 DWDM Procedure Guide.

•1.1 Loopback Description, page 1-2—Explains in general terms the types of loopback tests (facility

and terminal) that can be run along with general loopback behavior and card loopback behavior.

•1.2 Troubleshooting MXP or TXP Circuit Paths With Loopbacks, page 1-7—Explains how to use

loopbacks tests described in “1.1 Loopback Description” section on page 1-2 to isolate trouble on

muxponder (MXP) and transponder (TXP) circuits.

•1.3 Troubleshooting DWDM Circuit Paths With ITU-T G.709 Monitoring, page 1-24—Explains

how to utilize performance monitoring (PM) and threshold crossing alerts (TCA) to locate signal

degrades on DWDM circuit paths.

The remaining sections describe symptoms, problems, and solutions that are categorized according to

the following topics:

•1.4 Using CTC Diagnostics, page 1-32—Explains how to perform card LED tests and download a

diagnostic file for Cisco Technical Support.

•1.5 Restoring the Database and Default Settings, page 1-35—Provides procedures for restoring

software data and restoring the node to the default setup.

•1.6 PC Connectivity Troubleshooting, page 1-36—Provides troubleshooting procedures for PC and

network connectivity to the ONS 15454.

•1.7 CTC Operation Troubleshooting, page 1-41—Provides troubleshooting procedures for

Cisco Transport Controller (CTC) login or operation problems.

1-2

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.1 Loopback Description

•1.8 Timing, page 1-51—Provides troubleshooting procedures for circuit creation and error

reporting as well as timing reference errors and alarms.

•1.9 Fiber and Cabling, page 1-54—Provides troubleshooting procedures for fiber and cabling

connectivity errors.

•1.10 Power Supply Problems, page 1-58—Provides troubleshooting procedures for rack power

supply problems.

•1.11 Power Up Problems for Node and Cards, page 1-60—Provides troubleshooting procedures for

shelf and card power supply problems.

•1.12 Network Level (Internode) Problems, page 1-60—Provides troubleshooting procedures for

problems between nodes, such as fiber cuts and optical channel network connection (OCHNC)

circuit creation failure.

•1.13 Node Level (Intranode) Problems, page 1-84—Provides troubleshooting procedures for

variable optical attenuator (VOA) startup problems as well as internal VOA control loop problems

in the 32DMX-C and 32DMX-L cards that occur due to counter-propagating light.

1.1 Loopback Description

Use loopbacks and hairpin circuits to test newly created circuits before running live traffic or to logically

locate the source of a network failure. All ONS 15454 and ONS 15454 SDH TXP and MXP cards allow

loopbacks and hairpin test circuits. Other cards do not allow loopbacks, including OPT-BST, OPT-PRE,

OSC-CSM, AD-xB-xx.x, and AD-xC-xx.x cards.

To create a loopback on an ANSI or SONET port, the port must be in the Out-of-Service and

Management, Maintenance (OOS-MA,MT) service state. After you create the loopback, the service state

becomes Out-of-Service and Management, Loopback and Maintenance (OOS-MA,LPBK & MT).

To create a loopback on an SDH or ETSI port, the port must be in the Locked, maintenance

administrative state and the Locked-Enabled, loopback & maintenance administrative state.

Caution Facility or terminal loopbacks can be service-affecting. To protect traffic, apply a lockout or Force

switch to the target loopback port. Basic directions for these procedures exist in Chapter 2, “Alarm

Troubleshooting.” For more information about these operations, refer to the “Maintain the Node” chapter

in the Cisco ONS 15454 DWDM Procedure Guide.

Note In CTC, a facility loopback is sometimes called “facility (line)” loopback, and a terminal loopback is

sometimes called a “terminal (inward)” loopback. This is done to indicate the terminating direction of

the signal: a facility loopback is sent outward toward the span, whereas a terminal loopback is redirected

inward toward its originating port.

1.1.1 Facility Loopbacks

The following sections give general information about facility loopback operations and specific

information about ONS 15454 or ONS 15454 SDH card loopback activity.

1-3

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.1.1 Facility Loopbacks

1.1.1.1 General Behavior

A facility loopback tests the line interface unit (LIU) of a card, the electrical interface assembly (EIA),

and related cabling. After applying a facility loopback on a port, use a test set to run traffic over the

loopback. A successful facility loopback isolates the LIU, the EIA, or the cabling plant as the potential

cause of a network problem.

To test a card LIU, connect an optical test set to a trunk or client port and perform a facility loopback.

Alternately, use a loopback or hairpin circuit on a card that is farther along the circuit path. For example,

Figure 1-1 shows a facility loopback at a trunk port and at a client port on a TXP card.

Figure 1-1 Facility Loopback Path on a Near-End Transponder Card

Caution Before performing a facility loopback on a TXP card, be sure that the card contains at least two data

communications channel (DCC) paths to the node where the card is installed. A second DCC provides

a nonlooped path to log into the node after the loopback is applied, enabling you to remove the facility

loopback. Ensuring a second DCC is not necessary if you are directly connected to the node containing

the loopback card.

Caution Ensure that the facility being loopbacked is not being used by the node for line timing. If it is, a timing

loop will be created.

1.1.1.2 Card Behavior

Port loopbacks either terminate or bridge the loopback signal. All MXP and TXP facility loopbacks are

terminated as shown in Table 1-1.

TXP

Test Set

TXP

145758

Client port

Trunk port

Client port

Trunk port

Test Set

Facility loopback at trunk port

Facility loopback at client port

1-4

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.1.2 Terminal Loopbacks

When a port terminates a facility loopback signal, the signal only loops back to the originating port and

is not transmitted downstream. When a port bridges a loopback signal, the signal loops back to the

originating port and is also transmitted downstream.

Note In Table 1-1, no alarm indication signal (AIS) is injected if the signal is bridged. If the signal is

terminated, an applicable AIS is injected downstream.

The loopback itself is listed in the Conditions window. For example, the window would list the

LPBKFACILITY condition for a tested port. (The Alarms window would show the AS-MT condition

which means that alarms are suppressed on the facility during loopback.)

With a client-side SONET or ANSI facility loopback, the client port service state is OOS-MA,LPBK &

MT. However, any remaining client and trunk ports can be in any other service state. For SONET or

ANSI cards in a trunk-side facility loopback, the trunk port service state is OOS-MA,LPBK & MT and

the remaining client and trunk ports can be in any other service state.

With a client-side SDH or ESTI facility loopback, the client port is in the Locked-enabled,maintenance

& loopback service state. However, the remaining client and trunk ports can be in any other service state.

For MXP and TXP cards in a SDH or ETSI trunk-side facility loopback, the trunk port is in the

Locked-enabled,maintenance & loopback service state and the remaining client and trunk ports can be

in any other service state.

1.1.2 Terminal Loopbacks

The following sections give general information about terminal loopback operations and specific

information about ONS 15454 card loopback activity.

Table 1-1 DWDM Card Facility Loopback Behavior

Card/Port Facility Loopback Signal

TXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L client

ports

Bridged

TXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L trunk

ports

Terminated

TXP_MR_2.5G/TXPP_MR_2.5G client ports Terminated

TXP_MR_2.5G/TXPP_MR_2.5G trunk ports Terminated

MXP_2.5G_10E_C/MXP_2.5G_10E_L client ports Bridged

MXP_2.5G_10E_C/MXP_2.5G_10E_L trunk ports Terminated

MXP_MR_10DME client ports Bridged

MXP_MR_10DME trunk ports Terminated

MXP_MR_2.5G/MXPP_MR_2.5G client ports Bridged

MXP_MR_2.5G/MXPP_MR_2.5G trunk ports Terminated

1-5

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.1.2 Terminal Loopbacks

1.1.2.1 General Behavior

A terminal loopback tests a circuit path as it passes through a TXP or MXP card and loops back. For

example, as shown in Figure 1-2, there are two types of terminal loopbacks shown for a TXP card.

The first is a terminal loopback at the client port. In this situation, the test set traffic comes in through

the TXP trunk port, travels through the card, and turns around because of the terminal loopback in effect

on the card just before it reaches the LIU of the client port. The signal is then sent back through the card

to the trunk port and back to the test set.

The second is a terminal loopback at the trunk port. In this situation, the test set traffic comes in through

the TXP client port, travels through the card, and turns around because of the terminal loopback in effect

on the card just before it reaches the LIU of the trunk port. The signal is then sent back through the card

to the client port and back to the test set.

This test verifies that the terminal circuit paths are valid, but does not test the LIU on the TXP card.

Figure 1-2 Terminal Loopback on a TXP Card

1.1.2.2 Card Behavior

ONS 15454 and ONS 15454 SDH terminal port loopbacks can either terminate or bridge the signal. TXP

terminal loopbacks are terminated as shown in Table 1-2. During terminal loopbacks, if a port terminates

a terminal loopback signal, the signal only loops back to the originating port and is not transmitted

downstream. If the port bridges a loopback signal, the signal loops back to the originating port and is

also transmitted downstream. Client card terminal loopback bridging and terminating behaviors are

listed in Table 1-2.

TXP

Test Set

TXP

145760

Client port

Trunk port

Client port

Trunk port

Terminal loopback at client port

Terminal loopback at trunk port

Test Set

1-6

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.1.2 Terminal Loopbacks

Note In Table 1-2, no AIS signal is injected if the signal is bridged. If the signal is terminated, an applicable

AIS is injected downstream.

The MXP and TXP trunk and client ports can simultaneously maintain different service states:

•For SONET or ANSI TXP and TXPP cards with a client-side terminal loopback, the client port is in

the OOS-MA,LPBK & MT service state and trunk port must be in IS-NR service state.

•For SONET or ANSI MXP and MXPP cards with a client-side terminal loopback, the client port is

in the OOS-MA,LPBK & MT service state and the remaining client and trunk ports can be in any

service state.

•In SONET or ANSI MXP or TXP trunk-side terminal loopbacks, the trunk port is in the

OOS-MA,LPBK & MT service state and the client ports must be in IS-NR service state for complete

loopback functionality. A terminal loopback affects all client ports because it is performed on the

aggregate signal.

•For SDH or ETSI TXP and TXPP client-side facility loopbacks, the client port is in the

Locked-enabled,maintenance & loopback service state and the trunk port must be in

Unlocked-enabled service state.

•For SDH or ETSI MXP and MXPP cards with a client-side terminal loopback, the client port is in

the Locked-enabled,maintenance & loopback service state and remaining client and trunk ports can

be in any service state.

•In SDH and ETSI MXP or TXP trunk-side terminal loopbacks, the trunk port is in the

Locked-enabled,maintenance & loopback service state and the client ports must be in

Unlocked-enabled service state for complete loopback functionality. A facility loopback affects all

client ports because it is performed on the aggregate signal.

The loopback itself is listed in the Conditions window. For example, the window would list the

LPBKTERMINAL condition or LPBKFACILITY condition for a tested port. (The Alarms window

would show the AS-MT condition, which indicates that all alarms are suppressed on the port during

loopback testing.)

Table 1-2 Client Card Terminal Loopback Behavior

Card/Port Terminal Loopback Signal

TXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L client ports Bridged

TXP_MR_10E/TXP_MR_10E_C/TXP_MR_10E_L trunk ports Bridged

TXP_MR_2.5G/TXPP_MR_2.5G client ports Bridged

TXP_MR_2.5G/TXPP_MR_2.5G trunk ports Bridged

MXP_2.5G_10E_C/MXP_2.5G_10E_L client ports Bridged

MXP_2.5G_10E_C/MXP_2.5G_10E_L trunk ports Bridged

MXP_MR_10DME client ports Bridged

MXP_MR_10DME trunk ports Bridged

MXP_MR_2.5G/MXPP_MR_2.5G client ports Terminated

MXP_MR_2.5G/MXPP_MR_2.5G trunk ports Bridged

1-7

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.2 Troubleshooting MXP or TXP Circuit Paths With Loopbacks

Facility loopbacks and terminal loopbacks are often used together to test the circuit path through the

network or to logically isolate a fault. Performing a loopback test at each point along the circuit path

systematically isolates possible points of failure. MXP or TXP loopback tests differ from other testing

in that loopback testing does not require circuit creation. MXP or TXP client ports are statically mapped

to the trunk ports so no signal needs to traverse the cross-connect card (in a circuit) to test the loopback.

You can use these procedures on transponder cards (TXP, TXPP) or muxponder cards (MXP, MXPP)

cards. The example in this section tests an MXP or TXP circuit on a three-node bidirectional line

switched ring (BLSR) or multiplex section-shared protection ring (MS-SPRing). Using a series of

facility loopbacks and terminal loopbacks, the example scenario traces the circuit path, tests the possible

failure points, and eliminates them. The logical progression contains six network test procedures:

Note MXP and TXP card client ports do not appear when you click the Maintenance > Loopback tab unless

they have been provisioned. Do this in the card view by clicking the Provisioning > Pluggable Port

Modules tab. For information about provisioning client ports, refer to the Cisco ONS 15454 DWDM

Procedure Guide.

Note The test sequence for your circuits will differ according to the type of circuit and network topology.

1. A facility loopback on the source-node MXP or TXP port

2. A terminal loopback on the source-node MXP or TXP port

3. A facility loopback on the intermediate-node MXP or TXP port

4. A terminal loopback on the intermediate-node MXP or TXP port

5. A facility loopback on the destination-node MXP or TXP port

6. A terminal loopback on the destination-node MXP or TXP port

Note Facility and terminal loopback tests require on-site personnel.

1.2.1 Perform a Facility Loopback on a Source-Node MXP or TXP Port

This facility loopback test is performed on the node source port in the network circuit. In the testing

situation used in this example, the source muxponder or transponder port under test is located in the

source node. Facility loopback can be performed at the trunk port or at a client port. Completing a

successful facility loopback on this port isolates the source MXP or TXP port as a possible failure point.

Figure 1-3 shows the facility loopback examples on source ONS node TXP ports (client and trunk).

1-8

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.2.1 Perform a Facility Loopback on a Source-Node MXP or TXP Port

Figure 1-3 Facility Loopback on a Circuit Source MXP or TXP Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Note Facility loopbacks require on-site personnel.

Complete the “Create the Facility Loopback on the Source-Node MXP or TXP Port” procedure on

page 1-8.

Create the Facility Loopback on the Source-Node MXP or TXP Port

Step 1 Connect an optical test set to the port you are testing.

Note For specific procedures to connect, set up, and use the test set equipment, consult the

manufacturer.

Use appropriate cabling to attach the transmit (Tx) and receive (Rx) terminals of the optical test set to

the port you are testing. The Tx and Rx terminals connect to the same port.

Step 2 Adjust the test set accordingly. (Refer to manufacturer instructions for test set use.)

TXP

Destination

ONS Node

TXPTXP

Source

ONS Node

Source

ONS Node

Test Set

TXP

Destination

ONS Node

TXPTXP

145757

Client port

Trunk port

Client port

Test Set

Trunk port

1-9

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.2.1 Perform a Facility Loopback on a Source-Node MXP or TXP Port

Step 3 In node view (single-shelf mode) or shelf view (multishelf mode), double-click the card to display the card

view.

Step 4 Click the Maintenance > Loopback tabs.

Step 5 Choose OOS,MT (or locked,maintenance) from the Admin State column for the port being tested. If

this is a multiport card, select the appropriate row for the desired port.

Step 6 Choose Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport

card, select the appropriate row for the desired port.

Step 7 Click Apply.

Step 8 Click Yes in the confirmation dialog box.

Note It is normal for the “LPBKFACILITY (ESCON)” condition on page 2-95, “LPBKFACILITY

(FC)” condition on page 2-95, “LPBKFACILITY (GE)” condition on page 2-96,

“LPBKFACILITY (ISC)” condition on page 2-96or the “LPBKFACILITY (TRUNK)” condition

on page 2-97 to appear during loopback setup. The condition clears when you remove the

loopback.

Step 9 Complete the “Test and Clear the MXP or TXP Facility Loopback Circuit” procedure on page 1-9.

Test and Clear the MXP or TXP Facility Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test

set is capable of indicating.

Step 3 If the test set indicates no errors, no further testing is necessary with the facility loopback. Clear the

facility loopback:

a. Click the Maintenance > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state to place the port in service, out of service and disabled, out of service

for maintenance, or automatically in service from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 4 Complete the “Test the MXP or TXP Card” procedure on page 1-9.

Test the MXP or TXP Card

Step 1 Complete the “Physically Replace a Card” procedure on page 2-172 for the suspected bad card and

replace it with a known-good one.

1-10

Cisco ONS 15454 DWDM Troubleshooting Guide, R7.0.1

78-17742-02

Chapter 1 General Troubleshooting

1.2.2 Perform a Terminal Loopback on a Source-Node MXP or TXP Port

Warning

High-performance devices on this card can get hot during operation. To remove the card, hold it by

the faceplate and bottom edge. Allow the card to cool before touching any other part of it or before

placing it in an antistatic bag.

Statement 201

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this,

perform an external switch if a switch has not already occurred. Refer to the procedures in the

“2.8.1 Protection Switching, Lock Initiation, and Clearing” section on page 2-167. For more

information, refer to the “Maintain the Node” chapter in the Cisco ONS 15454 DWDM Procedure Guide.

Step 2 Resend test traffic on the loopback circuit with a known-good card installed.

Step 3 If the test set indicates no errors, the problem was probably the defective card. Return the defective card

to Cisco through the Return Materials Authorization (RMA) process. Contact Cisco Technical Support

(1-800-553-2447).

Step 4 Complete the “Physically Replace a Card” procedure on page 2-172 for the faulty card.

Step 5 Clear the facility loopback:

a. Click the Maintenance > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state to place the port in service, out of service and disabled, out of service

for maintenance, or automatically in service from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 6 Complete the “1.2.2 Perform a Terminal Loopback on a Source-Node MXP or TXP Port” procedure on

page 1-10.

1.2.2 Perform a Terminal Loopback on a Source-Node MXP or TXP Port

The terminal loopback test is performed on the node source MXP or TXP muxponder or transponder

port. For the circuit in this example, it is the source TXP trunk port or a client port in the source node.

Completing a successful terminal loopback to a node source port verifies that the circuit is through the

source port. Figure 1-4 shows an example of a terminal loopback on a source TXP port and a client TXP

port.

Other manuals for ONS 15454 DWDM

Table of contents

Other Cisco Network Hardware manuals

Cisco

Cisco Cisco ONS 15600 SDH User manual

Cisco

Cisco linksys PLTK300 User manual

Cisco

Cisco AIR-BR1410A-A-K9 - Aironet 1410 Wireless... Quick reference guide

Cisco

Cisco NSS2000 Series User manual

Cisco

Cisco 7246 - uBR Router User manual

Cisco

Cisco ASR 9000 Series User manual

Cisco

Cisco GRP= Quick guide

Cisco

Cisco CE 2000 Platform SCE 2000 4xGBE Operator's manual

Cisco

Cisco Firepower 2100 Series User manual

Cisco

Cisco MERAKI MX85 Series Manual

Cisco

Cisco Versatile Interface Processor (VIP6-80)... Operator's manual

Cisco

Cisco CSS11501 - 100Mbps Ethernet Load Balancing... User manual

Cisco

Cisco NCS 1004 User manual

Cisco

Cisco CX-FEIP-1FX= Installation and con?guration User manual

Cisco

Cisco 40-Gigabit and 100-Gigabit CFP Transceiver... Quick start guide

Cisco

Cisco IAD1101 User manual

Cisco

Cisco NCS 4016 User manual

Cisco

Cisco ISA550 Instruction Manual

Cisco

Cisco IAD2430 Series Installation guide

Cisco

Cisco ONS 15216-MD-40-ODD User manual

Cisco

Cisco Mc3810 - 16MB Flash Memory Quick guide

Cisco

Cisco NCS 4201 Manual

Cisco

Cisco ONS 15454 Instruction Manual

Cisco

Cisco MA-UMNT-MR-A3 User manual

Popular Network Hardware manuals by other brands

Matrix Switch Corporation

Matrix Switch Corporation MSC-HD161DEL product manual

B&B Electronics

B&B Electronics ZXT9-IO-222R2 product manual

Yudor

Yudor YDS-16 user manual

D-Link

D-Link ShareCenter DNS-320L datasheet

Samsung

Samsung ES1642dc Hardware user manual

Honeywell Home

Honeywell Home LTEM-PV Installation and setup guide

evertz

evertz 7700 Series manual

D-Link

D-Link ShareCenter DNS-343 Quick installation guide

Asante

Asante AsanteBridge 1012 user guide

FireBrick

FireBrick FB2700 user manual

Audiovox Electronics

Audiovox Electronics ARWH2T user manual

ADC

ADC DSX-1 Operational Procedures Guide

NetComm

NetComm NF2 user guide

Alcatel-Lucent

Alcatel-Lucent 5620 user guide

Dell EMC

Dell EMC XC6420 Series Solution guide

Lanner

Lanner NCA-1020A user manual

Allied Telesis

Allied Telesis AT-TN121-A Specifications

Arista

Arista CCA-ETM-Q12 quick start guide