Thursday, November 23, 2017

VSS Related Configuration of a Cisco Catalyst 6500 Series Switches

Here are the essential steps to configure Virtual Switching System (VSS) of a Cisco Catalyst 6509 switch. Here in my setup I have 2 6509 switches with 720 Supervisor Engines, one per each chassis. In this setup, only 1 Supervisor engine will be active and the other will stay standby.

STEP1: Assigning Virtual Switch Domain and Switch Numbers

Domain name must be same on both the switches.

ADM-6509-VSS(config)#switch virtual domain 100
ADM-6509-VSS(config-vs-domain)#switch 1
ADM-6509-VSS(config-vs-domain)#switch 1 priority 110
ADM-6509-VSS(config-vs-domain)#switch 2 priority 100

ADM-6509-VSS-2(config)#switch virtual domain 100
ADM-6509-VSS-2(config-vs-domain)#switch 2
ADM-6509-VSS-2(config-vs-domain)#switch 1 priority 110
ADM-6509-VSS-2(config-vs-domain)#switch 2 priority 100

If the priority is not configured, the lowest switch number will be the active switch..

STEP2: Configuring VSL Port Channel

Virtual Switch Link (VSL) is the port channel which the VSS parameters will be exchanged to sync data & management planes. They are configured as L3 port-channels.

ADM-6509-VSS(config)#int port-channel 11
ADM-6509-VSS(config-if)#no switchport
ADM-6509-VSS(config-if)#switch virtual link 1

ADM-6509-VSS-2(config)#int port-channel 12
ADM-6509-VSS-2(config)#no switchport
ADM-6509-VSS-2(config-if)#switch virtual link 2


STEP3: configure the VSL ports

ADM-6509-VSS(config)#int range te1/5/4-5
ADM-6509-VSS(config-if-range)#switchport mode trunk
ADM-6509-VSS(config-if-range)#channel-group 11 mode on

ADM-6509-VSS-2(config)#int range te2/5/4-5
ADM-6509-VSS-2(config-if-range)#switchport mode trunk
ADM-6509-VSS-2(config-if-range)#channel-group 12 mode on


STEP4: Converting the Switch to Virtual Switch Mode:

ADM-6509-VSS#switch convert mode virtual 

ADM-6509-VSS-2#switch convert mode virtual

If there are no mismatching parameters, switches will reboot after entering this command..

Verification Commands

Following outputs are from a currently working VSS..

show switch virtual







show switch virtual role








show switch virtual link







show switch virtual link port-channel

















show switch virtual redundancy


















show redundancy


















You can also find the mismatch parameters in VSS by issuing the following command
show switch virtual redundancy mismatch

Saturday, November 18, 2017

Stackwise Technology of Cisco IOS Switches

Stackwise technology is used to combine several physical switches into a one logical switch.
Layer 3 switches like Cisco 3750, 3750-X, 3850 and layer 2 switches like 2960-X support this technology. 3750 supports maximum of 9 physical switches per stack..

Special cable type called stack cables are used to create a daisy-chain (loop) between switches. They are connected to the stack ports at the back of the switch. Each supported switch has 2 stack ports.
Following picture shows how it is to be done when it has 3 or more switches.

Either way it works..

Anyhow what required is that each switch is connected to the one below it and the bottom switch should be connected to the one on top.






I am using 2 Cisco 3750 series switches to experience how it works.. IOS version must be equal.
Before stacking, lets see some the stack details of both switches.. I reset both the switches before starting this lab, but remember even resetting does not change the pre-configured stack priorities..
Default priority is 1 and the highest is 15. Switch with the highest priority becomes the master.

Issuing show switch command;









As you can see, both the switches think they are the Master. Let's stack them with a one cable and see what happens..

SW1 went to an auto reboot, and it became just a member. Now I'm consoling to the SW1 but I can see its name as SW2. This means all the configuration of SW1 is overwritten from the configuration of SW2. Issuing show switch command;


Let's see why it became a member.




Stack Master election process is like the following..

1. Configured priority
we can configure a priority to decide which switch becomes the master.

2. Hardware / software priority
The switch with the most extensive feature set has a higher priority than another switch (for example: IP Services vs IP base).

3. Default Configuration 
A switch that already has a configuration will take precedence over switches with no configuration.

4. Uptime
The switch with the longest uptime.

5. MAC address
The switch with the lowest MAC address.

It looks like the MAC address is the tie breaker here.

Anyhow only one stack cable was needed to bring the stack up for 2 switches..

But why we always see 2 stack cables are used to bring a 2 switch stack in production?

Let's issue another command; show switch stack-ring speed


Now let's connect another stack cable and see what happens..
You can see the ring speed has become 32G and the ring configuration is full. This is the reason we always prefer to add 2 cables even for 2 switches..



My real physical connection is like this..



















You can see which switch is connected to which switch by issuing show switch neighbors








Best practice is to change the priority of the switches to be master.  Let's say I want to make the switch 1 to be master,

SW2(config)#switch 1 priority 15


As you can see, it changed the priority, but it did not took over the master role. It will only be the master if the current master is down.




Things to keep in mind:-

1. IOS version must be equal to begin with.
2. Even resetting does not change stack priority values configured.
3. Default priority is 1 and highest is 15 and the highest will be the master.
4. There is no preemption here.
5. Master will overwrite member configuration by its running config.
6. Even though you issued a reload command by consoling to a switch in the stack, all the switches will reboot.

Friday, November 17, 2017

Correct Order of L3 Etherchannel Configuration

If you want to know how to configure L2 etherchannels, please visit here











1. Go to the physical member interfaces with blank configuration  & shut them down
SW1(config )#int range e0/0-1
SW1(config-if-range)#shutdown

2. Configure the physical member interfaces to be routed
SW1(config-if-range)#no switchport

3. Assign the channel group to the physical member interfaces
SW1(config-if-range)#channel-group 1 mode on

Configure SW2 in same manner (above 3 steps) and finally;

4. Go to the portchannel interface and flap it from both switches
SW1(config )#int po1
SW1(config-if)#shutdown
SW1(config-if)#no shutdown

After the portchannel is operational, you must do all other configuration (ip address assignment) only inside the portchannel interface.

If everything went ok, following outputs will be present.
















If it says (RU) it's an L2 etherchannel and it is working in Routed mode. (P) after the member interfaces indicate they are correctly bundled in. Protocol will be a blank because I used unconditionally "on" (neither LACP or PAGP).

Here are the 2 protocols to negotiate etherchannels..

1. LACP (active & passive)
2. PAGP (desirable & auto)

Active and desirable modes start negotiation while passive and auto modes listen only..
As an example, if I used LACP, one side must be considered as active while the other end must be configured as active or passive..









Additional load balancing can be done in global configuration mode.
SW1(config)#port-channel load-balance src-ip 

Thursday, November 16, 2017

Correct Order of L2 Etherchannel Configuration

Entire networks can go down if the etherchannel is not configured correctly or in correct order.











1. Go to the physical member interfaces with blank configuration  & shut them down
SW1(config )#int range e0/0-1
SW1(config-if-range)#shutdown

2. Configure the encapsulation type to match the portchannel to be configured
SW1(config-if-range)#switchport trunk encapsulation dot1q

3. Assign the channel group to the physical member interfaces
SW1(config-if-range)#channel-group 1 mode active

Configure SW2 in same manner (above 3 steps) and finally;

4. Go to the portchannel interface and flap it from both switches
SW1(config )#int po1
SW1(config-if)#shutdown
SW1(config-if)#no shutdown


After the portchannel is operational, you must do all other configuration (trunk/access, allowing vlans etc) only inside the portchannel interface.

If everything went ok, following outputs will be present.
















If it says (SU) it's an L2 etherchannel and it is working. (P) after the member interfaces indicate they are correctly bundled in..












Here the protocol I used is LACP. So the other end must be configured as channel-group mode active or passive..

Here are the 2 protocols to negotiate etherchannels..

1. LACP (active & passive)
2. PAGP (desirable & auto)

Active and desirable modes start negotiation while passive and auto modes listen only..









You can create a channel even without a negotiating protocol by issuing the both ends to be "on"..
Best practice is to use a protocol because "on" can lead to many catastrophes. Even the total network can go down because of a single misconfiguration.

Additional load balancing can be done in global configuration mode.
SW1(config)#port-channel load-balance src-ip 

Wednesday, November 15, 2017

Assigning Many IP Addresses to a Single Interface

When you have a network on an interface with 192.168.1.0/24 and you run out of IPs and you need to put more servers there, you can use secondary IP on that interface and bring up another range like 192.168.2.0/24. Another practical use of secondary address is during IP address space migrations.

Configuration is simple like the following..

R1(config)#int e0/0
R1(config-if)#ip address 192.168.1.1 255.255.255.0
R1(config-if)#ip address 192.168.2.1 255.255.255.0 secondary 
R1(config-if)#ip address 192.168.3.1 255.255.255.0 secondary 
R1(config-if)#ip address 192.168.4.1 255.255.255.0 secondary

Here I have assigned 192.168.1.1 as the primary interface IP address and other 3 IPs as secondary IP addresses.

This will be shown in the routing table under the same interface..





















But you will not see them on show ip interface brief output..







Note that you can assign IP addresses of the same range as secondary IP addresses too..

Wednesday, November 1, 2017

How ASA Real Time Log Viewer Displays Connections

This post will help you to understand how sessions / connections in ASA are built and teared down in ASA and how they are displayed in Real Time Log Viewer..

Outside router is connected to Internet directly and MGT cloud used here is only to get ASDM access to ASA. If you want to configure this lab, click here to go to the post which explains how to setup ASDM access to an ASA.













Interfaces on ASA are configured as following..



Service Policy to inspect ICMP & ICMP error is configured on ASA.





Configuration on R1..

R1(config)#ip route 0.0.0.0 0.0.0.0 192.168.10.254

R1(config)#int e0/0
R1(config-if)#ip address 192.168.10.1 255.255.255.0

Configuration on R2..

R2(config)#ip route 0.0.0.0 0.0.0.0 192.168.23.3
R2(config)#ip route 192.168.10.0 255.255.255.0 192.168.20.254

R2(config)#int e0/0
R2(config-if)#ip address 192.168.20.2 255.255.255.0
R2(config)#int e0/1
R2(config-if)#ip address 192.168.23.2 255.255.255.0

Configuration on R3..

R3(config)#ip route 0.0.0.0 0.0.0.0 10.1.1.2
R3(config)#ip route 192.168.10.0 255.255.255.0 192.168.20.2

R3(config)#int e0/0
R3(config-if)#ip address 10.1.1.10 255.255.255.0
R3(config-if)#ip nat inside
R3(config)#int e0/1
R3(config-if)#ip address 192.168.23.3 255.255.255.0
R3(config-if)#ip nat outside

R3(config)#access-list 10 permit any
R3(config)#ip nat inside source list 10 interface Ethernet0/0 overload


What happens when I send ICMP packets from R1?

Let's now ping 8.8.8.8 from R1 and see how it is displayed in Log Viewer..
(click on the image to view in full readable size)




Connection is built from source of 192.168.10.1 to the destination of 8.8.8.8
Connection is Teared down from source of 8.8.8.8 to the destination of 192.168.10.1
You can see that the port numbers are 0 because ICMP works at network layer.
However there can be a source port number sometimes for the 192.168.10.1 from the next time you send traffic.

Only these 2 lines will be displayed even for 1000 packets sent continuously because an ASA session is created for a flow, not to a single packet..

What happens when I send TCP packets from R1?

Let's try to telnet 8.8.8.8 from R1..




Connection is built from source of 192.168.10.1 to the destination of 8.8.8.8
Connection is Teared down from source of 8.8.8.8 to the destination of 192.168.10.1
You can see that the destination port number is 23 on google.

What happens if an ACL blocks the outgoing traffic on path?

R3's e0/0 is configured with an ACL as following..

R3(config)#access-list 20 deny any
R3(config)#int e0/0
R3(config-if)#ip access-group 20 out

I am pinging 8.8.8.8 from R1. 5 denies imply the 5 icmp packets generated from R1.







Connection is built from source of 192.168.10.1 to the destination of 8.8.8.8
Next 5 logs say it is denied from 192.168.23.3 which is the firewall side interface of R3
Final log says Connection is Teared down from source of 8.8.8.8 to the destination of 192.168.10.1 which means that tearing down log does not really say its a packet originated from 8.8.8.8, It is built from the ASA..

What happens if an ACL blocks the incomming traffic on path?

Now I removed the access-group command on e0/0 interface and applied it in e0/1 interface on R3.







It is the same as before. That means if it is blocked from an ACL on any interface on a router in the path, that router will send the deny log from its firewall side interface.

Is this "Deny" log comes just because it is denied from an ACL?

Well let's findout, I just deleted the ACL 20 from R3 router and shut the e0/0 interface of OUT.

It looks like the "Deny" log is not just saying it is blocked by an ACL. It just says it is unreachable..

But if you looked closer you will notice that the details say Type 3 Code 13 for the ACL blocking instances & it says Type 3 Code 1 when the interface is shut.

Let's see what are those, (source from iana.org)




























Ah, now you can say what has really happened ryt? It's a Administration Prohibit when it is blocked by an ACL and it is a destination unreachable when the e0/0 is shut.

If that was a TCP session, how will it be displayed?

Well, let's telnet to 8.8.8.8 while e0/0 of R3 is down..






It looks almost same. Denied packets will display an ICMP error & the connection will be displayed as a TCP connection.

What will be displayed if  the firewall side interface is down on a router?

Now let's see what will happen when the e0/1 interface of R3 is down..

Well, you don't see any issue. It looks exactly like it is working correctly. But It is not.

Let's continue the lab..




What will happen if a route is missing to the incoming path?

Let's findout, I removed the ip route 192.168.10.0 255.255.255.0 192.168.23.2 route from R3.

Well it looks like working too. But really it is not.. Let's continue with another scenario before we come to a conclusion..

What will happen if a route is missing to the outgoing path?

I removed the ip route 0.0.0.0 0.0.0.0 10.1.1.2 route from R3.







It is just same as scenario which the e0/0 of R3 was down. A destination unreachable from the firewall side interface of R3.

So it looks like  you can't really say it is working just by looking the log viewer. Log will not say anything if the firewall side interface of a router is down or a route to the firewall side is missing in the path.. Aslo the final tearing connection down log will be sourced from the real destination which the connection was to be initiated even though it was a failure, that one is just a firewall log (does not tell it is a packet came from a real hot)

Saturday, September 23, 2017

Port Priority based Root Port Election in STP

This is a concept which is misunderstood by many people. It is really because of the topologies they use to learn this concept.

When the switch has two interfaces connecting to the same switch, and the cost to reach the root bridge is the same it will use the interface with the lowest number as the root port..

For an example, let's say SW-A has the lowest MAC address hence SW-A will be the Root..

What will be the Alternate Port here?
Is it the e0/1 of SW-B?
Yes

By just looking at the port numbers, we can say that..
What will be the Alternate Port here?
Is it the e0/3 of SW-B?
No
E0/1 of SW-B will be the Alternate Port..




This is where most people will answer incorrectly.. The port priority which is considered is not the port priority of the of the SW-B. It is the port priority of the BPDU sender. Which in this case is the Root Bridge (SW-A).. Sender's port priority is what matters..

If we look at the output of show spanning-tree; (I use actual hardware from here)
































Because the priority is 128 default which is equal in all ports it has boiled down to the port number.. Lower port number will be the root port..

Let's change the port priority manually on SW-A..

SW-A(config-if)#int fa0/2
SW-A(config-if)#spanning-tree port-priority 16

































Now you can see the Alternate Port is changed to Fa0/3 of SW-B..

Additional Note:-
Decision making process of STP is like the following..

(1) Lowest bridge ID: the switch with the lowest bridge ID becomes the root bridge.
(2) Lowest path cost to Root Bridge: when the switch receives multiple BPDUs it will select the interface that has the lowest cost to reach the root bridge as the root port.
(3) Lowest sender bridge ID: when a switch is connected to two switches that it can use to reach the root bridge and the cost to reach the root bridge is the same, it will select the interface connecting to the switch with the lowest bridge ID as the root port.
(4) Lowest sender port ID: when the switch has two interfaces connecting to the same switch, and the cost to reach the root bridge is the same it will use the interface with the lowest number as the root port.

Tale of 2 BPDUs & Their Propagation in Classic STP

There are 2 types of Bridge Protocol Data Units (BPDUs)

(1) Configuration BPDUs
(2) Topology Change Notification BPDUs (TCNs)

Using these 2 types of frames, STP does it's everything..

A BPDU frame has the following format..






















Let's see a normal operation..

Configuration BPDUs are generated from the Root Bridge (Root Switch) and flow outward along the active paths and move away from the Root Bridge. Which means Non-root switches will receive these from their Root ports & Block ports and forwards out only from their Designated ports..

Each designated switch changes the Root Path Cost, Bridge ID & Port ID of the BPDU they receive before they send them downstream..

Following image shows the propagation of the Configuration BPDUs in a switching network..





















Let's see what happens when a link goes down..

For example, let's assume a link on S-5 (not the link connected to S-3) goes down.
Here is where the TCNs (PBDUs which can go upstream) were introduced.

TCNs are generated normally from Non-root switches and flow upstream towards the Root Bridge to inform the Root Bridge that the network topology has changed. Which means Non-root switches will forward out these only from their Root ports and receive only from their Designated ports..

The TCN is a very simple BPDU that contains absolutely no information that a bridge sends out every Hello Time seconds (this is locally configured Hello Time, not the Hello Time specified in configuration BPDUs which were set by the Root Bridge).

The designated bridge acknowledges the TCN by immediately sending back a normal configuration BPDU with the topology change acknowledgement (TCA) bit in Flag field set. The bridge that notifies the topology change does not stop sending its TCN until the designated bridge has acknowledged it.

Note that TCA is not a new BPDU.. It's just a Configuration BPDU with a bit changed..

Propagation of TCNs are indicated in red and the propagation of TCAs are indicated in green..





















Once the TCNs hit the Root, it also acknowledges the designated switch and it starts to send out its Configuration BPDUs with the topology change (TC) bit set.

These BPDUs are relayed by every bridge in the network with this bit set. As a result all bridges become aware of the topology change situation and it can reduce its CAM table Aging Time to Forward Delay.

Bridges receive TC BPDUs on both forwarding and blocking ports because they are actually Configuration BPDUs.

TC BPDUs are propagated like the following way.. They are also not a new BPDU type. Just the Configuration BPDUs with a bit change.





















These TC BPDUs are sent for a period of Max Age + Forward Delay seconds, which is 20+15=35 seconds by default. Here is why..

We deal with 4 timers in STP..

(1) Hello Time
(2) Max Age
(3) Forward Delay
(4) Aging Time

You can see those 4 timers in  the show spanning-tree output..

















Hello Time is the time frequency which the Configuration BPDUs are sent.. Default is 2s.
Max Age is the time which a BPDU is considered valid by a switch.. Default is 20s.
Forward Delay is the time which it takes to move from Listening State to Learning State and Learning State to Forwarding State.. Default is 15s.
Aging Time is the time which a CAM table entry will be valid.. Switches flush the MAC Address entry from it's CAM table after this time expires.. Default is 300s.

The reason for the TC BPDUs are sent for a period of Max Age + Forward Delay is that it will force the switch to flush the old remembered BPDUs and clear the MAC address table.

You can see the CAM Aging Time changes to Forward Delay on show spanning-tree output by entering the following debug command and unplugging a port of a switch..
S-1-ROOT#debug spanning-tree events




















Note:- You will not see the receipt of TC BPDUs on debug, but all the switches in the network will receive them silently and change their timers and relearn the MAC addresses..

We found 4 BPDUs in the above STP process.

(1) Config BPDU
(2) TCN
(3) TCA
(4) TC

But there are actually only 2 types which are Config BPDUs & TCNs. TCAs & TCs are just Config BPDUs with only a bit changed in the Flag field of the frame..