Communication through Multiple Switches

In a previous article we demonstrated how a single switch functions. One of the most frequent requests I receive is asking how the process would be different if there were multiple switches. This article will illustrate the process using two switches.

In the Packet Traveling series, we discussed the four specific functions of a switch: Learning, Flooding, Forwarding, and Filtering; we then illustrated each function in an animation that showed two hosts communicating through a single switch.

With multiple switches, each switch will still independently perform the exact same four functions. The process does not change, it is simply replicated separately by other switches.

We will illustrate how data moves between multiple switches using the following topology:

hssh-topology

Our topology has two switches, and each has their own, independent MAC address table — the information in the MAC address tables are never shared.

Host A and C are connected to ports 1 and 2 of the blue switch on the left. Host B and D are connected to ports 5 and 6 of the green switch on the right. Port 3 on the blue switch is connected to port 4 on the green switch.

We will illustrate each step that occurs for each of the following:

 

Host A to Host B

It starts with Host A having a frame to deliver to Host B. The contents of the frame are irrelevant, it could be an ICMP (ping) packet, it could be an ARP packet, or it could be other data.

The Layer3 header would include a Source IP address of 10.0.0.11 (Host A) and a Destination IP address of 10.0.0.22 (Host B).

The Layer2 header would include a Source MAC address of aaaa.aaaa.aaaa and a Destination MAC address of bbbb.bbbb.bbbb. The switches will use the information in the Layer2 header to move the frame between the two hosts.

To begin, the MAC address tables for both switches will be empty. They will populate as the switches learn of each device connected to each port by reading the Source MAC address field of each received frame.

Communication through Multiple Switches - Host A to Host B

When the frame arrives on the blue switch, the first thing that happens is the blue switch learns the MAC address aaaa.aaaa.aaaa exists on port 1. Then, since the blue switch does not yet have an entry in tis MAC address table for bbbb.bbbb.bbbb, the frame is duplicated and flooded out every port.

The frame arrives on Host C, who will inspect the frame and realize it is not the intended recipient. Host C will silently discard the frame.

The frame will also arrive on the green switch. Just like the other switch, the first thing the green switch will do is learn that it received a frame on port 4 with a source MAC address of aaaa.aaaa.aaaa. And again, just like the other switch, the green switch does not know where the MAC address bbbb.bbbb.bbbb exists, so the frame will again be duplicated and flooded out each switch port.

Notice in both cases, the frame was flooded out each port, except the port it was received on. This is an example of a switch’s filtering behavior. This behavior prevents a switch from sending a frame out the same port it was received.

Host D will receive the frame, and silently discard it since the frame was not addressed to Host D.

Host B will receive the frame and accept it for processing, since Host B was the intended destination..

 

Host B to Host A

On the way back things will go a little simpler. The switches have already learned about some of the connected devices, and that should alleviate some of the additional flooding that was required for the initial communication in the previous section.

Specifically, both switches know the location of the MAC address aaaa.aaaa.aaaa – port 1 on the blue switch and port 4 on the green switch. Each switch learned the location independent of the other; there was no communication between the switches or sharing of MAC address tables.

In the response frame sent by Host B to Host A, the Layer2 header will have a Source MAC address of bbbb.bbbb.bbbb and a Destination MAC address of aaaa.aaaa.aaaa.

Communication through Multiple Switches - Host B to Host A

The response frame will first arrive on the green switch on port 6. Therefore, the green switch will learn that the MAC address bbbb.bbbb.bbbb exists out port 6. The green switch then consults its MAC address table to determine that the frame destined to aaaa.aaaa.aaaa should be forwarded out port 4.

The response frame then arrives on the blue switch on port 3. Therefore, the blue switch will learn the MAC address bbbb.bbbb.bbbb exists out port 3. The blue switch then consults its MAC address table to determine that the frame destined to aaaa.aaa.aaaa should be forwarded out port 1.

Which will finally get the response frame back to Host A.  Notice on the way back no flooding was required. Both switches knew the location of the destination MAC address of the frame.

 

Communication with populated MAC Address Tables

Finally, with the both switches’ MAC address tables fully populated, communication between Host A and Host B resembles the following:

Communication through Multiple Switches - Populated MAC address Tables

Each time a frame is received, the switch first attempts to learn the MAC address mapping on the receiving switch port. If the mapping is already known, it is simply refreshed in the MAC address table.

Notice, Host C and D receive none of the frames sent between Host A and B. The switch is able to create an isolated path for the data between these two hosts (so long as the MAC address tables remain populated).

This is one of the benefits of upgrading to a Switch from a simple Hub. With a Hub, every frame is flooded out every port, every time. Whereas with a switch (or a “smart hub”, as they are sometimes referred to) only the first few frames will be flooded, but all remaining communication between two hosts is confined to only those two hosts.

 

All Hosts

At some point in time, all hosts will have sent some frames, providing both switches the opportunity to learn the location of each MAC address in the topology above. At that point, the switch MAC address tables will resemble the image below:

hssh-all-hosts

The key item to note is that each switch port can learn of multiple MAC addresses. Notice, from the blue switch’s perspective, the location of Host B and D is out port 3. Moreover, from the green switch’s perspective, the location of host A and Host C is out port 4.

 

Full Process

Below is the entire process of Host A and Host B communicating, from beginning to end, and including the ability to pause/forward/rewind the animation. If you’ve read through this article and understand each step described above, you can use this animation to study. Or even better, to explain the process to someone else.

If the animation above did not load, try this link
Tags:
0 0 votes
Article Rating
Subscribe
Notify of
35 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

Your posts are inspiring! I really enjoy this blog and wish I came across it many years ago when I was doing my CCNA! I’m for sure adding it to my bookmarks!

Hi Ed,
I am just ramping up on basic networking concepts.
I came across this website today and packet traversal article and it was very good to understand. Thanks for taking time to share your knowledge.
Planning to read the VLAN article next.
Can you please let me know if any write up or any details on L2 extension, LAG concepts?
Thanks,
Bala

U rock buddy. U just solved all my doubts about how the data flows..
U r best in ur profession

Hi

The article was much more informative than what I have ever read about switches before 🙂
But I am left with a single doubt about communication through multiple switches. Referencing above given terminologies, say blue switch has about 90 different devices connected excluding the green switch, same is the case with green switch.
Now each time data is communicated from blue switch to green switch or vice versa, mac address table of each switch will store only one mac address of the devices connected to other switch. Thus each time if destination mac address happen to be one of 89 devices of another switch whose mac address is not stored, the switch will end up flooding.
Thus at any point of time mac address table will store only one mac address of device connected to the other switch!
Is my understanding correct or do I need to add something else to it? Please let me know If I was unable to convey my thoughts as I am not as good as you are at this.

Hey Ed Harmoush

Your posts are so simple to understand that it has made my many concepts clear. Thanks alot for that
I have a doubt or if you can make it clear it will be great help

This is more related to Virtual environment I understand packet will follow as below

VM1 > vSwtich > ESX1NIC > PSwtich > ESX2NIC > vSwtich > VM2

Both VM are in same Subnets and a single physical swtich is connected to both ESXi’s

VM1 packet will contain

VM1 Source MAC
VM1 Source IP
VM2 Destination IP
VM2 Source IP

Process of packet flow :

1. As the vswtich does not find entry in its MAC cache , it forwards the traffic to pSwtich
2. pSwtich will have MAC entry of the destination VM2 which will be mapped to the port of the ESX2NIC
3. pswtich will forward the traffic to vSwtich and vSwtich will lookup its MAC Cache and the forward the packet to destination VM2

Questions :

1. Is my above understanding correct ?
2. will Physical Swtich have MAC entry of multiple VM’s residing on vSwtich mapped to a single physical port ?

Thank u man very simple explain, easy to understand best with you a lot of luck

Ed,

Wow, you have a gift sir! Never have I seen such complex topics covered so clearly and simply. Well done!
I do have one question though. Let’s say that Host A has to get moved, for whatever reason, to port 7 on the blue switch. The blue switch still believes Host A is on port 1. How does the switch determine that the host has been moved and update it’s MAC table? From what I understand it doesn’t receive any confirmations once it’s routed the packet.

Many thanks

Steve

hello

thank you very much for explanation ,now all is clear for me

Hi.
I like the way you are explaining the concept.
I would like to ask do you have any Application of this tutorial in Play store.

Thanks.

Great series, thank you so much for it. I need to ask the following – in the first frame from host A to host B, why does host A know the destination IP and destination MAC already? I’m confused because both switches have to flood to learn it, yet the host already knows both of these pieces of information. I’m sure I’ve missed something obvious so would be grateful if you pointed it out.

Just reading around through other parts of your site – I believe you have simplified this for teaching purposes and actually the first frame from host A will be a broadcast with an incomplete layer 2 header?

Hi ,

Thank you for the concepts explained here .I have a quick doubt , the first packet which goes out of the host would be an ARP packet right ? As , its ARP cache is empty .
so switch will flood the ARP request based on the L2 .

can you please clarify ?

Nice article, Ed. Does a switch always flood a broadcast ARP packet? To scale it further. Does each switch in the network flood every broadcast ARP packet (in turn, each host in the network receiving the ARP request)?

This might sound silly, will the switches need any trunk link to be able communicate between each other or it works just by connecting an Ethernet cable between the switch ports?

I have three queries. First is: A switch has many ports, do these ports have their own MAC addresses? Second: when there’s a broadcast frame to flood, does a switch first decide the method of forwarding n then flood? Third: u said every time when a switch receives a frame it checks d src MAC address, if it is already known it refreshes its Mac address table. What does it mean?

What my 2nd query is, is that a switch has 3 methods of forwarding a frame, right? So when it has to flood a broadcast frame, like ARP request, will switch use one of the 3 methods of forwarding to flood it. Store n forward, cut through or fragment free?
N I have to ask you on point C, if I’m creating a LAN with a Switch in it obviously, why would I unplug or plug d devices after every 5 mins, right? No one would do that, so y d people who standardized how a Switch should work would program it to keep refreshing its MAC table after every 5 mins? N what happens after 5 mins if it doesn’t receive a frame from a particular port? Does it delete that entry from its mac table?

Hi Ed,

I wanted to clarify something to make sure I understand correctly.

When host A is sending data to host B and it reaches the green switch, it does not know where bbbb.bbbb.bbbb is and floods the packet to all ports. Host B responds saying this is my MAC and receives the packet. Wouldn’t this add host B’s MAC address to the MAC address table of the green switch?

So in the subsequent communication from host B to A, the green switch would not have to learn the MAC address and would already have it in the MAC address table

Hello Ed, you are my life-saver. Thank you so much for this good article.

This is the most practical and in-depth description of what happens when two switches are connected that I’ve ever read.
Excellent article, now my confusion is resolved.

Hi Ed, I have two questions if you can answer me. 1st QUESTION: Computer A communicating with computer on the same Switch. If “Computer A” wants to talk to “Computer C” the blue Switch will flood to everyone who is connected to the blue Switch, it even goes to port 3, and the green Switch will not let through to Computers “D” and ” B”?
2nd QUESTION: Blue Switch does not know the MAC of Computer B of the green Switch, but the green Switch has the MAC of Computer B in the table. Assuming that the blue Switch table is empty and the green Switch table already has the Computer MAC “B”. When Computer A sends a frame with destination MAC FF:FF:FF:FF:FF:FF that goes to all devices on the blue Switch and when the frame arrives on the green Switch the frame is forwarded directly to Computer B? If the green switch already knows where the MAC address bbbb.bbbb.bbbb exists, then of “Computer A” frame will not be flooded on each port of the green switch, even if the “Computer A” frame has the “L2 header” the destination MAC “FF:FF:FF:FF:FF:FF”?