post

Communication through Multiple Switches

In a previous article we demonstrated how a single switch functions. One of the most frequent requests I receive is asking how the process would be different if there were multiple switches. This article will illustrate the process using two switches.

In the Packet Traveling series, we discussed the four specific functions of a switch: Learning, Flooding, Forwarding, and Filtering; we then illustrated each function in an animation that showed two hosts communicating through a single switch.

With multiple switches, each switch will still independently perform the exact same four functions. The process does not change, it is simply replicated separately by other switches.

We will illustrate how data moves between multiple switches using the following topology:

hssh-topology

Our topology has two switches, and each has their own, independent MAC address table — the information in the MAC address tables are never shared.

Host A and C are connected to ports 1 and 2 of the blue switch on the left. Host B and D are connected to ports 5 and 6 of the green switch on the right. Port 3 on the blue switch is connected to port 4 on the green switch.

We will illustrate each step that occurs for each of the following:

 

Host A to Host B

It starts with Host A having a frame to deliver to Host B. The contents of the frame are irrelevant, it could be an ICMP (ping) packet, it could be an ARP packet, or it could be other data.

The Layer3 header would include a Source IP address of 10.0.0.11 (Host A) and a Destination IP address of 10.0.0.22 (Host B).

The Layer2 header would include a Source MAC address of aaaa.aaaa.aaaa and a Destination MAC address of bbbb.bbbb.bbbb. The switches will use the information in the Layer2 header to move the frame between the two hosts.

To begin, the MAC address tables for both switches will be empty. They will populate as the switches learn of each device connected to each port by reading the Source MAC address field of each received frame.

Communication through Multiple Switches - Host A to Host B

When the frame arrives on the blue switch, the first thing that happens is the blue switch learns the MAC address aaaa.aaaa.aaaa exists on port 1. Then, since the blue switch does not yet have an entry in tis MAC address table for bbbb.bbbb.bbbb, the frame is duplicated and flooded out every port.

The frame arrives on Host C, who will inspect the frame and realize it is not the intended recipient. Host C will silently discard the frame.

The frame will also arrive on the green switch. Just like the other switch, the first thing the green switch will do is learn that it received a frame on port 4 with a source MAC address of aaaa.aaaa.aaaa. And again, just like the other switch, the green switch does not know where the MAC address bbbb.bbbb.bbbb exists, so the frame will again be duplicated and flooded out each switch port.

Notice in both cases, the frame was flooded out each port, except the port it was received on. This is an example of a switch’s filtering behavior. This behavior prevents a switch from sending a frame out the same port it was received.

Host D will receive the frame, and silently discard it since the frame was not addressed to Host D.

Host B will receive the frame and accept it for processing, since Host B was the intended destination..

 

Host B to Host A

On the way back things will go a little simpler. The switches have already learned about some of the connected devices, and that should alleviate some of the additional flooding that was required for the initial communication in the previous section.

Specifically, both switches know the location of the MAC address aaaa.aaaa.aaaa – port 1 on the blue switch and port 4 on the green switch. Each switch learned the location independent of the other; there was no communication between the switches or sharing of MAC address tables.

In the response frame sent by Host B to Host A, the Layer2 header will have a Source MAC address of bbbb.bbbb.bbbb and a Destination MAC address of aaaa.aaaa.aaaa.

Communication through Multiple Switches - Host B to Host A

The response frame will first arrive on the green switch on port 6. Therefore, the green switch will learn that the MAC address bbbb.bbbb.bbbb exists out port 6. The green switch then consults its MAC address table to determine that the frame destined to aaaa.aaaa.aaaa should be forwarded out port 4.

The response frame then arrives on the blue switch on port 3. Therefore, the blue switch will learn the MAC address bbbb.bbbb.bbbb exists out port 3. The blue switch then consults its MAC address table to determine that the frame destined to aaaa.aaa.aaaa should be forwarded out port 1.

Which will finally get the response frame back to Host A.  Notice on the way back no flooding was required. Both switches knew the location of the destination MAC address of the frame.

 

Communication with populated MAC Address Tables

Finally, with the both switches’ MAC address tables fully populated, communication between Host A and Host B resembles the following:

Communication through Multiple Switches - Populated MAC address Tables

Each time a frame is received, the switch first attempts to learn the MAC address mapping on the receiving switch port. If the mapping is already known, it is simply refreshed in the MAC address table.

Notice, Host C and D receive none of the frames sent between Host A and B. The switch is able to create an isolated path for the data between these two hosts (so long as the MAC address tables remain populated).

This is one of the benefits of upgrading to a Switch from a simple Hub. With a Hub, every frame is flooded out every port, every time. Whereas with a switch (or a “smart hub”, as they are sometimes referred to) only the first few frames will be flooded, but all remaining communication between two hosts is confined to only those two hosts.

 

All Hosts

At some point in time, all hosts will have sent some frames, providing both switches the opportunity to learn the location of each MAC address in the topology above. At that point, the switch MAC address tables will resemble the image below:

hssh-all-hosts

The key item to note is that each switch port can learn of multiple MAC addresses. Notice, from the blue switch’s perspective, the location of Host B and D is out port 3. Moreover, from the green switch’s perspective, the location of host A and Host C is out port 4.

 

Full Process

Below is the entire process of Host A and Host B communicating, from beginning to end, and including the ability to pause/forward/rewind the animation. If you’ve read through this article and understand each step described above, you can use this animation to study. Or even better, to explain the process to someone else.

If the animation above did not load, try this link

Comments

  1. Your posts are inspiring! I really enjoy this blog and wish I came across it many years ago when I was doing my CCNA! I’m for sure adding it to my bookmarks!

  2. Hi Ed,
    I am just ramping up on basic networking concepts.
    I came across this website today and packet traversal article and it was very good to understand. Thanks for taking time to share your knowledge.
    Planning to read the VLAN article next.
    Can you please let me know if any write up or any details on L2 extension, LAG concepts?
    Thanks,
    Bala

    • Hi Bala, glad you enjoyed the articles! I hope you enjoy the VLAN articles as well (which are all available here: pracnet.net/vlans.

      I don’t presently have articles in the pipeline regarding LAG and Etherchannels, but the best way to keep up to date is to sign up to get notified of udpates at the bottom of every page.

  3. U rock buddy. U just solved all my doubts about how the data flows..
    U r best in ur profession

  4. Hi

    The article was much more informative than what I have ever read about switches before :-)
    But I am left with a single doubt about communication through multiple switches. Referencing above given terminologies, say blue switch has about 90 different devices connected excluding the green switch, same is the case with green switch.
    Now each time data is communicated from blue switch to green switch or vice versa, mac address table of each switch will store only one mac address of the devices connected to other switch. Thus each time if destination mac address happen to be one of 89 devices of another switch whose mac address is not stored, the switch will end up flooding.
    Thus at any point of time mac address table will store only one mac address of device connected to the other switch!
    Is my understanding correct or do I need to add something else to it? Please let me know If I was unable to convey my thoughts as I am not as good as you are at this.

    • Hi Rajan,

      You asked a great question. It is actually one I’ve gotten before, so I updated the article to speak to the situation.

      To answer your question though, each switch port can learn of multiple MAC addresses. So the switch port connected to the other switch will learn all 90 devices exist out that port. Any traffic destined to those MAC addresses will only be forwarded out the appropriate port, not flooded.

      Hope it helps.

  5. Saurabh Shah says:

    Hey Ed Harmoush

    Your posts are so simple to understand that it has made my many concepts clear. Thanks alot for that
    I have a doubt or if you can make it clear it will be great help

    This is more related to Virtual environment I understand packet will follow as below

    VM1 > vSwtich > ESX1NIC > PSwtich > ESX2NIC > vSwtich > VM2

    Both VM are in same Subnets and a single physical swtich is connected to both ESXi’s

    VM1 packet will contain

    VM1 Source MAC
    VM1 Source IP
    VM2 Destination IP
    VM2 Source IP

    Process of packet flow :

    1. As the vswtich does not find entry in its MAC cache , it forwards the traffic to pSwtich
    2. pSwtich will have MAC entry of the destination VM2 which will be mapped to the port of the ESX2NIC
    3. pswtich will forward the traffic to vSwtich and vSwtich will lookup its MAC Cache and the forward the packet to destination VM2

    Questions :

    1. Is my above understanding correct ?
    2. will Physical Swtich have MAC entry of multiple VM’s residing on vSwtich mapped to a single physical port ?

    • Hi Saurabh. Whether the switch is a virtual switch or physical switch it will operate the same — performing the same four switch functions: Learning, Flooding, Forwarding, and Filtering. Your topology of “VM1 > vSwtich > ESX1NIC > PSwtich > ESX2NIC > vSwtich > VM2” will work the same as if it were PC1 > Switch > Switch > Switch > PC2. In which case the same process as described in the article will occur. Hope this helps.

  6. Duong ho says:

    Thank u man very simple explain, easy to understand best with you a lot of luck

Speak Your Mind

*