Part-2- Enterprise Internet Multi-Homing Design – Technical Design Considerations

Part-2- Enterprise Internet Multi-Homing Design – Technical Design Considerations

After you have obtained enough information to help you decide what are the business and applications’ needs and drivers (refer to Part-1 of this blog series for more info), it’s time to look into the technical considerations that may impact the desired outcome and expectations from the business, some of which may not very obvious at the design phase.

Commonly the top-down design approach is used to create business driven design. However, in this type of scenario, I’d recommend you to use both (Top-Down and Bottom-Up) design approaches. This will simplify the creation of a reliable enterprise multi-homing solution in a business driven approach that takes into consideration the unforeseen technical aspects.

You might be wondering, how such an approach can simplify the design, specially it is not something commonly recommended or used in design papers or books (where both top-down and bottom-up used together). Let’s find out how 🙂

Part-1 of this blog series discussed the different business drivers and application(s) requirements that influence the design decision, whether to go with a single-homed, multi-homed or dual-homed connectivity model (top-down). Next you need to look at the technical aspects that may impact achieving the intended design goal(s). This includes, physical connectivity considerations, routing protocol selection and routing policies design (bottom-up), which may influence your choices as well. As illustrated in the figure below, you can adopt both of the common design approaches ‘top-down and bottom-up’ to ensure you are able to achieve the desired result ‘reliably’.


Physical Connectivity

Let’s start from the bottom and look at the physical connectivity considerations. Multi-homing to a single or two different ISPs is a connectivity model that aims primarily to overcome single point of failure, by having two diverse links and possibly Internet providers. Practically, in many scenarios when you have a deeper look at the physical connectivity and optical layout etc. from the customer site to the Internet provider(s), you may notice that both links of the same or different providers are either; Sharing the same access POP premises or, using the same optical ring provided by a local service provider within the geographical area to provide the last mile connectivity!

In other words, fate-sharing in this scenario can be a consequence of any or a combination of the following:

  • Fiber links that share the same fiber conduit, or it could be the optical channels that share the same fiber.
  • Customers links connect to the same access/aggregation switch, POP or last mile provider.
  • Network devices share same power source.

Consequently, even though when you have two different edge routers, links and Internet providers, the last mile provider is the single point of failure.


Some network architects suggest to use completely different physical transport mediums like wired and wireless to guarantee ‘to a higher degree’ there is no fate sharing.

The logical question someone may ask from design point of view; “What about the SLA of these different transports? will wireless or microwave offer an equivalent bandwidth, latency and level of reliability to wired/fiber based transport?”. We all know the answer is No. Nevertheless, whether to select two different links with different SLAs and capabilities or not, this is something must be based on your applications’ requirements. For instance, will your application(s) be able to tolerate higher latency and lower bandwidth or not.

Alternatively, you can ask your provider(s) to share with you a certificate or any type of legal paper to confirm that your links are provisioned over completely diverse paths.

From my experience, sometimes, it’s easier to obtain such paper or official confirmation if you are buying your service from a single provider (end to end), because they know their optical layout, the more providers you deal with, the harder you can obtain such paper or an official confirmation.

Routing Protocol

Routing protocol selection is the simplest part, if you know what you will build in terms of connectivity model (single vs. multi-homed/dual-homed) and what your applications requirements (refer to part-1, applications requirements section) these requirements will tell you if you need to distribute your traffic load on the available links, inbound or outbound etc. or only you need simple internet connectivity.

Generally speaking, if you have, single link to an ISP, without any future plan to add a new link, static default route is the best and simplest option in this scenario.

On the other hand, if you have two internet links, I would recommend you to consider BGP even if you are not using any special policies today. This will give you the flexibility in the future to easily optimize your traffic flows and path selection when you need it.


What if you decide to use multi-homing connectivity over two different physical transport, fiber and 3G/4G, also, you will need BGP to control path selection, however, the ISP does not support BGP peering over 3G/4G?

Here you will realize the benefit of combining the bottom-up design approach along with the top-down design approach, because this may result in changing the link type or the Internet provider to achieve your goal!

Routing Polices

After understanding the business drivers, goals, needs, etc. along with applications requirements, as a design engineer or network architect you need some sort of tools to map these different requirements into a workable solution.

Typically, when you are designing and implementing enterprise Multi-homing, BGP attributes are your powerful tools to control and influence BGP/routing path selection.


As I mentioned in part-1 of this blog series “If you do a quick google search you will find a lot of blogs and vendors’ papers discussing how to configure BGP and how to steer the traffic over the different paths”. However, if you are unaware of how the Internet routing (or routing between ISPs) works, this can be a tricky (not complex) task to achieve. Let’s consider the below hypothetical scenario:

  • ISP-A owns AS 105 and ISP-B owns AS 106
  • Customer-X with AS 65001 is multi-homed to both ISPs, AS 105 and AS 106
  • ISP-B allocated the public IP range (PA) to Customer-X
  • Customer-X advertised this range over both links toward both ISPs

Customer-X goal is use upstream AS 106 as the primary path for both inbound and outbound traffic, and AS 105 as the back path.


Part-3 of this blog series will discuss the routing policies design considerations of this scenario, in the meantime, based on the provided information, do you see any issue(s) to achieve Customer-X traffic routing goal? If yes, what and why?

Marwan Al-shawi – CCDE No. 20130066 – is a Cisco Press author (author of the Top Cisco Certifications’ Design Books “CCDE Study Guide & the upcoming CCDP Arch 4th Edition”). He is Experienced Technical Architect. Marwan has been in the networking industry for more than 12 years and has been involved in architecting, designing, and implementing various large-scale networks, some of which are global service provider-grade networks. Marwan holds a Master of Science degree in internetworking from the University of Technology, Sydney. Marwan enjoys helping and assessing others, Therefore, he was selected as a Cisco Designated VIP by the Cisco Support Community (CSC) (official Cisco Systems forums) in 2012, and by the Solutions and Architectures subcommunity in 2014. In addition, Marwan was selected as a member of the Cisco Champions program in 2015 and 2016.


  • Mohamed Nidhal Baeyrem Jaziri (3xCCIE # 38232) says:

    Excellent post Marwan. However I have 2 minor remarks:
    1 – You said at the end of the article (in the BGP table) that BGP local preference can be used to influence inbound and outbound traffic flows. I think that it can only influence outbound traffic because it is a nontransitive BGP path attribute.
    2 – There is a typo at the end of the article: ISP-A owns AS 105 (not AS 106) and ISP-B owns AS 106.

    Concerning the question that you asked at the end, I think that Customer-X can easily achieve the outbound traffic goal (by using BGP weight or local preference attributes) however it will not achieve the inbound goal. In fact, given that ISP-B owns the IP addresses block from which they allocated a subnet to Customer-X, they most likely will advertise the aggregation of this block to upstream ASes (300 and 600), therefore the return traffic will flow through the more specific route (longest match rule) which is advertised by ISP-A, and hence Customer-X will not achieve their goal.
    Thanks !

    • Marwan Al-shawi says:

      The table in the blog only highlight the attributes that commonly used as tools to control path selection. Local_Preference also mentioned to be used within the same AS. This might be useful in scenarios where you have few core routers behind the internet edge and you need to do some traffic engineering for the inbound traffic from the edge routers to different internal networks (within the same AS). Is this the best option or attributed to be used? This is something different and depends on different factors which are out of the scope of this blog, hope this clarified it to you 🙂

  • Amit Temkar says:

    Customer X received public IP from ISP B, so it can’t advertise the same to ISP A.
    Customer X should have public IP from APNIC/ARN to get the same advertised to both ISPs. Also I believe customer X should also have public AS for this,
    Then customer X can play with BGP attributes to have local preference to prefer ISP B for outbound and AS prepend to have ISP B for inbound.
    Waiting for blog 3 for this clarity.

Leave a Comment