small multihomed organizations

[Scott Richard Slater]

objective: "small multhomed organizations with the complete ability to
balance all inbound and outbound traffic over both connections".

I think there are a few issues here.  The major issue is "filtering".  We
need to adjust filtering to accomodate smaller multihomed organizations and
enable these smaller multihomed organizations to control/balance their
inbound and outbound traffic over each connection.

There are three methods of "filtering":
1) filtering using route information (access-list, distribute-list)
2) filtering using path information (ip as-path access-list, filter-list)
3) filtering using communities(ip community-list)

Once we identify how to filter with each method then we next determine if
any of the filtering methods accomodate our objective.  If none of the
filtering methods accomodates our objective then we should create a new
method for "filtering".

note:  There is another option I just thought of.  If the customer is
delegated address space from ARIN, and the inbound and outbound traffic is
not balanced.  The customer can use the "set as-path prepend" command to
balance the traffic over each connection.

Also, please note, that the filtering methods above can be used together.
You can filter using "community-list" and "ip as-path access-list" on the
same router bgp configuration.

Also, I was just thinking of what I wrote yesterday, about if one of the two
ISPs delegates address space to the customer then the delegating ISP will
most probably be the backup connection for the customer.  This can be
corrected using the BGP "backdoor" command.

Also, note that the customers configuration will absolutely be different
dependent on the number of BGP neighbors.  The more BGP neighbors chances
are the customer will use some BGP metrics to choose packet flow.

Just another thought,  its hard to develop an answer to adjust the present
"filtering" configurations when we do not know how every provider filters.

Another solution for the customer's 24 bit address space to be accepted by
providers is the following, if the provider uses prefix-lists.  This way the
customers specific address space is accepted, a mask length of up to 24 bits
in routes is accepted  EXAMPLE:  " ip prefix-list abc permit 0.0.0.0/8 le
24"  If the provider uses "le" they can specify the prefix length they will
permit or deny.

I am working on some solutions and drawings.

Scott

----- Original Message -----
From: "Scott Richard Slater" <scottslater77 at hotmail.com>
To: "Lee Howard" <lhoward at UU.NET>
Sent: Tuesday, August 07, 2001 7:52 PM
Subject: Re: small multihomed organizations


> some thoughts, see below.    I sort of said the same things you were
saying.
> I agree with you.  I am checking with some customers I worked with that
are
> mutli-homed.  I can not remember exactly who allocated their address
space.
>
>
> ----- Original Message -----
> From: "Lee Howard" <lhoward at UU.NET>
> To: "Scott Richard Slater" <scottslater77 at hotmail.com>
> Sent: Tuesday, August 07, 2001 3:15 PM
> Subject: Re: small multihomed organizations
>
>
> > The justification for an ASN is a routing policy that differs from
> > adjacent Autonomous Systems.  Multihoming is accepted as sufficient
> > difference.
>
> I agree.  I basically stated the same thing below. :)
>
> >
> >
> > On Mon, 6 Aug 2001, Scott Richard Slater wrote:
> >
> > > Date: Mon, 6 Aug 2001 18:54:07 -0400
> > > From: Scott Richard Slater <scottslater77 at hotmail.com>
> > > To: Lee Howard <lhoward at UU.NET>
> > > Subject: Re: small multihomed organizations
> > >
> > >
> > > Let me reiterate your example.  You are talking about a customer who
is
> > > multi-homed, that is connected to two ISPs.  First, the customer will
> > > require an Autonomous System(AS).   Then the customer will require a
> unique
> > > Autonomous System Number(ASN) from ARIN.  This ASN identifies the
> customer's
> > > network to the rest of the Internet through the Border Gateway
> Protocol(BGP)
> > > protocol.  BGP is implemented on the customer's router as well as the
> two
> > > ISPs routers.  Since the customer's AS is talking to both ISPs ASs
using
> > > BGP, this implementation is called External BGP(eBGP).
> > >
> > > With the basic design stated above, we must identify the customer's
> > > objective before we can determine more answers.
> > >
> > > More often than not, the customer is multi-homed to the two ISPs to
> obtain
> > > "redundancy" for their network.  In addition, most probably one ISP
> > > connection will be used strictly for backup.  If this is NOT the case
> and
> > > the customer wants to "balance" traffic over the two ISP connections
> this
> > > may be tricky, dependent on the ISPs BGP policies.
> >
> > I'm not sure that most multi-homing orgs use one only as a backup.  But
> > I agree that it's tricky to split use between two paths.  Outbound
> > balancing is easy (two equal static routes) but inbound balancing is
> > impossible between two differently-homed connections.  I find that most
> > customers are satisfied with "some" traffic going over each link.
>
> Both ISPs can statically insert the customers routes, but the customer
wants
> to use BGP to take full routes from both ISPs to make more intelligent
> routing decisions, and to be able to add routes without having to call the
> ISP.
>
> >
> >
> > > I do not think that a
> > > /24 (Class C) is justifiable solely due to a customer being
multi-homed.
> > > The customer should provide thorough justification of their AS. '
> >
> > Well, sure, I never said we should change the justification for ASNs.
> >
> > >  Which
> > > includes their network table, their number of hosts specific to
hardware
> > > (routers, firewalls, et cetera..), and their subnetting should be
> detailed.
> > > This is important for the ISP to look at.  The ISP may be able to
> suggest
> > > different network designs for the customer to conserve address space
> based
> > > on the network table the customer provides.  Normally, customer routes
> that
> > > are a /24 or greater are not filtered.
> >
> > Right, exactly.  So if a customer has 24 hosts, a T1 to me and a T1 to
> you,
> > what do we do?  If I give him a /27, nobody except you will take your
> path.
> > If you give him a /27, nobody except me will take my path.
>
> Kinda,  let's look at it like this.  If the customer receives address
space
> from me(example 138.39.1/24), normally I would then be the customers
backup
> connection, and you would announce the more specific route to the
customers
> address space and you would attract more traffic.  This is due to the fact
> that I would normally announce my larger aggregate address space(example
> 138.39/16).  Now, this is slightly fixable, if the delegating space
provider
> (example: me)announces the customers address space(example 138.39.1/24).
> Most delegating providers will announce their customers address space to
> their peers.  Some delegating providers want to keep their routing table
> small and not announce a customer prefix out of their aggregate address
> space.  This is an individual-case-basis. The other problem is that there
is
> no gurantee that any of my peers will except the customers address space,
> due to the peers routing policies (this is an ever-so-changing policy
> between peers - some peers filter at /19,/20,/24).
>
> So, the customer will not be satisfied if they receive address space from
> either me or you. Now, even if me and you give the customer address space
> (example a /24 from me to the customer & a /24 from you to the customer)
> there is still the problem of our peers filtering out our customers
specific
> /24 address space.
>
> Now, even if the customer is delegated a /24 from ARIN the customer will
> still encounter the peers filtering.  The only advantage of the ARIN
> delegation of address space to the customer is the customer will never
need
> to renumber their domains only their physical interfaces with the provider
> giving them the connections.
>
> So,  there is only a few options.  It is understandable that smaller
> organizations will not and do not receive the same benefits as
medium-large
> organizations.    Let me first state that "multi-homing with true
> load-blancing" (the objective we are trying to achieve) is a WANT from any
> customer with an internet connection.  Who wouldn't want redundancy and
more
> reliability and better performance.  BUT, every customer can not HAVE
> "multi-homing with true load-blancing".  It is the same as a customer who
is
> using a Cisco 2500 series router.  I am sure the customer would much
prefer
> using a Cisco 12000 series router but every customer can not justify.
>
> Maybe AS-based configurations will go away and community-based
> configurations with local-pref will be used throughout.   Though, I know
> every provider does not currently use community tags.  And there is still
> the chance of corruption using community tags as with not using filters.
>
> >
> >
> > > The key to finding a solution for this customer is to understand the
> > > customers objective first, next understand how many hosts does the
> customer
> > > actually need, third to understand the two ISPs BGP policies.  The
> reason I
> > > say this is that ISPs have very different BGP policies.  Some ISPs
> adjust
> > > their local-pref values.   The customer must also get a fine look at
the
> > > ISPs BGP 'import' AND 'export' policies.  Obviously, the customer will
> have
> > > their own routing decisions based on the ISPs performance and cost.
> >
> > I understand well the differences between different ISPs' routing
> policies,
> > but I don't understand how that will change what the ARIN policy should
> be.
> >
> > Maybe we should consider what customers want. . .
> >
> > CUSTOMER WANTS PRIMARY AND BACKUP
> > Even if most people use one ISP as primary and one as backup, any
failure
> > catastrophic enough to cause primary ISP to fail will prevent traffic
from
> > taking backup.  What I mean is, Customer connected to Primary Networks
and
> > Backup ISP Inc., gets a /27 from Primary's /16.  Great, all traffic
takes
> > Primary (except maybe traffic originating in Backup's network).  But if
> > Primary goes down, or some failure causes the /27 route to be withdrawn,
> > Unrelated ISP's traffic will still try to go through Primary, and get
> > blackholed.  If Customer is lucky enough that Backup peers with Primary,
> > and if Primary allows more-specifics of its own aggregates to be
announced
> > to it, then Primary may shuffle traffic to Backup in case of an outage,
> but
> > if case of Primary going completely down (or otherwise withdrawing the
> /16)
> > nobody will be able to reach Customer.  No redundancy is achieved.
> >
> > Looked at the other way, let's say Customer gets IPs from Backup.  Nope,
> > doesn't work--no Internet traffic will take Primary because nobody will
> hear
> > the /27.
>
> no static routing :)
>
> >
> > CUSTOMER WANTS LOAD BALANCING
> > Ain't no such animal.
> > OK, fine, you want some traffic to use each link, so you buy a
connection
> > from Big Mama Networking MegaTelco, and Joe Bob's Bait, Tackle & ISP.
You
> > get a /27 from Joe Bob, and you're barely reachable.  Maybe Joe Bob has
> > transit from somebody, maybe he has a few peers, or maybe not.  If Joe
> > Bob's aggregate block (/20 or greater) is globally routed, then fine,
you
> > have access.  Big Mama MegaTelco will send locally originated traffic to
> > you, too; it could be a significant amount of traffic.   But if Joe Bob
> > drops his pager while fishing, and there's an outage, Big Mama is the
only
> > ISP that will be able to reach you; everybody else will filter your /27.
> > It works the other way, too.  Sure, you have some traffic using each
link,
> > but you have almost zero redundancy if Joe Bob gets eaten by a bear.
Our
> > goal was load sharing, and we've got it, but it's less than most
customers
> > are looking for, in my experience.
>
> Customers must be very specific which provider connections they want and
> which provider connections they need.
>
> >
> > I think customers should reasonably expect transit to include global (or
> > at least nearly-global) routability.  The reason that doesn't exist for
> > longer prefixes now is that too many people make mistakes and leak all
> > of their internal routes.  So other people filter, to avoid hearing
> > those routes--they assume that all routes smaller than a /24 are
leakage.
> > I'm saying that not all of them are, and we should make it possible for
> > smaller customers to multihome.  How do we do that?
>
> just can't right now.  Due to everyone's filtering policies.  Unless there
> was some sort of "ARIN approval system" for non-aggregate address space.
> First we would need to determine what is aggregate address space. (Is it
/19
> or /20 or /24) and then every provider would have to adjust their filters
> accordingly.  Second, we would then need this "ARIN approval system" that
> grants a non-aggregate address block the same routing privledges that an
> aggregate address block has.  The only problem with this would be the
> adjustment on everyone's filters.  If filters did not exist then everyone
> would rely on faith of the customer.  AND we know customers make mistakes.
> The only solution I can think of is written above.
>
> >
> > Lee
> >
> >
>
> Scott
>
> > >
> > > Scott
> > >
> > > ----- Original Message -----
> > > From: "Lee Howard" <lhoward at UU.NET>
> > > To: <ppml at arin.net>
> > > Sent: Monday, August 06, 2001 3:42 PM
> > > Subject: small multihomed organizations
> > >
> > >
> > > >
> > > > We occasionally have customers who will be multihomed with us and
> another
> > > > provider, and who need address space, but can't justify a /24 based
on
> > > > usage and host count.  If we give them less than a /24, no traffic
> will
> > > > come down their other line. [1]
> > >
> > > If no traffic is coming down their "other line" the customer should
> check
> > > the BGP metrics set by both ISPs.  There are a few possibilities.  The
> ISP
> > > with the line that is receiving all the traffic may have adjusted
their
> > > local-pref values to the customers advertisements on the "other line".
> I
> > > think I said that right. :)
> > >
> > > I would also be curious of both ISPs peering connections and their
> > > relationships with other "tier one" ISPs, especially the two ISPs
> involved.
> > >
> > > >
> > > > ARIN's current policy, as I understand it, is that multihoming is
not
> > > > justification for a /24.  I would like to hear recommendations and
> best
> > > > practices from others for how to address the customer (pun intended)
> > > > without violating ARIN policy.  Should ARIN accept multihoming as
> > > > justification for a /24?
> > > >
> > > > Thanks,
> > > >
> > > > Lee Howard
> > > >
> > > >
> > > >
> > > >
> > > > [1] Here's why:  Most ISPs do not accept prefixes longer than 24
bits,
> so
> > > > they will ignore our propagation of our customer's /24, and they
will
> > > > ignore the route announced from the customer's other ISP.  Since we
> also
> > > > announce a large aggregate (say, a /13 from which that /24 was
> allocated)
> > > > those peers will send traffic to us, which we can then forward to
our
> > > > customer.  But they'll never hear the other ISP's announcement,
unless
> our
> > > > network was completely cleared off the map and the aggregate were
> > > withdrawn.
> > > >
> > > >
> > > >
> > >
> >
> >
>
>