small multihomed organizations

[Scott Richard Slater]

see below.

----- Original Message -----
From: "Redisch, Jason" <jredisch at virtela.net>
To: "'Scott Richard Slater'" <scottslater77 at hotmail.com>; <ppml at arin.net>
Cc: "Lee Howard" <lhoward at UU.NET>
Sent: Wednesday, August 08, 2001 5:08 PM
Subject: RE: small multihomed organizations


> All,
> Rather than writing filter lists for the ISP members of ARIN, I
> think we should focus on things within the ARIN charter that can solve the
> issues described below.  I listed a few potential policy changes that it
may
> be worth discussing in relation to these issues.
>
> 1) ARIN could look into allowing upstream providers to automatically
assign
> a /24 to a downstream if the downstream meets the same criteria required
to
> justify an AS number.  This assignment could be made regardless of the
> utilization of the block. It could be limited to one /24 per justified AS
> number.

First, an upstream provider can already assign a /24 from their larger
aggregate address space, well.. if the ISP(upstream provider) possesses
aggregate address space greater than a /24, as long as the
customer(downstream) provides justification to their ISP(upstream provider)
for 256 hosts.  Second, are the requirements to obtain an ASN the same as
the requirements to obtain a /24 address block or any address space for that
matter ?   I don't think so.  You can be a multihomed network with two
connections to ISPs(upstream providers) and require an ASN and not require
256 hosts (/24).  ASN justification
(http://www.arin.net/regserv/templates/asntemplate.txt).  Let's just say for
a minute that ARIN assigned a /24 to every ASN.  There are approximately
11628 ASNs assigned currently (I think) so 11628 * 256 hosts = 2976768
hosts.  That's 11628 /24s or about 6 /13s.

Next,  does a /24 assigned to a customer(downstream) still solve the
filtering problem the customer "may" face with some ISPs ?  Is there a
policy anywhere that defines aggregate address space filtering by ISPs ?

>
> 2) ARIN could look into revisiting the micro-allocation policy which would
> encourage such downstream customers to go directly to ARIN for a /24. They
> could then apply the same criteria listed in policy change number one.

First, let's just state that the micro-allocation policy limits the address
space allocation size from ARIN to a /24 and not greater.  So if the
customer needs 257 hosts they will need to request address space from their
ISP(upstream provider) as well.  Second, does a small multi-homed
organization meet that guidelines of the micro-allocation policy?  I guess
that would be discussed when revisiting the policy.  Third, this is a great
policy to revisit, since ARINs current minimum allocation is a /20.
ISPs(upstream providers) would end up retaining more IPs.  ISPs(upstream
providers) would be happy if the micro-allocation policy justified an ARIN
allocation of /24 blocks to their customers(downstream).  I think a good
policy change directed towards the ISPs(upstream providers) would be in
regards to their IP allocation policy.  Maybe a policy change such as
allowing ISPs(upstream providers) to only break up their aggregate address
space for customers(downstream) into blocks smaller than a /24.  So, maybe
the policy change would only allow IPs(upstream providers) to create
non-aggregate address blocks of /25,/26,/27,/28,/29,/30.  The only problem
with this idea is that ARINs IP address allocation policy seems a bit more
refined and stringent than the alocation policy of the ISPs(upstream
providers) to the customer(downstream).   In addition, is our ficticious
customer going to encounter difficulty with routing allocated ARIN address
space ?  Just an idea.

>
> 3) ARIN could take no action and wait for filters to change or other
> solutions to arrive.

sounds fine for now.

>
> Jason Redisch
>
>
>
> -----Original Message-----
> From: Scott Richard Slater [mailto:scottslater77 at hotmail.com]
> Sent: Wednesday, August 08, 2001 2:30 PM
> To: ppml at arin.net
> Cc: Lee Howard
> Subject: Re: small multihomed organizations
>
>
> objective: "small multhomed organizations with the complete ability to
> balance all inbound and outbound traffic over both connections".
>
> I think there are a few issues here.  The major issue is "filtering".  We
> need to adjust filtering to accomodate smaller multihomed organizations
and
> enable these smaller multihomed organizations to control/balance their
> inbound and outbound traffic over each connection.
>
> There are three methods of "filtering":
> 1) filtering using route information (access-list, distribute-list)
> 2) filtering using path information (ip as-path access-list, filter-list)
> 3) filtering using communities(ip community-list)
>
> Once we identify how to filter with each method then we next determine if
> any of the filtering methods accomodate our objective.  If none of the
> filtering methods accomodates our objective then we should create a new
> method for "filtering".
>
> note:  There is another option I just thought of.  If the customer is
> delegated address space from ARIN, and the inbound and outbound traffic is
> not balanced.  The customer can use the "set as-path prepend" command to
> balance the traffic over each connection.
>
> Also, please note, that the filtering methods above can be used together.
> You can filter using "community-list" and "ip as-path access-list" on the
> same router bgp configuration.
>
> Also, I was just thinking of what I wrote yesterday, about if one of the
two
> ISPs delegates address space to the customer then the delegating ISP will
> most probably be the backup connection for the customer.  This can be
> corrected using the BGP "backdoor" command.
>
> Also, note that the customers configuration will absolutely be different
> dependent on the number of BGP neighbors.  The more BGP neighbors chances
> are the customer will use some BGP metrics to choose packet flow.
>
> Just another thought,  its hard to develop an answer to adjust the present
> "filtering" configurations when we do not know how every provider filters.
>
> Another solution for the customer's 24 bit address space to be accepted by
> providers is the following, if the provider uses prefix-lists.  This way
the
> customers specific address space is accepted, a mask length of up to 24
bits
> in routes is accepted  EXAMPLE:  " ip prefix-list abc permit 0.0.0.0/8 le
> 24"  If the provider uses "le" they can specify the prefix length they
will
> permit or deny.
>
> I am working on some solutions and drawings.
>
> Scott
>
> ----- Original Message -----
> From: "Scott Richard Slater" <scottslater77 at hotmail.com>
> To: "Lee Howard" <lhoward at UU.NET>
> Sent: Tuesday, August 07, 2001 7:52 PM
> Subject: Re: small multihomed organizations
>
>
> > some thoughts, see below.    I sort of said the same things you were
> saying.
> > I agree with you.  I am checking with some customers I worked with that
> are
> > mutli-homed.  I can not remember exactly who allocated their address
> space.
> >
> >
> > ----- Original Message -----
> > From: "Lee Howard" <lhoward at UU.NET>
> > To: "Scott Richard Slater" <scottslater77 at hotmail.com>
> > Sent: Tuesday, August 07, 2001 3:15 PM
> > Subject: Re: small multihomed organizations
> >
> >
> > > The justification for an ASN is a routing policy that differs from
> > > adjacent Autonomous Systems.  Multihoming is accepted as sufficient
> > > difference.
> >
> > I agree.  I basically stated the same thing below. :)
> >
> > >
> > >
> > > On Mon, 6 Aug 2001, Scott Richard Slater wrote:
> > >
> > > > Date: Mon, 6 Aug 2001 18:54:07 -0400
> > > > From: Scott Richard Slater <scottslater77 at hotmail.com>
> > > > To: Lee Howard <lhoward at UU.NET>
> > > > Subject: Re: small multihomed organizations
> > > >
> > > >
> > > > Let me reiterate your example.  You are talking about a customer who
> is
> > > > multi-homed, that is connected to two ISPs.  First, the customer
will
> > > > require an Autonomous System(AS).   Then the customer will require a
> > unique
> > > > Autonomous System Number(ASN) from ARIN.  This ASN identifies the
> > customer's
> > > > network to the rest of the Internet through the Border Gateway
> > Protocol(BGP)
> > > > protocol.  BGP is implemented on the customer's router as well as
the
> > two
> > > > ISPs routers.  Since the customer's AS is talking to both ISPs ASs
> using
> > > > BGP, this implementation is called External BGP(eBGP).
> > > >
> > > > With the basic design stated above, we must identify the customer's
> > > > objective before we can determine more answers.
> > > >
> > > > More often than not, the customer is multi-homed to the two ISPs to
> > obtain
> > > > "redundancy" for their network.  In addition, most probably one ISP
> > > > connection will be used strictly for backup.  If this is NOT the
case
> > and
> > > > the customer wants to "balance" traffic over the two ISP connections
> > this
> > > > may be tricky, dependent on the ISPs BGP policies.
> > >
> > > I'm not sure that most multi-homing orgs use one only as a backup.
But
> > > I agree that it's tricky to split use between two paths.  Outbound
> > > balancing is easy (two equal static routes) but inbound balancing is
> > > impossible between two differently-homed connections.  I find that
most
> > > customers are satisfied with "some" traffic going over each link.
> >
> > Both ISPs can statically insert the customers routes, but the customer
> wants
> > to use BGP to take full routes from both ISPs to make more intelligent
> > routing decisions, and to be able to add routes without having to call
the
> > ISP.
> >
> > >
> > >
> > > > I do not think that a
> > > > /24 (Class C) is justifiable solely due to a customer being
> multi-homed.
> > > > The customer should provide thorough justification of their AS. '
> > >
> > > Well, sure, I never said we should change the justification for ASNs.
> > >
> > > >  Which
> > > > includes their network table, their number of hosts specific to
> hardware
> > > > (routers, firewalls, et cetera..), and their subnetting should be
> > detailed.
> > > > This is important for the ISP to look at.  The ISP may be able to
> > suggest
> > > > different network designs for the customer to conserve address space
> > based
> > > > on the network table the customer provides.  Normally, customer
routes
> > that
> > > > are a /24 or greater are not filtered.
> > >
> > > Right, exactly.  So if a customer has 24 hosts, a T1 to me and a T1 to
> > you,
> > > what do we do?  If I give him a /27, nobody except you will take your
> > path.
> > > If you give him a /27, nobody except me will take my path.
> >
> > Kinda,  let's look at it like this.  If the customer receives address
> space
> > from me(example 138.39.1/24), normally I would then be the customers
> backup
> > connection, and you would announce the more specific route to the
> customers
> > address space and you would attract more traffic.  This is due to the
fact
> > that I would normally announce my larger aggregate address space(example
> > 138.39/16).  Now, this is slightly fixable, if the delegating space
> provider
> > (example: me)announces the customers address space(example 138.39.1/24).
> > Most delegating providers will announce their customers address space to
> > their peers.  Some delegating providers want to keep their routing table
> > small and not announce a customer prefix out of their aggregate address
> > space.  This is an individual-case-basis. The other problem is that
there
> is
> > no gurantee that any of my peers will except the customers address
space,
> > due to the peers routing policies (this is an ever-so-changing policy
> > between peers - some peers filter at /19,/20,/24).
> >
> > So, the customer will not be satisfied if they receive address space
from
> > either me or you. Now, even if me and you give the customer address
space
> > (example a /24 from me to the customer & a /24 from you to the customer)
> > there is still the problem of our peers filtering out our customers
> specific
> > /24 address space.
> >
> > Now, even if the customer is delegated a /24 from ARIN the customer will
> > still encounter the peers filtering.  The only advantage of the ARIN
> > delegation of address space to the customer is the customer will never
> need
> > to renumber their domains only their physical interfaces with the
provider
> > giving them the connections.
> >
> > So,  there is only a few options.  It is understandable that smaller
> > organizations will not and do not receive the same benefits as
> medium-large
> > organizations.    Let me first state that "multi-homing with true
> > load-blancing" (the objective we are trying to achieve) is a WANT from
any
> > customer with an internet connection.  Who wouldn't want redundancy and
> more
> > reliability and better performance.  BUT, every customer can not HAVE
> > "multi-homing with true load-blancing".  It is the same as a customer
who
> is
> > using a Cisco 2500 series router.  I am sure the customer would much
> prefer
> > using a Cisco 12000 series router but every customer can not justify.
> >
> > Maybe AS-based configurations will go away and community-based
> > configurations with local-pref will be used throughout.   Though, I know
> > every provider does not currently use community tags.  And there is
still
> > the chance of corruption using community tags as with not using filters.
> >
> > >
> > >
> > > > The key to finding a solution for this customer is to understand the
> > > > customers objective first, next understand how many hosts does the
> > customer
> > > > actually need, third to understand the two ISPs BGP policies.  The
> > reason I
> > > > say this is that ISPs have very different BGP policies.  Some ISPs
> > adjust
> > > > their local-pref values.   The customer must also get a fine look at
> the
> > > > ISPs BGP 'import' AND 'export' policies.  Obviously, the customer
will
> > have
> > > > their own routing decisions based on the ISPs performance and cost.
> > >
> > > I understand well the differences between different ISPs' routing
> > policies,
> > > but I don't understand how that will change what the ARIN policy
should
> > be.
> > >
> > > Maybe we should consider what customers want. . .
> > >
> > > CUSTOMER WANTS PRIMARY AND BACKUP
> > > Even if most people use one ISP as primary and one as backup, any
> failure
> > > catastrophic enough to cause primary ISP to fail will prevent traffic
> from
> > > taking backup.  What I mean is, Customer connected to Primary Networks
> and
> > > Backup ISP Inc., gets a /27 from Primary's /16.  Great, all traffic
> takes
> > > Primary (except maybe traffic originating in Backup's network).  But
if
> > > Primary goes down, or some failure causes the /27 route to be
withdrawn,
> > > Unrelated ISP's traffic will still try to go through Primary, and get
> > > blackholed.  If Customer is lucky enough that Backup peers with
Primary,
> > > and if Primary allows more-specifics of its own aggregates to be
> announced
> > > to it, then Primary may shuffle traffic to Backup in case of an
outage,
> > but
> > > if case of Primary going completely down (or otherwise withdrawing the
> > /16)
> > > nobody will be able to reach Customer.  No redundancy is achieved.
> > >
> > > Looked at the other way, let's say Customer gets IPs from Backup.
Nope,
> > > doesn't work--no Internet traffic will take Primary because nobody
will
> > hear
> > > the /27.
> >
> > no static routing :)
> >
> > >
> > > CUSTOMER WANTS LOAD BALANCING
> > > Ain't no such animal.
> > > OK, fine, you want some traffic to use each link, so you buy a
> connection
> > > from Big Mama Networking MegaTelco, and Joe Bob's Bait, Tackle & ISP.
> You
> > > get a /27 from Joe Bob, and you're barely reachable.  Maybe Joe Bob
has
> > > transit from somebody, maybe he has a few peers, or maybe not.  If Joe
> > > Bob's aggregate block (/20 or greater) is globally routed, then fine,
> you
> > > have access.  Big Mama MegaTelco will send locally originated traffic
to
> > > you, too; it could be a significant amount of traffic.   But if Joe
Bob
> > > drops his pager while fishing, and there's an outage, Big Mama is the
> only
> > > ISP that will be able to reach you; everybody else will filter your
/27.
> > > It works the other way, too.  Sure, you have some traffic using each
> link,
> > > but you have almost zero redundancy if Joe Bob gets eaten by a bear.
> Our
> > > goal was load sharing, and we've got it, but it's less than most
> customers
> > > are looking for, in my experience.
> >
> > Customers must be very specific which provider connections they want and
> > which provider connections they need.
> >
> > >
> > > I think customers should reasonably expect transit to include global
(or
> > > at least nearly-global) routability.  The reason that doesn't exist
for
> > > longer prefixes now is that too many people make mistakes and leak all
> > > of their internal routes.  So other people filter, to avoid hearing
> > > those routes--they assume that all routes smaller than a /24 are
> leakage.
> > > I'm saying that not all of them are, and we should make it possible
for
> > > smaller customers to multihome.  How do we do that?
> >
> > just can't right now.  Due to everyone's filtering policies.  Unless
there
> > was some sort of "ARIN approval system" for non-aggregate address space.
> > First we would need to determine what is aggregate address space. (Is it
> /19
> > or /20 or /24) and then every provider would have to adjust their
filters
> > accordingly.  Second, we would then need this "ARIN approval system"
that
> > grants a non-aggregate address block the same routing privledges that an
> > aggregate address block has.  The only problem with this would be the
> > adjustment on everyone's filters.  If filters did not exist then
everyone
> > would rely on faith of the customer.  AND we know customers make
mistakes.
> > The only solution I can think of is written above.
> >
> > >
> > > Lee
> > >
> > >
> >
> > Scott
> >
> > > >
> > > > Scott
> > > >
> > > > ----- Original Message -----
> > > > From: "Lee Howard" <lhoward at UU.NET>
> > > > To: <ppml at arin.net>
> > > > Sent: Monday, August 06, 2001 3:42 PM
> > > > Subject: small multihomed organizations
> > > >
> > > >
> > > > >
> > > > > We occasionally have customers who will be multihomed with us and
> > another
> > > > > provider, and who need address space, but can't justify a /24
based
> on
> > > > > usage and host count.  If we give them less than a /24, no traffic
> > will
> > > > > come down their other line. [1]
> > > >
> > > > If no traffic is coming down their "other line" the customer should
> > check
> > > > the BGP metrics set by both ISPs.  There are a few possibilities.
The
> > ISP
> > > > with the line that is receiving all the traffic may have adjusted
> their
> > > > local-pref values to the customers advertisements on the "other
line".
> > I
> > > > think I said that right. :)
> > > >
> > > > I would also be curious of both ISPs peering connections and their
> > > > relationships with other "tier one" ISPs, especially the two ISPs
> > involved.
> > > >
> > > > >
> > > > > ARIN's current policy, as I understand it, is that multihoming is
> not
> > > > > justification for a /24.  I would like to hear recommendations and
> > best
> > > > > practices from others for how to address the customer (pun
intended)
> > > > > without violating ARIN policy.  Should ARIN accept multihoming as
> > > > > justification for a /24?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Lee Howard
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > [1] Here's why:  Most ISPs do not accept prefixes longer than 24
> bits,
> > so
> > > > > they will ignore our propagation of our customer's /24, and they
> will
> > > > > ignore the route announced from the customer's other ISP.  Since
we
> > also
> > > > > announce a large aggregate (say, a /13 from which that /24 was
> > allocated)
> > > > > those peers will send traffic to us, which we can then forward to
> our
> > > > > customer.  But they'll never hear the other ISP's announcement,
> unless
> > our
> > > > > network was completely cleared off the map and the aggregate were
> > > > withdrawn.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
> >
>