[arin-ppml] Multihomed Microallocations

[William Herrin]

1. Policy Proposal Name: Multihomed Microallocations
2. Proposal Originator:
  a. name: William Herrin
  b. email:  bill at herrin.us
  c. telephone: 703-534-2652
  d. organization: self
3. Proposal Version: 1.0
4. Date: 3 August 2009
5. Proposal type: new
6. Policy term: permanent
7. Policy statement:

4.4 IPv4 Allocations and Assignments to Small Multihomed Organizations

4.4.1 Section 4.4 specifies criteria for allocating /23 and /24 IPv4
address blocks to end users and ISPs where the requesting organization
is multihomed with multiple Internet vendors but does not meet the
minimum usage criteria for address allocation or assignment under
Sections 4.2 and 4.3.

4.4.2 Except as specified in section 4.4, the requesting organization
must also meet all criteria for receiving addresses specified in
section 4.2 if an ISP or section 4.3 if an end user.

4.4.3 Criteria for allocation or assignment

4.4.3.1 The requesting organization must hold exactly one AS number
and must already announce IPv4 addresses to the Internet via BGP using
its AS number.

4.4.3.2. The requesting organization must announce IPv4 addresses to
the Internet via at least two distinct Internet vendors.

4.4.3.3. The requesting organization must spend at least $8000/year on
the Internet services in 4.4.3.2.

4.4.3.4. Upon annual renewal of the allocation or assignment received
under section 4.4, if the requesting organization fails to demonstrate
that it continues to announce IPv4 addresses to the Internet via at
least two distinct Internet vendors the allocation or assignment is
revoked and returned to ARIN.

4.4.3.5. The requesting organization must agree to withdraw any other
BGP routes it announces from the BGP table within 6 months of
receiving an allocation or assignment under section 4.4. If the
organization continues to receive IP addresses from its ISPs, those IP
addresses will be single-homed within the ISP's larger aggregate
announcement.

4.4.3.6. If the requesting organization fails to announce the
allocation or assignment received under section 4.4 to the Internet
using its AS number for at least 4 months total within a service year,
the allocation or assignment is revoked and returned to ARIN.

4.4.3.7. If the requesting organization already holds IPv4 addresses
directly from ARIN, from any other RIR or legacy addresses, the
organization must agree to renumber out of those addresses and
surrender them to the appropriate RIR within 6 months of receiving an
allocation or assignment under section 4.4.

4.4.3.8. The requesting organization agrees to return the allocation
or assignment received under section 4.4 to ARIN within 6 months of
receiving another allocation or assignment from any RIR.

4.4.3.9. For allocations of /23 and larger, the requesting
organization shall meet the utilization rate criteria described in
section 4.2 for ISPs and section 4.3 for end users. As /24 is the
smallest address block known to be generally routable on the Internet,
no utilization criteria will be applied to requests for a /24.


8. Rationale

The reason behind a /20 minimum assignment for single-homed orgs is
fairly straightforward: an ARIN allocation adds a route to the BGP
table which wouldn't otherwise be needed. Routes are expensive and the
cost falls into overhead since it isn't recoverable directly from the
org announcing the route. And we're not really certain how many routes
we can handle before the network falls over. So, we restrict the
availability of non-aggregable IP addresses to just very large
organizations. For smaller orgs, renumbering sucks but at least it
only costs the renumbering org, not everyone else.

The reason behind nothing smaller than a /24 is also straightforward:
many if not most ISPs filter out BGP announcements smaller than /24.
There is tremendous inertia behind /24 as the minimum
backbone-routable quantity going back to the pre-CIDR days of class-C
addresses. So, an ARIN allocation smaller than /24 would generally be
wasted addresses, unusable on the Internet.

But why peg multihomed orgs at /22 instead of /24? Multivendor
multihomed orgs have to announce a route anyway, regardless of whether
the addresses are from an ISP or directly from ARIN. Their routes are
not aggregable, even if assigned from ISP space. That's the way the
technology works and no new tech in the pipeline is likely to change
it.

With load balanced server clusters and NAT you can pack a heck of a
lot of punch into a multihomed /24 if you want to. And as a community
it's to our benefit to want registrants to pack the maximum punch into
their address space: IPv4 addresses are becoming scarce. So why
restrict ARIN assignments to folks who can write papers which justify
a /22?

FAQ

Q. Why not just use ISP addresses if you're too small?

A1. ISPs have conflicting requirements placed on their use of IP
addresses. They want, for example, to prevent address spoofing at
their borders by not allowing traffic from their addresses to enter
their network from another ISP. These goals conflict with multivendor
multihoming and generally make a mess when a customer wants to
multihome.

A2. Renumbering is expensive and painful. We should require it only
when it serves a reasonable public policy goal such as reducing the
consumption of BGP routing slots.

A3. We've seen common counterproductive situations where multihomed
end users make many discontiguous /24 announcements until they
eventually seek ARIN address space.

Q. But aren't your routes aggregable with your ISP's routes if you use
your ISP's address space?

A. No. For routes to be aggregable, two things must be true:

1. The routes must be contiguous, sharing a common network/netmask.
2. The routes must share an identical network topology.

In the mutlivendor multihomed case addressed by this proposal, the
routes almost never share an identical network topology. As a result
the routes can not be aggregated even if cut from the ISP's address
space. Single-homed networks are excluded from this proposal precisely
because they always share a network topology with the ISP.

Q. Can't other organizations filter routes at the RIR minimums and
user the ISP's covering route to reach you?

A. Maybe. With a bunch of ifs. You might not be connected to the ISP
who has the covering route, and what if he doesn't have the more
specific route to you?

The answer is more decisive on a practical level: The author was
unable to identify any ISPs who presently both filters on RIR minimums
and chooses not to carry a default route to an upstream ISP that
doesn't filter. Hence there is no apparent route filtering value to
the RIR minimums.

Q. What's so messy about multihoming with a cutout from an ISP's
allocation?

A. Many things. Here are some of them:

* As an ISP I want to drop packets from the Internet that purport to
be from my addresses (spoofed packets). I can't do that with packets
from a multihomed network: in the normal course of failure recovery or
traffic engineering, the multihomed user may originate packets to
hosts on my network using the IP addresses I assigned him, but via his
other ISPs. These packets will arrive at my Internet interfaces and if
I drop them as spoofed packets, I've broken my customer's
connectivity.

* As an ISP I want to reject false route announcements entering my
system which purport to serve my addresses. And I want my pager to go
off and let me know someone is trying to hijack my address space. My
multihomed customer will, in some circumstances, want me to route all
packets to him via his other ISP. That means I have to accept his
route announcement from other ISPs. To do this, I have to write some
tricky filtering rules that accept his routes right smack in the
middle of the address space where I generally want to reject routes.

* As an ISP, I want to aggregate my contiguous address space into a
single route announcement. It's part of being a good citizen: don't
waste the TCAM slots in everybody else's router. But I have to
carefully exclude my multihomed customer's routes from that
aggregation. Packets follow the more specific route. If he announces
his more specific route to his other ISP but I roll his route up into
my less specific route when I announce it then all his packets will go
to his other ISP instead of to me and I won't get paid.

* Lots of folks disaggregate their route announcements for the purpose
of traffic engineering. Two or three hops away from their system,
those TE routes are irrelevant. In theory I should be able to filter a
lot of that out by discarding the routes smaller than the RIR minimum
allocation for that /8. That would save me money and make my routing
updates work faster. But if I try it, I end up filtering mutlihomed
customers so that I can only reach them via the ISP that assigned
their addresses. At best that damages the effectiveness of my routing.
At worst it cuts me off from sites my customers want to access when my
competitors who just accept /24 everywhere don't have a problem. Oops.

Q. Where do the announced addresses in 4.4.3.1 come from?

A. Most likely from one of the ISPs as described in NRPM section
4.2.3.6. You go through the process of getting them assigned and
announced to demonstrate that you're not a poser. Then you get
addresses from ARIN under this proposal and return the 4.2.3.6
addresses to your ISP.

Q. What does "distinct vendors" mean?

A. It means two different ISPs like Verizon and Sprint. Two T1s with a
single ISP can be accommodated without announcing a route into the
Internet backbone, so such a connection does not qualify for addresses
under this proposal.

Q. $8000? What's that all about?

A. The best available estimate of the systemic cost of carrying a
route in the Internet backbone table is around $8000/year. See
http://bill.herrin.us/network/bgpcost.html for the cost estimate.
If you're going to play in the backbone, you should really be putting
more money into the system than you're taking out. If you have two
$600 T1's then you're spending nearly twice that anyway. This  limits
the folks who want to multihome their $50 DSL and cable modems.

Q. How reliable is that estimate? Does it change? Shouldn't ARIN
routinely update that estimate rather than codifying the specific
number in the policy?

A. In theory ARIN staff should set the number. In practice,
professional cost analysts are expensive and hard data on things like
router count is almost impossible to get anyway. Even if a more
reliable cost analysis could be produced, we still wouldn't know what
multiple of that cost was "fair" for the pay-in. 1x? 2x? 5x? Let's
just pick a number that's our best guess at fair, and move forward
with it.

Besides, the $8k rule will probably turn out to be a non-operative
part of the policy. Companies providing $50 DSL service are
disinclined to set up BGP sessions with their customers anyway. I
include it in the name of caution so that we're proof against a deluge
of multihomed cable-modem users but I expect that with some experience
under our belts we'll find that we can safely submit a follow-on
policy proposal that removes the $8k requirement.

Q. I have to renumber when exactly?

A. If you have IP addresses under section 4.4, you get to announce
that one allocation or assignment to the Internet via BGP. In fact,
we'd really prefer that you only announce one single route, even if
you get a /23 or larger. You don't get to collect two assignments and
then ten discontiguous assignments and burn up the BGP table. Until
you reach the minimums in sections 4.2 or 4.3, you renumber each time
you grow large enough to justify the next bigger allocation or
assignment. Yes, that's unfortunate and painful. But burning up the
BGP table would be even more unfortunate.

Practically speaking, you'll renumber zero or one times. Either you'll
never renumber because the /24 was enough to do the job, or when you
run out of space in your original assignment, you'll be big enough to
find a way to meet the minimum size criteria in section 4.2 or 4.3 so
that you don't have to renumber again.

Q. But renumbering is expensive! What's the difference between having
to renumber under this proposal and having to renumber when I change
ISPs?

A. You'll have to renumber less often if at all. The big deal is that
you don't have to renumber merely because you changed vendors and you
don't run into a sticky mess of   filtering rules as ISPs try to keep
control of their address space.

Q. Doesn't this discriminate against some kinds of multihoming?

A. In addition to multihoming with your own AS number, its possible to
have two ISPs separately announce your addresses or announce with a
private AS number that they strip from their peer announcements. This
proposal is more than complex enough. For the sake of making
verification simple, let's just say that tiny registrants will
announce with their own AS number, period.

Q. Does this proposal affect IPv6 allocations and assignments?

A. It does not appear to impact ISP allocations whose criteria is
spelled out in NRPM section 6.5.1.1. It does impact end user
assignments under NRPM section 6.5.8.1. End users who qualify for
addresses under this policy will also be qualified for an IPv6 /48.

Q. Have there been previous policy proposals to extend allocations and
assignments from ARIN to /24?

A. Yes. See the discussions in March and April of 2007 for proposal
2007-6. http://lists.arin.net/pipermail/arin-ppml/

In proposal 2007-6, the /22 size for multihomers in section 4.3 was
simply changed to /24. It met the following criticisms:

* Could make it easier for spammers. This seems to reflect some
concern at the time over whether ARIN's policies made things easy for
spammers to hop IP addresses and was probably a red herring. Spammers
aren't interested in registering with anybody. They want address space
as anonymously as possible for as long as possible as cheaply as
possible exerting as little effort as possible. All of these things
make address space under this proposal unattractive to spammers.

* Could create a land rush. This seems like an unreasonable concern
for the instant proposal: anyone who can justify addresses under this
proposal can justify the same addresses from their ISP already. So why
hassle them with ISP addresses?

* Could create a new swamp. The renumbering requirements in this
proposal prevent that problem.

* Unfair to ISPs since it only applies to end users. This proposal
applies to both.

* Staff worried that it could increase request workload. If it does,
the fees could presumably be set accordingly.

Proposal 2007-6 also tried to make the following point: Don't penalize
the honest. An org that has ponied up the cash to be multihomed with
multiple vendors can often restructure their network to require a /22
long enough to get one. Refusing ARIN assignments smaller than /22
encourages behavior which is contrary to ARIN's general public policy
goal of conservation. We'll be better off as a community if the folks
who are completely honest about their need get the same or better
treatment than the ones who lie.


Implementation notes:

This proposal asks for verification of multihoming somewhat beyond
what the rest of the NRPM requires. It does this in order to prevent a
land rush of cheaters. Not that there's likely to be a land rush of
cheaters (or if there is that they're likely to ask for less than a
/22), but better safe than sorry.

Verifying that there's a BGP announcement is trivial: go to any of the
hundreds of looking glasses. For the four-month rule, staff may want
to let it be practiced in the breach for now. That is, don't go out
and look unless someone complains. Writing software that actively
checks for it can be part of the address recovery strategy after
depletion.

Same deal with the route withdrawals: if slot consumption bugs the
ISPs, let them write a script which trolls for cheaters and then
complain.

To verify multihoming, I would suggest asking the registrant to
provide a letter of service from two ISPs in which they indicate that
they're under contract to announce routes for AS XXX.

To verify $8k, you might ask for a month's bills and check that 12
months worth adds up to $8k.

There's a respectable chance that folks requesting a /23 or larger
will continue to grow. It would be nice if ARIN staff made reasonable
efforts to reserve adjacent space for a year or so they may be able to
get the next larger size without having to renumber. With the scarcity
of IPv4 addresses this won't always be possible, but it would be good
to do as a "best efforts" kind of thing. .


9. Timetable for implementation: immediate


-- 
William D. Herrin ................ herrin at dirtside.com  bill at herrin.us
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004