[arin-ppml] IPv4 is depleted today - unrealistic statements about IPv6 inevitability

[Robin Whittle]

Short version:   I think that the routing scaling problems will not
                 be so severe as to lead to widespread IPv6-only
                 adoption in the foreseeable future.

                 The only routers which will be affected
                 significantly by the growing number of DFZ routes
                 will be those of transit providers and large ISPs
                 where the routers have the highest number of
                 neighbours.  I believe the operators of those
                 routers will always find it better to upgrade or
                 replace those routers, or to limit the number of
                 neighbours they connect to, rather than drop some
                 DFZ routes and therefore provide a second-rate
                 service.

                 "Dual Stack Lite" is the most plausible model for
                 using IPv4 address space more intensively for
                 DSL/DOCSIS customers - but it would be costly to
                 implement and to run, and probably can't do port
                 forwarding. Therefore, it would be a second-rate
                 service a significant proportion of end-users -
                 primarily those who use P2P file sharing -

                 would find unworkable.  This would make it very
                 difficult to market, due to support calls and
                 competitors who offer a full IPv4 service touting
                 the limitations of "Dual Stack Lite".

                 The most likely response to the IPv4 routing
                 scaling problem and the related shortage of
                 IPv4 space (caused largely by the current BGP
                 only system being unable to slice the space
                 up finely without bloating the DFZ routing table)
                 is a Core-Edge Separation solution such as LISP,
                 APT, Ivip or TRRP.  (Six/One Router is also a
                 Core Edge Separation solution, but it only works
                 for IPv6.)  This could be used as the basis for a
                 new kind of IPv4 global mobility.


Hi Paul,

I largely agree with you.

Mailing lists thrive on disagreement, so here goes.  You wrote:

> when other networks can't get new IPv4 or can't deaggregate the IPv4 they
> bought off of e-bay fast enough because the other, other networks can't
> grow their routing tables fast enough, 

I don't think this will be a significant problem.  DFZ routers can
handle more then the current ~260k routes.  If it suddenly shot up
to 360k tomorrow, I guess virtually every DFZ router would work
fine.  The system may become more fragile, the CPUs more busy etc.
but the only hard limits are FIB capacity and RIB memory.

I understand some routers with TCAM FIBs hit their limit of 244k in
the last year or so:

  http://psg.com/lists/rrg/2008/msg01851.html

I understand that RIB memory requirements depends largely on the
number of DFZ routes multiplied by the number of BGP neighbours.

If there was a gradual increase in the number of DFZ routes, most
operators would be ensuring their routers didn't connect to
sufficient neighbours to come close to their RIB memory limitations,
so the increase in DFZ routes would lead to gradual acceleration of
replacement or upgrade costs, and/or to limiting the number of
neighbours any one particular router is used with.

As the DFZ routing table grows, every network which finds its router
stretched to its limit needs to decide whether to use the router in
a role with less neighbours, whether to replace it or whether to
keep it on station, and by some strategy have it ignore some growing
subset of the routes.

You seem to be suggesting that this last option will occur, and that
an increasing number of networks will fail to have their DFZ routers
carry the whole IPv4 DFZ routing table.  That would have the effect
of making certain parts of the Net unreachable from each network.
Unless all such networks coordinated their activities to ignore the
same subset of routes, then there would be a generalised loss of
connectivity affecting an increasingly large subset of routes -
presumably mainly /24s in some foreign country.

For this to occur, I think some conditions would have to be met:

1 - The networks concerned figured it was better to disrupt their
    customers' connectivity to an increasingly large subset of
    /24s than to wear the costs of:

    a - Replacing or upgrading routers.

    b - Using existing routers with less neighbours than they would
        like.

2 - That the networks (presumably ISPs, but also AS end-user
    networks such as of universities, large corporations etc.)
    would continue to do this despite the complaints they would
    get from their customers / users.


I could imagine the administrator of a small AS end-user network
might take a look at the /24s and define a subset of them which are
advertised in some far-distant country they don't expect to connect
much.  The administrator might then decide it was better to drop
those routes than buy a new router.  However, such small end-user
networks are not running routers with the large numbers of
neighbours which causes the most likely limitation to be reached -
RIB memory limits.

AFAIK, no modern router has an FIB problem with the current DFZ size
 - I understand they could all handle a million or more prefixes,
since many large networks have many more internal prefixes than the
260k in the DFZ.

So I think the only routers where the RIB limit will be reached will
 be of big ISPs, transit providers etc. - those with ten or more
(very rough guess) neighbours.

I believe there is no way any network operating such a router would
want to make it fail to connect to a subset of the Net.  Such lack
of connectivity would surely impact one of the many networks whose
traffic flows through this router.  In some cases it would cause the
traffic to flow in some other way, but maybe there is no other path,
and maybe the operator of this router has a contract to provide full
connectivity to other networks.

So I cannot imagine how your scenario would actually develop: a
significant number of networks being unreachable due to those or
other networks hitting RIB limits with their routers and deciding to
keep running them with the same number of neighbours, but by not
accepting some subset of the /24s.


> to start doing their growth some other way.  they won't stop growing.  GIH
> seems to think that the "other way" this growth will happen is going to be
> NAT and while i agree that there will be an uptick i cannot think that this
> will be all that happens.

So you are suggesting that many ISPs will purchase, obtain or
whatever IPv4 space suitable for their growth needs (I agree with
this) but won't be able to use it because a significant number of
networks refuse to accept their route, for reasons discussed above.


> many of you here won't need IPv6 because you've run out of space.  you will
> need IPv6 because other networks have run out of space, and because new
> deployments will have no choice.  

I don't think they will "need" IPv6, because IPv6 does not give them
what their customers need: IPv4 connectivity.

A pure IPv6 service is completely unsellable now and I think this
will remain so for the foreseeable future - except perhaps where the
end-users are only using 3G cellphones and are getting all the
communications they want with the IPv6 services provided largely by
their mobile carrier.

The only way IPv6 might be useful in ordinary DSL/DOCSIS/FTTH access
networks is for ISPs to implement something like "Dual Stack Lite".

David Conrad and I had a long discussion about this on the Routing
Research Group mailing list:

   http://psg.com/lists/rrg/2008/msg02045.html  RW
   http://psg.com/lists/rrg/2008/msg02077.html  DC
   http://psg.com/lists/rrg/2008/msg02079.html  RW
   http://psg.com/lists/rrg/2008/msg02080.html  RW  P2P usage rates
   http://psg.com/lists/rrg/2008/msg02081.html  DC
   http://psg.com/lists/rrg/2008/msg02083.html  RW

There would be many cost problems installing and running the
"Carrier Grade NAT" boxes required for this proposal.  IPv4
addresses are still needed, but some number of DSL/DOCSIS customers
would share a single IPv4 address.

A major limitation is that this service would not allow the
end-user's PCs to grab a port on the NAT boxes public address, as I
understand is often done today for P2P applications and perhaps some
games.

I haven't studied this depth, but I understand this is normally done
via uPnP IGD - and there's no way this can be done for multiple
end-users sharing the one public NAT address.  Even if the NAT box
somehow supported it, the first end-user to request  TCP 80 would
get it, and the others wouldn't be able to - and would soon be
pestering the help desk with complaints.  (Though maybe it could be
made to provide it, as long as the applications requesting the ports
would accept some other port number than the one they first
requested, since some other customer might already have that port.)


> but you don't have to take my word for it;
> you can wait for hard evidence and then do a bunch of in-year hurry-up
> unbudgeted upgrades.  i think that approach is a little bit silly, since the
> actual costs of running dual-stack, at least near your edge or in your labs,
> is not high enough to be worth avoiding.  even if you have to tunnel at
> first because your upstreams and/or peers aren't as visionary as yourself.

I think the costs of running dual stack and making it work would be
pretty high.  There are plenty of apps which don't work with IPv6
and there are few servers on IPv6.

IPv6 is not a fix for the problem of not being able to reach every
part of the IPv4 Internet, because it doesn't reach any of the IPv4
Internet.  The IPv6 Internet is a different Internet from the IPv4
Internet.

Theoretically, "Dual Stack Lite" is a means of using IPv6 access
networks to concentrate one or more dozen domestic/SOHO end-users
onto a a single IPv4 address, but this is not at all the same as
genuine IPv6 adoption which makes the IPv6 Internet actually reach
more computers.  If those "Dual Stack Lite" users all ran dual-stack
then they could start to use their native IPv6 connectivity as well.
 This would be good for P2P and games for instance - assuming their
peers also had native IPv6 and were successfully running their home
network and PCs dual stack (and most of them wouldn't be).

But what sort of configuration work, software upgraded,
complexities, flakiness, support calls etc. would be required for
ordinary end-users to run their home networks dual stack successfully?

I don't believe "Dual Stack Lite" is marketable, since a significant
and important subset of customers require uPnP IGD port forwarding.
 There's no way of successfully mass-marketing a service like this
if even a few percent of people find it second rate.  Word would get
around and the cost of support calls would be too high.  All a
competitor has to do is advertise full IPv4 support, rather than the
second-rate "Dual Stack Lite" approach, and it would be much more
difficult to attract and retain customers, including those who
didn't need uPnP IGD.


> speaking of vision, here are the competing ones as i see them.  on one hand
> we've got a massively deaggregated IPv4-only core with the density of pure
> neutronium 

Yes - and I reach for the patchcord which is less than one micron long!

> and only the largest carriers able to handle the route churn and
> FIB storage, and where the market value of one more /24 is set by the cost
> of trying to find someone who will route it for you, and there are harsh
> pressures on old PI, and there's more or less no new PI, and there will be
> no new entrants into the IPv4 core routing market.  the rest of us will be
> paying high dollars for edge PA and using NAT universally.  

It could get this bad if there is no widely deployed Core-Edge
Scheme to solve the routing scaling problem (LISP, APT, Ivip, TRRP
or Six/One Router).  I predict it would get this bad and still it
would be cheaper to keep using IPv4 directly, or via "Dual Stack
Lite" than to try and sell customers a pure IPv6 service which is
for an Internet most people don't connect to.


> all new apps will
> either not depend on end-to-end or they will fail, so things like VoIP will
> have to be carrier-side and most new apps will ride on TCP/80.

I haven't thought through all the details, but it would be bad.


> on the other hand we've got a brief unpleasant period like described above
> before enough other networks add native IPv6 to make it worthwhile for late
> adopters to finally add native IPv6, at which point new apps have a choice
> of "IPv6 native, or IPv4 NAT" and new entrants to the core routing arena
> have that choice and while it's risky some folks will make it work somehow.

There's always a point in an "IPv6 will start to be widely adopted
real soon now, really" story where the logic takes leave of this Earth.

IPv4 and IPv6 is the biggest IT upgrade chicken and egg problem we
have ever seen.  There is a barrier between the current state and
the desired state and the barrier is much higher than for any
comparable transition problem, such as Windoze to Unix/Linux or
Windows to Mac.

The 100% connectivity the IPv4 Internet offers to all other Internet
users is not at all provided by IPv6 and won't be for decades, if
ever.  As long as that remains the case, then there's reason to use
IPv6 exclusively, because it doesn't work: it doesn't provide
connectivity to 100% of users, or anything like it.


I think you are saying that while networks would be able to get IPv4
space for quite a long time, as people split it more finely and use
it more intensively, that this will cause widespread adoption of
IPv6 (either without any IPv4 NAT arrangement, or with the
address-efficient "Dual Stack Lite" approach), because the RIB or
perhaps FIB limitations of DFZ routers cause many networks not to
carry the whole DFZ routing table.

However, I have argued above that this is only likely to kick in
with the RIB of the transit routers with the most neighbours - and
that the providers who operate those will always find it more viable
to upgrade the router, or restrict the number of neighbours, rather
than offer incomplete connectivity.


> pressure against PI will remain about the same as it is now.  NAT will
> gradually shrink down to places where its security features are worthwhile,
> or where there's just no good enough reason to tear it out and go native.
> 
> i don't merely like the second vision better.  i am the parent of teenagers
> i have studied history and am sure that if new generations of humans and
> their corporations are told that they will have to live out their lives
> according to the values and compromises of their forebears and that their
> opporunities must all be shackled by tithes to existing landowners, then
> they WILL find another way.

I agree - but I think the way is likely to be a Core Edge Separation
scheme to solve the IPv4 routing scaling problem.  Then, very large
numbers (hundreds of millions or even a billion or two) networks
(many or most with a single IPv4 address) will have what I call
"Scalable PI" (SPI) address space.

SPI space will be portable between ISPs (which have Egress Tunnel
Routers) and will support multihoming.

I think there is money to be made creating such a system - so it
won't necessarily wait for IETF standards to be established.  This
is particularly so since the Core Edge Separation scheme could be
profitably extended to provide a global mobility system, with a
stable IPv4 (or IPv6) address, portable all over the world, with
generally optimal path lengths, while being compatible with all
existing applications and with all existing hosts as correspondent
hosts.  Steve Russert and I just finished a paper on this:

  http://www.firstpr.com.au/ip/ivip/#mobile


Pure IPv6 connectivity won't be any value to most end-users until
some very high fraction of all other end-users - including those at
home, in offices, and all web servers, game servers etc. - have
native IPv6 connectivity too.  Whether that fraction is 99% or
99.99% I don't know, but it is so far from the current small
fraction of one percent that I believe we can safely conclude that
pure IPv6 will remain unmarketable for the mass-market of home/SOHO
users for the foreseeable future (say to 2020 at least), while there
will be a growing demand for an IPv4 Core Edge Separation scheme.

Such a scheme will produce a new kind of address space which is in
high demand - portable, multihomable etc. without the need for BGP
expertise or investment.  It will work fine for very small slabs of
IPv4 address space, including single IPv4 addresses.

That will be a real and saleable service, while pure IPv6 will not
be saleable, and while IPv6 with "Dual Stack Lite" will be difficult
to sell, since 20% or more of home/SOHO customers need the port
forwarding it (probably) can't provide.

 - Robin