> I hardly see what a single presentation I did with Avi several years ago has
> to do with the issue at hand. As it happens, I can count on one hand the
> number of conversations I've had with Avi this year.
Ummm, it's what you are using one of our IPs for to promote. It's on one
of your multiple web servers. If it's not important to you any more,
perhaps you should do some cleaning up and return some IPs.
> The fact that I may not have experience with specifically parsing WWW log
> files by no means implies that I have no experience doing that sort of thing
> in other applications. See, standard WWW transfer logs have tons of data in
> them that does not relate to calculating bandwidth utilization. That extra
> data all has to be looked at before the bandwidth numbers can even be
> retrieved. Let's look at a line of a standard Apache transfer log:
>
> 128.220.221.16 - - [05/Mar/1998:18:20:32 -0500] "GET / HTTP/1.0" 200 1195
>
> Now depending on how you count there are 6 fields on that one line of log
> file, and the number of bytes transfered number is the very last field. So
> that means that one way or another you need to look at each of the fields in
> the file and check if it's the right one before you can even get the
> appropriate data. I have to agree that parsing that logfile for bandwidth
> utilization is a major pain.
>
> But what if we changed the log file format to just look like this:
>
> 128.220.221.16 1195
>
> Or perhaps an even better way would be to write over the same line in the
> file again and again every time, so your utilization program just has to
> look at the file once to see how much has been used. Granted you can't just
> use Apache's mod_log_config for that, but it isn't a lot more work than
> that.
You know I agree with you on most of what you are getting at, but I need
to keep most of that log file anyway. Customers sometimes need to see
where there traffic is coming from and what pages they are hitting, so if
I need to log all that information it should go to the same file, so I
don't double the amount of writes I need for each request.
These days we spend almost the same amount of time explaining people's web
statistics as we do configuring their modems. That information is
important to them.
> My second statement about it not being a 'really efficient way to do it'
> meant to say that the 30 seconds I spent thinking of how to make the parsing
> process more efficient was probably not sufficient to come up with the
> optimal solution. Perhaps I should have said 'probably not optimally
> efficient' instead. Sorry about that.
Fair enough. I pulled out all the stops since you insinuated I was just
whining.
> I didn't mean to say it was no big deal. Making the changes I proposed
> would certainly take some work. However, contrary to what some other people
> said, the problem is not insoluble.
>
> My point was that I can't stand excuses for doing 'the right thing',
> especially when people insist on working against an organization that is
> only trying to help. ARIN is not making these policy changes to make
> everybodys' lives more difficult. ARIN is making the changes because it has
> a responsibility to stretch IP space in its region of the world as far as
> possible.
Hey, we try to do the right thing. I think this is where part of the
problem lies. There seems to be this impression that ISPs are guilty
before proven innocent, and not just during the long process of trying to
get new IP space. We are not greedy, whiny, little pricks.
Instead of putting the clamps on the ISPs why not focus on:
1) Reclaiming unused IP space to hold us out a little longer
2) Push a plan to get better client server technology out there, and once
it is out there get people using it. As an rotten example, but feeling
one is needed, what if the top 10 most popular sites had a message pop up
that informed people if they were using an old browser and encouraged them
to upgrade.
I'm not bitching just to bitch. I'm looking out for my ecommerce
customers. 90% of my revenue comes from businesses. If I don't watch out
for their bottom line, they sure the hell aren't going to look out for
mine. If I switch them to a name-based system, before the world is ready
for it and they lose hits do to software incompatibilites, or don't notice
that their traffic died, or they can't see how effective a commercial was
by using real-time accounting stats, or one of my customers gets DOSed and
I can't control the traffic at my core routers or at my upstream so I have
to take everyone down because they all share an IP, they are going to host
with someone who cheats the system and gets them an IP.
> Also, as I tried to say before people on the 'net have come up with some
> truly brilliant ways to deal with the issues that face us when they need
> to. I really think it would be a far better use of our time here if we all
> put our heads together to try and figure out a feasible way for everybody to
> use name-based virutal hosts in as many applications as possible than
> arguing about how hard it is. Then, if we as a group find that it is truly
> not possible then we can state that (from experience, as opposed to just
> from theoretical conjecture) at the next ARIN meeting and recommend an
> appropriate policy change.
Wonderful. I am with you 100% here. Like I said many times in my other
post, I'm sure you and most of the others here are a lot smarter than I
am. And I'm very willing to make changes that work, but no one seems to
be pointing me in the right direction, so all I can do is point out the
obvious nature of how things work as I understand them.
> You may think that just because I don't run a web hosting outfit today I
> don't understand the issues, and you're welcome to think that. It is true
> that I don't know how every single web hosting outfit out there accounts for
> usage, but I daresay you probably don't know that either. You know how you
> do things, and that's all you need to know. This is the exact reason why I
> or you alone are not responsible for creating ARIN policies. It is done by
> member participation in ARIN. And the general idea is that ARIN and its
> members benefit from having a hand in shaping what happens to IP allocation
> policy.
Obviously. And all I can do is let the group (ARIN) know that I for one
have a problem with it. And from judging by the number of responses sent
only to me last night, I'm not the only one. I'm not sure why most of
these people have not responded to the group. Maybe they don't want to be
labeled as a trouble maker and have even a tougher time getting IPs from
ARIN next time.
> Speaking to your request for a solution to your accounting woes, I really
> don't think you want that from ARIN. See, if that happens then people will
> start screaming about how ARIN dictates the way people must do business,
> which gets into another rat-hole that we really don't want to go down.
> There are many ways to skin this cat.
>
> So I will say again, instead of arguing with me about how easy or not easy
> this problem is to deal with, why don't we try actually solving the issues?
> And if they are not solvable then we will know we have tried our best and we
> can report those findings at the next ARIN meeting in an effort to get the
> recently adopted policy changed.
>
> And FYI, demanding a solution to your specific problem without providing any
> suggestions of your own is not the best way to engage help from others.
Maybe the some of the hosting world just isn't ready for this new policy.
It's not like I've sat on my ass for the last 6 years without changing how
I've done business. I wouldn't be around if I didn't evolve as you put
it. I've watched just about everyone that has not sold their business, go
out of business, and there are plenty of reasons for that, but primary it
was due to a lack of adopting new technologies. The point being I have
tested and adopted my own suggestions already.
My suggestion to the core problem, is to reclaim unused IP space, push for
IPv6, and make appropriate IP saving changes when it does not
significantly hinder the ability to do business on the Internet.
> Not a bad setup. I don't really see what I said before would not apply to
> this setup.
If eliminate multiple IPs I'm unsure how to:
1) Address the HTTP/1.0 issues in an acceptable clean fashion
2) Do real time web accounting. Remember we buy bandwidth by the Mbit, so
we need to sell it by the Mbit
3) Provide controls against DOS attacks. No we don't host porn sites
4) Provide secure server certificates
5) Provide database support from server to server. I'm not a programmer
any more so I don't know how big an issue it is, but my programmer told me
it would be a mess
Those are some of the issues that I don't know how to handle with single
IP hosting.
> > And like I said before, when appropriate we have assigned multiple sites
> > to a single IP. We actually do it by sending all requests into a CGI
> > script that grabs the HTTP_HOST env variable and creates the customized
> > web site on the fly with MySQL. So yes, we are trying to conserve IP
> > addresses, we are not greedy, whiny bastards trying to screw the Internet
> > up for everyone else.
>
> Nobody ever said you were, and I truly resent having words put into my
> mouth. Please refrain from doing so in the future.
Miscommunication. I didn't mean to imply that you called me a greedy,
whiny bastard. I was trying to emphasize the point that ISPs, at least
mine, are trying to conserve IP space. ARIN's policy implies that ISPs
are not doing enough to conserve the space. And like I said before there
seems to be this mis-conception that ISPs are fighting change and IP
conservation. Hell, our business depends on more people getting
access. We of all people should be, and I beleive most are, promoting IP
conservation.
> If you recall, I was addressing a specific post where a person was demanding
> specific solutions to every problem that this policy change would be
> causing. I, for one, don't respond well to demands for help.
Look. My point is I live this business. I realize I'm not the smartest
guy out here, but I've been doing this a long time in Internet
years. ARIN has come up with a policy that I think is premature. With
the needs and tools I have, and I have do have a few, it's a bad
policy. But if someone can show me how I'm mistaken, I will gladly listen
and change my technologies.
Actually I think the policy would make a wonderful "Guideline". It
shouldn't affect IP allocation, but it should be encouraged at this time.
> > Cool! Now we all know how to do name based hosting... er, wait... what
> > about all those HTTP/1.0 browsers!? You don't think they exist any
> > more? Check this out. In fairness I sampled all my virtual hosts off of
> > one server from a selective time period. All my logs files are in the
> > www.domain.com format. Here are my commands and results:
> >
> > webserver3: {17} % grep 'HTTP/1.1' www.*.com | wc -l
> > 400441
> > webserver3: {18} % grep 'HTTP/1.0' www.*.com | wc -l
> > 375412
> >
> > 48.4% of the browsers out there that accessed my customers' sites used
> > HTTP/1.0. For the uninitiated the 1.0 version of the HTTP protocol does
> > NOT support name based hosting.
>
> That's the first number I've seen on the subject that is greater that 2%,
> and I will confess it does concern me a great deal.
>
> Does anybody else have any numbers they'd like to share?
As someone pointed out. Apparently HTTP/1.0 can support name based
hosting. I was unaware of this.
And if that truely is the case, I would like to see some numbers. I would
have guessed ARIN would know this before instituting a policy. Perhaps
they would like to share.
> > Can I tell all my customers to call you when their online business drops
> > by almost 50%. By the way, can you use a shared IP for secure server
> > certificates?
>
> No, you can't, which is why there are exceptions to the policy. Granted
> there isn't a specific exception for SSL, which I think is one place where
> the group (myself included) erred in Calgary last March.
That's an easy one.
> Look, bottom line is that name-based virtual hosts have the ability to
> stretch our IP utilization even further (and the way IPv6 is looking means
> we'll really need to do this). Moreover, if you think the name-based
> virtual hosting policy should be changed or repealed, then by all means
> start participating in the process to make that happen.
That's what I'm trying to do! Or is this not the right place to
participate?
> And finally, there may well be some websites out there that cannot be
> handled any way except for giving them their own IP address. I don't know
> this for sure, but I'd say it's a pretty good guess.
>
> Similarly, there are some dial-up users out there who insist on having a
> static IP address. ISPs are free to do that, _JUST AS LONG AS MOST OF THEIR
> LOW-END CLIENTS USE DYNAMIC IP ADDRESSES_. This can easily be extended to
> virtual hosting. And I agree that this should be stated specifically, but I
> really think ARIN's true intent was to change the default mom-and-pop
> hosting account from a dedicated IP address to a name-based virtual host.
>
> So perhaps the policy should be re-worded to state that for providers who
> sell 'cheap' web-hosting for domains that get relatively few hits per month
> they should use name-based virtual hosting for those clients? From what I
> recall from the discussion in Calgary those were the accounts the policy was
> targeted at...
Ah! Now we are getting somewhere. Where to draw the line though? That
extra 5% business for a company whether they are doing $10,000 or a
million or more is still pretty important, especially now with everyone's
margins so low while the fight for market share appears to be paramount.
Almost all my account are $50/month. Is this considered cheap? Do you
have to be a IBM selling $2500 accounts to gain the exception? Or, are
the $19.95 joints where the cutoff would be drawn? Just curious.
Alec, I understand your and ARIN's points. However if a "policy" is going
to be created and enforced I think we some of these issues need to be
better addressed and defined so legit ISPs don't have to wait over a
month to get new IP space and go through a process of defending web
hosting IP space.
As I read the policy I'm out of compliance, and I have issues with that.
Mury
GoldenGate Internet Services