[ppml] Access to Bulk WHOIS data - a possible proposal? (fwd)

[william at elan.net]

Reposting my answer with less errors made (when I type way too fast I make 
lots of errors, sorry).

----

I have been advocating direct access to bulk whois for a while and I also 
represent company (or rather non-profit public-service project by the 
company) that would really benefit from being able to get current whois 
data at least once per day. Plus to that I'm beginning a work on real-time 
more specific bogons list based on whois data (i.e if ip space is present 
in whois or not) - first part of this project will involve only old internic
blocks and I hope to have it ready by end of June, but then I'd like to 
move to all blocks where arin is making current registrations as well and 
this will require being able to get bulk whois data daily.

A already made one proposal for last meeting:
http://www.arin.net/policy/2003_9.html

And new version of this proposal that will most go to next meeting is at:
http://www.elan.net/~william/arin_proposal_whois_aup-v2.htm

If the latest version does not meet your needs for access to bulk whois, I 
would very much like to hear about that as well as about any other comments
people have regarding the proposal and its text. I'll incorporate all good
suggestions to come up with acceptable proposal text by next meeting.

On Wed, 21 May 2003, Ian Baker wrote:

> Hi,
>     After some e-mail conversation with Mike at the ARIN helpdesk, I am
> thinking about putting-forward a proposal concerning access to bulk WHOIS
> data.
> 
> However, being a newbie at this, I thought it better to open-up discussions
> /before/ submitting such a proposal. If this is the wrong way of doing
> things, or has already been rejected in the past (couldn’t see anything
> obvious in the archives) then please just let me know..
> 
> Basically, I have written an IP to country/continent translator. It is
> initially being used to provide a geographical visitor profile to my web
> site, and as a filter mechanism of the anti-spam e-mail server that is
> currently awaiting release.
> 
> RIPE an APNIC data is processed using the bulk databases, taking around 3
> minutes, whereas the ARIN portion means sitting on the WHOIS throttling
> limit for a continuous 2 to 7 days. The reason being, I’m a private
> individual.
> 
> Apparently the rules are such that bulk WHOIS data is only available to
> corporations, and not individuals.
> 
> I can understand the reasoning behind such a rule – an individual
> spammer/cracker would be pretty difficult to track-down in the event of an
> abuse of access – but I’m not convinced that this should be an /absolute/
> rule. My reasoning is thus:
> 
> 1. The rule does not make a distinction between “white-hat” and “black-hat”
> activity – a corporation that later goes on to, or is acquired by an
> organization that, employs undesirable practices (e.g. spamming) may be
> permitted access, while a private individual is not.
> 
> 2. Much of the data is publicly available, if one is willing to wait long
> enough
> 
> 3. The data provided by a general WHOIS search is less accurate, as – from a
> sample of the run thus far – large blocks are allocated to individual
> organizations with no real way of determining whether smaller blocks have
> been sub-allocated to different organizations and countries. Which greatly
> reduces the accuracy of the data sampled.
> 
> 4. The majority of spamming data would appear to originate from trawls of
> Usenet, the Web, and SMTP servers.
> 
> 5. Depending upon what checks are made, there may be nothing to stop an
> individual from falsely claiming to represent an organization (e.g. Chief
> Fan Sanitation Engineer for ImadeThisUp Inc., or - if checks are in place -
> simply using their employer’s name without their knowledge). This allows the
> possibility of "black-hat" individuals gaining access to the data, while
> still inhibiting access by those of a more honest nature.
> 
> My conclusion is therefore that the ban on private access does not
> particularly aid the development of the Internet as a whole, while it most
> definitely inhibits certain aspects.
> 
> My proposal is, therefore, to remove the ban on private individuals having
> access to bulk WHOIS data and decide the issue purely on merit.
> 
> Now – what have I missed? ;o)
> 
> Regards,
> 
> Ian Baker
> Webmaster, codecutters.org &
> EMEA Support Manager, OpenConnect Systems Ltd.
>