[consult] Additional background information

[michael.dillon at bt.com]

> avoid undue server and bandwidth strain. An example illustrating the 
> need for some type of limit is the query of "! n net-*". Without a 
> limit, the result will be all 1.5+ million network records. Current 
> functionality truncates it after the first 256 records.

Clearly a query length limit is required just to protect against
mistakes.

> In addition to truncating results, there is a query rate limiter in 
> place to prevent excessive querying by a single IP address during a 
> prescribed period of time.

This sounds like a good idea to stop data mining. I don't think it
should change.

> ARIN seeks input in order to determine the magnitude of the problem 
> caused by the current 256 results limit. For those impacted by the 
> limit, what alternative limit do you suggest and why?

First, I think that the staff should monitor and adjust this limit as
required. If the server would log all queries that hit the limit, then
staff could analyze them to see what was causing the issue. In some
cases it will be legitimate queries for organizations with lots of
records, or organizations with similar names. If the limit can be raised
to accomodate more of these queries then just do it without consulting
us.

In addition, I think that you should implement some type of regular
expression matching to search a result set. For instance, let me see all
records with "smith" in them is too big. So let me search only records
with "smith" in them to find the subset which also have "richard" in
them. If a result set is greater than the query limit, then allow me to
reduce the size of that result set in stages until I get it small
enough. This is a common search engine function outside the Internet,
for instance in a library database where people search for journal
articles. And it is not hard to program either.

This would also enable searches like searching for all Smiths in
Albuquerque which are not possible today.

By the way "richard smith" returns entirely different results than
"smith, richard".

I have occasionally run into this problem and it is annoying but the
only way in which upping the 256 limit would have helped would have been
to remove the limit so I can get the full result set and Ctrl-F (in
browser) or grep it.

--Michael Dillon

P.S. there are so many open source search engines out there that I think
you should scrap the whois search engine and replace it with one of the
more standard search engines available. Or leave the old one alone with
its limits intact and install an alternate one that has proper search
options so that the limited size of the result set does not prevent
people from finding the records needed.