From info at arin.net Mon Mar 12 13:06:35 2007 From: info at arin.net (Member Services) Date: Mon, 12 Mar 2007 13:06:35 -0400 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit Message-ID: <45F5889B.6030008@arin.net> ARIN received a request to remove the ARIN WHOIS query limit of 256 results. This limit has been in place since ARIN's inception as a means of curtailing data mining. ARIN is now considering a suggestion to raise the result limit, and would like to solicit input from the community. Rather than raise the limit to an arbitrary number, ARIN requests that you provide your specific feedback as to what the query results limit should be. ARIN asks for feedback based on your experiences with using ARIN's WHOIS directory service. It would be of particular interest to ARIN to receive input from larger organizations with many registered resources or sub-delegations regarding an appropriate result limit. The ARIN Consultation and Suggestion Process documentation is available at: http://www.arin.net/about_us/corp_docs/acsp.html Discussion on this list will close at Noon ET 19 March. A poll on the topic will be conducted beginning Tuesday, 20 March. Only subscribers on the consult at arin.net list when the poll opens will be eligible to participate. Poll results will be publicly available and will be used by the ARIN President to help determine what course of action, if any, ARIN should take regarding the suggestion. We welcome community-wide participation. Please address any process questions to info at arin.net. Regards, Member Services American Registry for Internet Numbers (ARIN) From andrew.dul at quark.net Mon Mar 12 20:38:16 2007 From: andrew.dul at quark.net (Andrew Dul) Date: Mon, 12 Mar 2007 17:38:16 -0700 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit In-Reply-To: <45F5889B.6030008@arin.net> Message-ID: <20070313003700.5E1961444D8@smtp2.arin.net> At 01:06 PM 3/12/2007 -0400, Member Services wrote: >ARIN received a request to remove the ARIN WHOIS query limit of 256 >results. This limit has been in place since ARIN's inception as a means >of curtailing data mining. ARIN is now considering a suggestion to raise >the result limit, and would like to solicit input from the community. > Does anyone know what the current operational practice is of the other RIRs? That might be valuable information when deciding how to proceed. Andrew From Ed.Lewis at neustar.biz Mon Mar 12 21:41:09 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Mon, 12 Mar 2007 21:41:09 -0400 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit In-Reply-To: <20070313003700.5E1961444D8@smtp2.arin.net> References: <20070313003700.5E1961444D8@smtp2.arin.net> Message-ID: What kinds of numbers are we looking at? I mean, like, of all the "meaningful" queries, what would return the most? How many would be over 1000? When I say "meaningful" I am subjectively insinuating we don't care to honor data mining queries. (O *) There are only two reasons to limit the number. Budget for servers and bandwith is one, the other is data mining. It's hard to know what makes sense without more detail on why 256 is a problem and whether it should be, say, doubled or more. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From michael.dillon at bt.com Tue Mar 13 05:45:22 2007 From: michael.dillon at bt.com (michael.dillon at bt.com) Date: Tue, 13 Mar 2007 09:45:22 -0000 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit Message-ID: <78B96171FDCA6E4D8EFA8F39A9CD3E0040619E@i2km07-ukbr.domain1.systemhost.net> What on earth are you talking about? Please, this is not composition class. If you want to consult us on something, please state what the issue is in plain English with *FULL SENTENCES* not just one phrase copied from a point form list that was written on somebody's memo pad during a meeting. While you are at it, model the consultation query on policy proposals. Instead of a RATIONALE section, give us a BACKGROUND section. I did a search at ARIN's website for "whois query limit" and found nothing to indicate what you are talking about. Perhaps a single query returns no more than 256 results? Perhaps an IP address is limited to 256 queries per day? Perhaps it is something else. Please repost this query properly and restart the discussion time period. --Michael Dillon From michael.c.loevner at verizon.com Tue Mar 13 09:08:14 2007 From: michael.c.loevner at verizon.com (michael.c.loevner at verizon.com) Date: Tue, 13 Mar 2007 09:08:14 -0400 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit In-Reply-To: Message-ID: I've run into situations in the past where I can't lookup all of the resources of a company that we may be acquiring or merging with because of the 256 record limit. One improvement that could be made to the WHOIS that might help is to include a mechanism that allows one to query for direct allocations only for a particular OrgID or registration holder name. Also, ARIN staff has been helpful in the past in providing the full query when the number of returned records exceed 256. For myself, the infrequency with which I run into the query limit doesn't warrant a change, especially since ARIN staff is willing to provide the larger dataset if needed. If anybody runs into this limit frequently, I'd like to hear about it. -Mike "Edward Lewis" Sent by: consult-bounces at arin.net 03/12/2007 09:41 PM To consult at arin.net cc Subject Re: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit What kinds of numbers are we looking at? I mean, like, of all the "meaningful" queries, what would return the most? How many would be over 1000? When I say "meaningful" I am subjectively insinuating we don't care to honor data mining queries. (O *) There are only two reasons to limit the number. Budget for servers and bandwith is one, the other is data mining. It's hard to know what makes sense without more detail on why 256 is a problem and whether it should be, say, doubled or more. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. _______________________________________________ consult mailing list consult at arin.net http://lists.arin.net/mailman/listinfo/consult From Juan.Duque at movilnet.com.ve Tue Mar 13 09:22:26 2007 From: Juan.Duque at movilnet.com.ve (Duque Juan) Date: Tue, 13 Mar 2007 09:22:26 -0400 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit In-Reply-To: References: Message-ID: Hello, Pleasure of greeting them, I sent an e-mail several days ago, to do a consultation on IP's acquisition and ACE Public, but the response already her indicated the group of www.lacnic.net. Thank you very much Call me questions Greetings Juan Duque Consultor de proyectos Gerencia de Desarrollo e Integraci?n de Productos Telecomunicaciones Movilnet. +58-212-705.72.90 +58-416-606.27.10 juan.duque at movilnet.com.ve -----Mensaje original----- De: consult-bounces at arin.net [mailto:consult-bounces at arin.net] En nombre de michael.c.loevner at verizon.com Enviado el: Martes, 13 de Marzo de 2007 09:08 a.m. Para: Edward Lewis CC: consult-bounces at arin.net; consult at arin.net Asunto: Re: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit I've run into situations in the past where I can't lookup all of the resources of a company that we may be acquiring or merging with because of the 256 record limit. One improvement that could be made to the WHOIS that might help is to include a mechanism that allows one to query for direct allocations only for a particular OrgID or registration holder name. Also, ARIN staff has been helpful in the past in providing the full query when the number of returned records exceed 256. For myself, the infrequency with which I run into the query limit doesn't warrant a change, especially since ARIN staff is willing to provide the larger dataset if needed. If anybody runs into this limit frequently, I'd like to hear about it. -Mike "Edward Lewis" Sent by: consult-bounces at arin.net 03/12/2007 09:41 PM To consult at arin.net cc Subject Re: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit What kinds of numbers are we looking at? I mean, like, of all the "meaningful" queries, what would return the most? How many would be over 1000? When I say "meaningful" I am subjectively insinuating we don't care to honor data mining queries. (O *) There are only two reasons to limit the number. Budget for servers and bandwith is one, the other is data mining. It's hard to know what makes sense without more detail on why 256 is a problem and whether it should be, say, doubled or more. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. _______________________________________________ consult mailing list consult at arin.net http://lists.arin.net/mailman/listinfo/consult _______________________________________________ consult mailing list consult at arin.net http://lists.arin.net/mailman/listinfo/consult From info at arin.net Wed Mar 14 10:44:59 2007 From: info at arin.net (Member Services) Date: Wed, 14 Mar 2007 10:44:59 -0400 Subject: [consult] Additional background information Message-ID: <45F80A6B.4080207@arin.net> The introductory post to consult at arin.net on this topic, http://lists.arin.net/pipermail/consult/2007-March/000001.html, states that the ARIN WHOIS query limit of 256 results has been in place since ARIN's inception as a means of curtailing data mining. As noted on the WHOIS help page, http://ws.arin.net/whois, queries that return more than 256 results will stop displaying data after the 256th result. This arbitrary limit was put in place both to control data mining and to avoid undue server and bandwidth strain. An example illustrating the need for some type of limit is the query of "! n net-*". Without a limit, the result will be all 1.5+ million network records. Current functionality truncates it after the first 256 records. In addition to truncating results, there is a query rate limiter in place to prevent excessive querying by a single IP address during a prescribed period of time. Regarding other RIRs, RIPE WHOIS and AfriNIC WHOIS truncate responses at 300 records. LACNIC WHOIS response results are limited by the number of bytes, not records. The maximum size for a LACNIC WHOIS result response is 65536 bytes. We will post APNIC's information upon receipt. ARIN seeks input in order to determine the magnitude of the problem caused by the current 256 results limit. For those impacted by the limit, what alternative limit do you suggest and why? Regards, Member Services American Registry for Internet Numbers (ARIN) From michael.dillon at bt.com Wed Mar 14 12:07:50 2007 From: michael.dillon at bt.com (michael.dillon at bt.com) Date: Wed, 14 Mar 2007 16:07:50 -0000 Subject: [consult] Additional background information Message-ID: <78B96171FDCA6E4D8EFA8F39A9CD3E004740C5@i2km07-ukbr.domain1.systemhost.net> > avoid undue server and bandwidth strain. An example illustrating the > need for some type of limit is the query of "! n net-*". Without a > limit, the result will be all 1.5+ million network records. Current > functionality truncates it after the first 256 records. Clearly a query length limit is required just to protect against mistakes. > In addition to truncating results, there is a query rate limiter in > place to prevent excessive querying by a single IP address during a > prescribed period of time. This sounds like a good idea to stop data mining. I don't think it should change. > ARIN seeks input in order to determine the magnitude of the problem > caused by the current 256 results limit. For those impacted by the > limit, what alternative limit do you suggest and why? First, I think that the staff should monitor and adjust this limit as required. If the server would log all queries that hit the limit, then staff could analyze them to see what was causing the issue. In some cases it will be legitimate queries for organizations with lots of records, or organizations with similar names. If the limit can be raised to accomodate more of these queries then just do it without consulting us. In addition, I think that you should implement some type of regular expression matching to search a result set. For instance, let me see all records with "smith" in them is too big. So let me search only records with "smith" in them to find the subset which also have "richard" in them. If a result set is greater than the query limit, then allow me to reduce the size of that result set in stages until I get it small enough. This is a common search engine function outside the Internet, for instance in a library database where people search for journal articles. And it is not hard to program either. This would also enable searches like searching for all Smiths in Albuquerque which are not possible today. By the way "richard smith" returns entirely different results than "smith, richard". I have occasionally run into this problem and it is annoying but the only way in which upping the 256 limit would have helped would have been to remove the limit so I can get the full result set and Ctrl-F (in browser) or grep it. --Michael Dillon P.S. there are so many open source search engines out there that I think you should scrap the whois search engine and replace it with one of the more standard search engines available. Or leave the old one alone with its limits intact and install an alternate one that has proper search options so that the limited size of the result set does not prevent people from finding the records needed. From Ed.Lewis at neustar.biz Wed Mar 14 12:21:56 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Wed, 14 Mar 2007 12:21:56 -0400 Subject: [consult] Additional background information In-Reply-To: <78B96171FDCA6E4D8EFA8F39A9CD3E004740C5@i2km07-ukbr.domain1.systemhost.ne t> References: <78B96171FDCA6E4D8EFA8F39A9CD3E004740C5@i2km07-ukbr.domain1.systemhost.ne t> Message-ID: At 16:07 +0000 3/14/07, michael.dillon at bt.com wrote: >First, I think that the staff should monitor and adjust this limit as >required. If the server would log all queries that hit the limit, then >staff could analyze them to see what was causing the issue. In some >cases it will be legitimate queries for organizations with lots of >records, or organizations with similar names. If the limit can be raised >to accomodate more of these queries then just do it without consulting >us. I would like to follow up Michael's thought with a question - has staff be doing that monitoring and if so, what are the results? (That's a general question.) How many times are responses truncated, is there a pattern of a truncated response being followed by a modified query (evidence that the client is adjusting or "diving" deeper)? Further, is the staff uncomfortable with changing the limit without membership approval? I.e., why has this consultation been opened instead of following the recommendation Michael has made? >In addition, I think that you should implement some type of regular >expression matching to search a result set. For instance, let me see all I think that is out of scope for the consultation. Said not in disagreement, but it seems that the question on the table is the 256 object limit in the response. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From kkargel at polartel.com Thu Mar 15 11:27:47 2007 From: kkargel at polartel.com (Kevin Kargel) Date: Thu, 15 Mar 2007 10:27:47 -0500 Subject: [consult] Call for Consultation - Suggestion to Remove the WHOIS Query Result Limit Message-ID: <70DE64CEFD6E9A4EB7FAF3A063141066706CEB@mail> Regarding the ARIN WHOIS query limit of 256 results.. I am sysadmin for an ISP in the central US and would like to provide input to the Call for Consultation. I have only rarely been impacted by the 256 result WHOIS limit and I am quite happy with it. At this time, from my point of view, the 256 result limit is fine and the protections it offers outweigh any benefit my organization would see from removing it. Kevin Kargel Polar Communications ND 58270 $s/worry/happy,g -------------- next part -------------- An HTML attachment was scrubbed... URL: From heather.skanks at verizonbusiness.com Fri Mar 16 23:29:56 2007 From: heather.skanks at verizonbusiness.com (Heather Skanks) Date: Sat, 17 Mar 2007 03:29:56 +0000 (GMT) Subject: [consult] Additional background information In-Reply-To: <45F80A6B.4080207@arin.net> References: <45F80A6B.4080207@arin.net> Message-ID: As for the limit, it seems that certain searches have the limit and others do not. For example, ">NET-65-192-0-0-1" to find the subdelegations we have made from 65.192.0.0/11, returns 27,114 results .. all of them, every single one. Searching for "n UUnet" returns only 256 truncated results. Since I'm the person who submitted the request, I'm happy to provide some information on how we've been impacted. There are some very large organizations, some of whom happen to be our customers that have >256 resources. To be more specific >256 netblocks, with no way to further narrow down the search. In evaluating their requests for address space, we look at the space they already hold. When the number of resources exceeds 256 we have no way to obtain the additional data on our own. There is no way to page through the results, or even know how many results there are. Most of the time, the customer doesn't even know how many netblocks they have. With regard to how often this happens.. twice this week, and numerous times in the past. I think it's more of an issue for isp's, who often have to look at what space an organization has, and for large organizations who are trying to figure out what space is theirs. At some point, I think I was told I could not obtain the info through a hostmaster request, because it was not about my company. However I haven't had any problems requesting the information from ARIN recently. At the time I submitted the request to the consultation and suggestion process, to alter or remove the response limit, I was unaware that the full query must still be performed in order to be able to page through results. Having learned that, I became rather resigned to having to request the info from ARIN each time I need it. So, I asked that they place a more prominent banner at the top of results page, so that folks will be aware at the outset that they are not viewing the full list of results. That said, I would like to see the limit raised or a way found to remove it. It's a little thing, it probably doesn't affect a significant number of people, but it would make life a little easier, the day run a little smoother. I won't have to wait a few hours or a day on a response from ARIN in order to finish whatever task I am on at the moment. Maybe they could put in one of those are-you-a-bot-verification boxes, atleast for the website, so the results are obtainable in some way. As for a limit, I don't know what the right number is. Someone suggested looking at the number of records returned, when people submit requests to hostmaster. The one I most recently requested, had 333 netblocks. 500 or 1,000 seems reasonable. --Heather Heather Skanks 800.900.0241 security at mci.com On Wed, 14 Mar 2007, Member Services wrote: > The introductory post to consult at arin.net on this topic, > http://lists.arin.net/pipermail/consult/2007-March/000001.html, states > that the ARIN WHOIS query limit of 256 results has been in place since > ARIN's inception as a means of curtailing data mining. As noted on the > WHOIS help page, http://ws.arin.net/whois, queries that return more > than 256 results will stop displaying data after the 256th result. > > This arbitrary limit was put in place both to control data mining and to > avoid undue server and bandwidth strain. An example illustrating the > need for some type of limit is the query of "! n net-*". Without a > limit, the result will be all 1.5+ million network records. Current > functionality truncates it after the first 256 records. > > In addition to truncating results, there is a query rate limiter in > place to prevent excessive querying by a single IP address during a > prescribed period of time. > > Regarding other RIRs, RIPE WHOIS and AfriNIC WHOIS truncate responses > at 300 records. LACNIC WHOIS response results are limited by the number > of bytes, not records. The maximum size for a LACNIC WHOIS result > response is 65536 bytes. We will post APNIC's information upon receipt. > > ARIN seeks input in order to determine the magnitude of the problem > caused by the current 256 results limit. For those impacted by the > limit, what alternative limit do you suggest and why? > > Regards, > > Member Services > American Registry for Internet Numbers (ARIN) > > > > > _______________________________________________ > consult mailing list > consult at arin.net > http://lists.arin.net/mailman/listinfo/consult > From Ed.Lewis at neustar.biz Sun Mar 18 08:33:42 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Sun, 18 Mar 2007 13:33:42 +0100 Subject: [consult] Additional background information In-Reply-To: References: <45F80A6B.4080207@arin.net> Message-ID: Thanks for this input, this is what is really needed to start the discussion on the topic. (The official problem statement is too vague.) First, uh, some questions I have (these are for clarification, not to criticize): is how much trouble is it to use non-automated means to get this information? Can you (now) just submit a free-form request via mail to hostmaster and receive the data back in a later email? How much of a pain is it? How hard is it to justify the request? How long do you have to wait for the response, especially relative to your deadline for getting the answer (or relative to when you are prepared to deal with the answer). This last question is rather odd-ball, so let me retry - when you ask for the data are you already in a position where you need the answer to proceed with a request, and has there been times you had to delay an answer because you had to wait for the data. Second set directed at the staff/hostmasters: How often to you see requests like those Heather mentions - I mean, not how often from Heather/Verizon but across the board? Is this a general problem? (The answer maybe skewed because another ISP may not think to ask for the data once denied by the WhoIs server.) I think that it is quite acceptable that this is a need for ISPs and not for the general public. (I'll plug IRIS here - with IRIS a client can authenticate as an ISP POC and then be allowed to have the higher limit. Yadda, yadda, yadda. Perhaps standing up an experimental IRIS server and handing out credentials that allow over-size responses is something that can be put to trial.) Also - I sent an earlier mail asking for input from the staff. As this consultation is supposed to end soon, it would be real nice if there was a response from the staff (including "we can't give that feedback") in time to respond before the imposed discussion termination deadline. At 3:29 +0000 3/17/07, Heather Skanks wrote: ... >Since I'm the person who submitted the request, I'm happy to provide some >information on how we've been impacted. ... -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From michael.dillon at bt.com Sun Mar 18 10:15:57 2007 From: michael.dillon at bt.com (michael.dillon at bt.com) Date: Sun, 18 Mar 2007 14:15:57 -0000 Subject: [consult] Additional background information Message-ID: <78B96171FDCA6E4D8EFA8F39A9CD3E004D9957@i2km07-ukbr.domain1.systemhost.net> > There are some very large organizations, some of whom happen > to be our > customers that have >256 resources. To be more specific >256 > netblocks, > with no way to further narrow down the search. I think this is the key thing. Whois was not designed as a search tool and therefore did not make use of the prior work done in the search field. Nowadays there are numerous examples of search tools which DO provide ways to narrow down the search. Even Google allows this because you can add search terms, and you can also subtract search terms by using -keyword. OpenFTS, htdig, SWISH++, Nutch, Xapian and so on, are open source search tools which could be implemented to supplement the whois lookups when result-sets exceed some limit like 256. Imagine that the result set is calculated but not sent to the user. Instead, it is fed locally into a search engine and the user is given a ticket to narrow down their search for some time period. > However I > haven't had any problems requesting the information from ARIN > recently. > At the time I submitted the request to the consultation and > suggestion > process, to alter or remove the response limit, I was unaware > that the > full query must still be performed in order to be able to > page through > results. Having learned that, I became rather resigned to having to > request the info from ARIN each time I need it. In essence, the current whois search tool refers the user to the hostmaster who finds some human being to calculate the entire result set locally. Then they either send the entire result set to the end user, or use some local tools to narrow down the search. The question here is where the narrowing down should be done and whether or not human beings have to be in the loop for searches with large result sets. My view is that more modern and sophisticated tools will make this problem go away, even if we maintain the limit of 256 results as a general rule. Of course, there is another way that allows us to keep the creaky old whois engine and put the extra load on the endusers. That is to allow people to sign up for special service where they can do whois queries with no limits. That way an end user can login, do a long query, save it locally and use local tools to narrow down the search. > That said, I would like to see the limit raised or a way > found to remove > it. It's a little thing, it probably doesn't affect a > significant number > of people, but it would make life a little easier, the day > run a little > smoother. It is also generally a good thing for ARIN's public image if it is seen to be cleaning up dusty corners of its domain. The processes and tools inherited from the Internic were antiquated and poorly designed, even by the standards of the 1990s. ARIN has done a lot of work in fixing this, much of it invisible to outsiders. But I believe that ARIN should not rest on their laurels but should continue fixing and replacing these annoyances. --Michael Dillon P.S. even if there is no per-query limit to logged-in users, there should still be a daily limit, and some way of recovering any records that were truncated due to that limit. I.e. I login on Monday and run a query which would return 26,000 records but the daily limit is 10,000. On Tuesday I login and rerun the query requesting results >10,000. And on Wednesday I log in again and request results >20,000. This may be slightly annoying but it is justified to prevent data mining and likely to be infrequently encountered by real users. From info at arin.net Mon Mar 19 07:53:02 2007 From: info at arin.net (Member Services) Date: Mon, 19 Mar 2007 07:53:02 -0400 Subject: [consult] Consultation period extended - WHOIS Query Results Limit Message-ID: <45FE799E.5050500@arin.net> The discussion period is extended until Noon ET Monday, 26 March on the topic of removing the WHOIS query result limit of 256 records. ARIN staff will review recent inquiries on the list and respond later today with additional information. A poll on the topic will be conducted beginning Tuesday, 27 March. Subscribers on the consult at arin.net list at the end of the discussion period will be eligible to participate. Poll results will be publicly available and will be used by the ARIN President to help determine what course of action, if any, ARIN should take regarding the suggestion. We welcome community-wide participation. Please address any process questions to info at arin.net. Regards, Member Services American Registry for Internet Numbers (ARIN) From info at arin.net Mon Mar 19 15:57:18 2007 From: info at arin.net (Member Services) Date: Mon, 19 Mar 2007 15:57:18 -0400 Subject: [consult] ARIN staff input on WHOIS Query Result Limits Message-ID: <45FEEB1E.5040307@arin.net> The suggestion to remove the 256 WHOIS query results limit was referred to the consultation process in order to determine if the current limit presents a problem for many in the community. ARIN currently does not track the frequency that WHOIS responses are truncated. If ARIN were asked to track and report on this number, the results would include both valid uses of WHOIS and questionable uses of WHOIS. Absent a judgment-based analysis of the queries, the results provided would not correlate to the actual number of users impacted by the 256 response limit. ARIN rarely sees requests from people asking for expanded WHOIS queries for organizations and their associated resources such as the ones that Heather mentions. If we had to give an approximate number for those received within the last year, it would probably be less than 5 requests. The discussion period has been extended one week so that anyone with information to share regarding current experience with the limit may speak up. Regards, Member Services American Registry for Internet Numbers (ARIN) From Ed.Lewis at neustar.biz Tue Mar 20 05:07:31 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Tue, 20 Mar 2007 10:07:31 +0100 Subject: [consult] ARIN staff input on WHOIS Query Result Limits In-Reply-To: <45FEEB1E.5040307@arin.net> References: <45FEEB1E.5040307@arin.net> Message-ID: At 15:57 -0400 3/19/07, Member Services wrote: > >ARIN currently does not track the frequency that WHOIS responses are >truncated. If ARIN were asked to track and report on this number, the >results would include both valid uses of WHOIS and questionable uses of >WHOIS. Absent a judgment-based analysis of the queries, the results >provided would not correlate to the actual number of users impacted by >the 256 response limit. > >ARIN rarely sees requests from people asking for expanded WHOIS queries >for organizations and their associated resources such as the ones that >Heather mentions. If we had to give an approximate number for those >received within the last year, it would probably be less >than 5 requests. Given this latter paragraph, it sounds to me that the incremental benefit to altering the current situation would be low, unless the burden of the 5 requests on the client is quite high. (Heather?) This assumes that the estimate is only around 5, that there isn't a silent majority that hasn't "spoken up." (I doubt it in this case, but that's the question.) -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From kkargel at polartel.com Tue Mar 20 13:39:55 2007 From: kkargel at polartel.com (Kevin Kargel) Date: Tue, 20 Mar 2007 12:39:55 -0500 Subject: [consult] WHOIS Query Result Limits In-Reply-To: Message-ID: <70DE64CEFD6E9A4EB7FAF3A063141066706D1A@mail> Polar Communications is an ISP in rural North Dakota. We have not been seriously impacted by the 256 WHOIS query limit. We are fine with the limit staying as-is. Best regards, Kevin Kargel Polar Communications,110 4th Street East,Park River, ND 58270 Tel: (701)284-7221 x330 (800)284-7222 Fax: (701)284-2758 kkargel at polartel.com or sysadmin at polarcomm.com $s/worry/happy,g > > Message: 1 > Date: Mon, 19 Mar 2007 15:57:18 -0400 > From: Member Services > Subject: [consult] ARIN staff input on WHOIS Query Result Limits > To: consult at arin.net > Message-ID: <45FEEB1E.5040307 at arin.net> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > The suggestion to remove the 256 WHOIS query results limit > was referred to the consultation process in order to > determine if the current limit presents a problem for many in > the community. > > ARIN currently does not track the frequency that WHOIS > responses are truncated. If ARIN were asked to track and > report on this number, the results would include both valid > uses of WHOIS and questionable uses of WHOIS. Absent a > judgment-based analysis of the queries, the results provided > would not correlate to the actual number of users impacted by > the 256 response limit. > > ARIN rarely sees requests from people asking for expanded > WHOIS queries for organizations and their associated > resources such as the ones that Heather mentions. If we had > to give an approximate number for those received within the > last year, it would probably be less than 5 requests. > > The discussion period has been extended one week so that > anyone with information to share regarding current experience > with the limit may speak up. > > Regards, > > Member Services > American Registry for Internet Numbers (ARIN) From kkargel at polartel.com Fri Mar 23 12:43:48 2007 From: kkargel at polartel.com (Kevin Kargel) Date: Fri, 23 Mar 2007 11:43:48 -0500 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited Message-ID: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> In an earlier post I had stated that I was not impacted by the 256 result limit. Well, Murphy being all powerful has seen fit to humble me yet again. I wanted to find out what other organizations in my area had IPv6 blocks so I could do some social networking with them on local IPv6 issues, but there is no way to whois search for State/Prov . The suggestion from ARIN was to try searching with "+ n NET6-*" to get a set of results to grep locally for state. I thought I was good as that did not return the limit error, but apparantly it was limited, and just didn't announce the failure. At this point it appears I am just SOoL, and I wish to revise my comment. I would suggest that there be some method of obtaining larger search results, whether: a. being able to apply for a temporary exemption from the search limit for good cause b. being able to request a custom report, perhaps for a fee c. raise the limit (my least favorite option) Kevin $s/worry/happy,g From Ed.Lewis at neustar.biz Fri Mar 23 16:06:32 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Fri, 23 Mar 2007 21:06:32 +0100 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> References: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> Message-ID: Prompted by the fact that I might not have network access over the weekend or until the end of the consultation process I'd like to throw the following input on the table. 1) At the previous Public Policy meeting CRISP (really IRIS) was presented. (see: http://www.arin.net/meetings/minutes/ARIN_XVIII/ppm2_transcript.html#anchor_11) I can't find a count of the hands raised (somewhere there was a count of 6 interested and 9 wanted a prototype which caused some confusion - about motivation) at the end, but I thought there was a significant amount of interest in this. (Question to staff - is there a record of the counts at the end of the presentation? Is my question clear?) Assuming there is will to look further into IRIS, my suggestion is to use this as a trial balloon for IRIS. I.e., IRIS clients with approoriate credentials get un-capped results. 2) I don't want to penalize those that need access now by waiting for a decision on #1 and then, if it is a "go for it" implementation and deployment, there be some method for larger responses. I would prefer at least an off-line option, like emailing in the query to the hostmasters and have the result mailed back. I wouldn't mind just upping the limit but I have 2 reasons to be negative on that. First, there's the "if the limit is raised, then why bother with the IRIS experiment. That's a cheap reason I admit. Second, there's the "we set the limit for a reason and it would be unwise to just blow it off without realizing the tradeoff of risks." That's harder to quantify but a bit more real than the former reason. 3) Raise the limit a bit. Maybe from 256 to 260. No, kidding. Up to 1024 maybe. But that is arbitrary. 4) Promote the issue and ask for anyone who has been limited and felt they had a legit need to get the data (as opposed to "it was a mistake") to register the event and ask the staff to report the activity at the meeting in October. These suggestions are more like wild ideas than something I'd argue myself into a corner over. I have a question...if the output of this consultation is to generate a policy proposal, will this have to wait for the October meeting as it is close to the April meeting already? And, who is going to be the stuckee to generate it? -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From heather.skanks at verizonbusiness.com Fri Mar 23 16:30:20 2007 From: heather.skanks at verizonbusiness.com (Heather Skanks) Date: Fri, 23 Mar 2007 20:30:20 +0000 (GMT) Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: References: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> Message-ID: I'm not going to comment on everything.. but wanted to get this in while folks were still around for the day. regarding #3 - yeah, even raising it a little bit would be helpful. Double it, triple it.. find a way to log how many queries hit the limit, review the number of responses those queries would have returned and you have a place to start. Regarding this bit: > > I have a question...if the output of this consultation is to generate > a policy proposal, will this have to wait for the October meeting as > it is close to the April meeting already? And, who is going to be > the stuckee to generate it? The output of this consultation is *not* to generate a policy proposal. The consultation and suggestion process was developed specfically to handle things regarding ARIN services and practices ... not related to addressing and number resource policies. http://www.arin.net/about_us/corp_docs/acsp.html The end result of this process is not something that will be added to the Number Resource Policy Manual (NRPM) -- but rather will change how ARIN staff/services function operationally. That said, there doesn't seem to be anything that specifically prohibits an item from being discussed at an in person meeting. ARIN is seeking input from the community on this item and has chosen to do so by using the consult mailing list. At the conclusion of the consultation period, the Board or President may make a final decision, or in special circumstances, "determine that a vote by the members of ARIN is useful or appropriate" --Heather On Fri, 23 Mar 2007, Edward Lewis wrote: > Prompted by the fact that I might not have network access over the > weekend or until the end of the consultation process I'd like to > throw the following input on the table. > > 1) At the previous Public Policy meeting CRISP (really IRIS) was presented. > (see: > http://www.arin.net/meetings/minutes/ARIN_XVIII/ppm2_transcript.html#anchor_11) > > I can't find a count of the hands raised (somewhere there was a count > of 6 interested and 9 wanted a prototype which caused some confusion > - about motivation) at the end, but I thought there was a significant > amount of interest in this. (Question to staff - is there a record > of the counts at the end of the presentation? Is my question clear?) > > Assuming there is will to look further into IRIS, my suggestion is to > use this as a trial balloon for IRIS. I.e., IRIS clients with > approoriate credentials get un-capped results. > > 2) I don't want to penalize those that need access now by waiting for > a decision on #1 and then, if it is a "go for it" implementation and > deployment, there be some method for larger responses. I would > prefer at least an off-line option, like emailing in the query to the > hostmasters and have the result mailed back. > > I wouldn't mind just upping the limit but I have 2 reasons to be > negative on that. First, there's the "if the limit is raised, then > why bother with the IRIS experiment. That's a cheap reason I admit. > Second, there's the "we set the limit for a reason and it would be > unwise to just blow it off without realizing the tradeoff of risks." > That's harder to quantify but a bit more real than the former reason. > > 3) Raise the limit a bit. Maybe from 256 to 260. No, kidding. Up > to 1024 maybe. But that is arbitrary. > > 4) Promote the issue and ask for anyone who has been limited and felt > they had a legit need to get the data (as opposed to "it was a > mistake") to register the event and ask the staff to report the > activity at the meeting in October. > > These suggestions are more like wild ideas than something I'd argue > myself into a corner over. > > I have a question...if the output of this consultation is to generate > a policy proposal, will this have to wait for the October meeting as > it is close to the April meeting already? And, who is going to be > the stuckee to generate it? > > -- > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Edward Lewis +1-571-434-5468 > NeuStar > > Sarcasm doesn't scale. > _______________________________________________ > This message sent to you through the ARIN Consult Mailing List > (consult at arin.net). > Manage your mailing list subscription at: > http://lists.arin.net/mailman/listinfo/consult > From Ed.Lewis at neustar.biz Fri Mar 23 16:35:56 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Fri, 23 Mar 2007 21:35:56 +0100 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: References: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> Message-ID: At 20:30 +0000 3/23/07, Heather Skanks wrote: >The output of this consultation is *not* to generate a policy proposal. The That's right. I've just been so used to hearing that the only input is via the policy process for so long it's hard to get used to this. Plus, I don't know of any change so far that has come this way - maybe an unadvertised change has been made, I just mean nothing I have seen. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From michael.dillon at bt.com Fri Mar 23 16:39:47 2007 From: michael.dillon at bt.com (michael.dillon at bt.com) Date: Fri, 23 Mar 2007 20:39:47 -0000 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: <70DE64CEFD6E9A4EB7FAF3A063141066706D4B@mail> Message-ID: > At this point it appears I am just SOoL, and I wish to revise my > comment. I would suggest that there be some method of > obtaining larger > search results, whether: Earlier in your message you said that you wanted more flexible searching so that you can just specify the records you need. If ARIN would simply take some of the existing database search software or web search software, and adapt it to the whois database, this problem would be solved. > a. being able to apply for a temporary exemption from the > search limit > for good cause > b. being able to request a custom report, perhaps for a fee On the other hand, option b here may actually be LESS expensive for ARIN to implement. --Michael Dillon From michael.dillon at bt.com Fri Mar 23 16:43:21 2007 From: michael.dillon at bt.com (michael.dillon at bt.com) Date: Fri, 23 Mar 2007 20:43:21 -0000 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: Message-ID: > Assuming there is will to look further into IRIS, my suggestion is to > use this as a trial balloon for IRIS. I.e., IRIS clients with > approoriate credentials get un-capped results. IRIS is a protocol based on HTML. Not a search application. In order to use this, ARIN staff would have to learn XML, learn IRIS, build a better search application that accepts queries via IRIS, and then develop an IRIS client that runs on Windows, Mac OS/X and Unix. This is a multi-year 6 figure project and it still doesn't address the real issues which are illicit copying of the whois database and the functionality of the search engine that sits behind the scenes. --Michael Dillon From Ed.Lewis at neustar.biz Fri Mar 23 16:59:15 2007 From: Ed.Lewis at neustar.biz (Edward Lewis) Date: Fri, 23 Mar 2007 21:59:15 +0100 Subject: [consult] "Removal of WHOIS Query Result Limit" revisited In-Reply-To: References: Message-ID: I don't think that this is the appropriate venue to debate IRIS. I did throw out the idea as a possibility. At 20:43 +0000 3/23/07, wrote: >IRIS is a protocol based on HTML. Not a search application. I think you meant to say XML. IRIS has no relationship with HTML. The reason I raise IRIS is that one of the protocol features is the ability for the client to authenticate itself and then be authorized to access data that might otherwise not be freely available. >In order to use this, ARIN staff would have to learn XML, learn IRIS, >build a better search application that accepts queries via IRIS, and >then develop an IRIS client that runs on Windows, Mac OS/X and Unix. I believe that ARIN staff members have been active in the development of the IRIS specifications for quite a while. Look at http://tools.ietf.org/html/draft-ietf-crisp-iris-areg-15 and you will see in Appendix D and E the names of some of ARIN's staff. ARIN staff quietly participates in lot of forward-looking activities. You have to be present at the standards development and engineering meetings to notice. ARIN doesn't highlight this activity. As far as clients - I think you are also underestimating the power of open source. >This is a multi-year 6 figure project and it still doesn't address the >real issues which are illicit copying of the whois database and the >functionality of the search engine that sits behind the scenes. That may be, but sometimes benefit outweighs cost. The search engine...isn't that already built? The issue at hand is capping the output of the search. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis +1-571-434-5468 NeuStar Sarcasm doesn't scale. From info at arin.net Tue Mar 27 12:00:09 2007 From: info at arin.net (Member Services) Date: Tue, 27 Mar 2007 12:00:09 -0400 Subject: [consult] Announcing ARIN Consultation Poll Message-ID: <20070327160009.99DC71FFF5@mercury.arin.net> The suggestion to remove the 256 WHOIS query results limit was referred to the consultation process in order to determine if the current limit presents a problem for many in the community. After a two week discussion period on the topic, the Consultation Mailing List will close with this message. Each e-mail address currently subscribed to consult at arin.net will be e-mailed individually with instructions to participate in the poll. This poll will expire at noon ET on Monday, 2 April 2007. This is a simple polling question to gauge where the community stands. The discussion on this topic is archived and available at: http://lists.arin.net/pipermail/consult/. Issue: Given the modest amount of community interest expressed, ARIN will make no changes to the existing 256 WHOIS query results limit at this time. Poll results will be publicly available and will be used by the ARIN President to help determine what course of action, if any, ARIN should take regarding the suggestion. Regards, Member Services American Registry for Internet Numbers (ARIN) From info at arin.net Tue Mar 27 12:10:36 2007 From: info at arin.net (Member Services) Date: Tue, 27 Mar 2007 12:10:36 -0400 Subject: [consult] Announcing ARIN Consultation Poll Message-ID: <20070327161036.7356D1FFF5@mercury.arin.net> The suggestion to remove the 256 WHOIS query results limit was referred to the consultation process in order to determine if the current limit presents a problem for many in the community. After a two week discussion period on the topic, the Consultation Mailing List will close with this message. Each e-mail address currently subscribed to consult at arin.net will be e-mailed individually with instructions to participate in the poll. This poll will expire at noon ET on Monday, 2 April 2007. This is a simple polling question to gauge where the community stands. The discussion on this topic is archived and available at: http://lists.arin.net/pipermail/consult/. Issue: Given the modest amount of community interest expressed, ARIN will make no changes to the existing 256 WHOIS query results limit at this time. Poll results will be publicly available and will be used by the ARIN President to help determine what course of action, if any, ARIN should take regarding the suggestion. Regards, Member Services American Registry for Internet Numbers (ARIN)