From jayb at braeburn.org Tue Dec 17 12:56:56 2013 From: jayb at braeburn.org (Jay Borkenhagen) Date: Tue, 17 Dec 2013 12:56:56 -0500 Subject: [arin-tech-discuss] ARIN RPKI hiccup on Saturday Message-ID: <21168.36968.86059.899120@oz.mt.att.com> Hi ARIN, I saw ARIN's announcements of scheduled maintenance for this past Saturday 14-December-2013, including a statement that RPKI Repository Services would be available during that outage. I also saw the 'all clear' message following the maintenance. But I have not yet seen any explanation of the hiccup in ARIN's RPKI that coincided with the maintenance. Can someone from ARIN please comment? Thank you. Jay B. From brak at gameservers.com Tue Dec 17 15:31:33 2013 From: brak at gameservers.com (Brian Rak) Date: Tue, 17 Dec 2013 15:31:33 -0500 Subject: [arin-tech-discuss] IPv6 Alternative to Reassignment Report? In-Reply-To: References: Message-ID: <52B0B4A5.5050109@gameservers.com> We're now updating our SWIP scripts to deal with IPv6, and this report works perfectly! Thanks for the help. On 11/22/2013 4:25 PM, Andy Newton wrote: > Brian, > > This fix will be in our next planned release, which is scheduled for > December 14. > > Andy Newton, > Chief Engineer, ARIN > > On 11/22/13 8:57 AM, "Brian Rak" wrote: > >> Any progress on implementing this? >> >> On 6/12/2013 4:54 PM, Andy Newton wrote: >>> On 6/12/13 3:53 PM, "Brian Rak" wrote: >>> >>>> Is there any alternative to the Reassignment Report that works with >>>> IPv6? >>>> >>>> We use this report as part of our SWIP process, as it's the only real >>>> way to get a list of all the reassignments that ARIN knows about. >>>> Using >>>> the whois api with /children is not really something that is useful, as >>>> it's limited to 256 entries. >>> Brian, >>> >>> Unfortunately the reassignments report doesn't work with IPv6 resources. >>> We consider this an oversight, and I have entered an issue in our bug >>> tracking system to have it rectified. >>> >>> Sorry for the confusion and the inconvenience. >>> >>> Andy Newton, >>> Chief Engineer, ARIN >>> >> > From markk at arin.net Tue Dec 17 16:45:42 2013 From: markk at arin.net (Mark Kosters) Date: Tue, 17 Dec 2013 21:45:42 +0000 Subject: [arin-tech-discuss] ARIN RPKI hiccup on Saturday In-Reply-To: <21168.36968.86059.899120@oz.mt.att.com> References: <21168.36968.86059.899120@oz.mt.att.com> Message-ID: Hi Jay Yes - we caught wind of this earlier today. We validate before publishing to the community and this was a new twist. The manifest certificate on the repository was invalid from 8:00AM EST until 2:05PM EST on 12/14. The repository is generated 2x a day each with a 24hr expiry on the manifest certificate. Unfortunately, we missed the evening run on 12/13 as we were shutting non-public services down preparing for the database conversion. It was "refreshed" the afternoon of 12/14 when the database was up and certified to be healthy post-conversion. As for RPKI, this was an oversight on our part and we are reevaluating having such a limited-duration cert. Other RIR's who have the same characteristics are also looking at changing their manifest certs as well. Our apologies for any inconvenience that this may have caused. Regards, Mark On 12/17/13, 12:56 PM, "Jay Borkenhagen" wrote: >Hi ARIN, > >I saw ARIN's announcements of scheduled maintenance for this past >Saturday 14-December-2013, including a statement that RPKI Repository >Services would be available during that outage. I also saw the 'all >clear' message following the maintenance. > >But I have not yet seen any explanation of the hiccup in ARIN's RPKI >that coincided with the maintenance. Can someone from ARIN please >comment? > >Thank you. > > Jay B. > > >_______________________________________________ >arin-tech-discuss mailing list >arin-tech-discuss at arin.net >http://lists.arin.net/mailman/listinfo/arin-tech-discuss From jayb at braeburn.org Tue Dec 17 19:41:39 2013 From: jayb at braeburn.org (Jay Borkenhagen) Date: Tue, 17 Dec 2013 19:41:39 -0500 Subject: [arin-tech-discuss] ARIN RPKI hiccup on Saturday In-Reply-To: References: <21168.36968.86059.899120@oz.mt.att.com> Message-ID: <21168.61251.347081.721376@oz.mt.att.com> Thanks for providing the explanation, Mark. There are bound to be some surprises along the way as we all get comfortable with the RPKI. You're probably already thinking along these lines, but could ARIN put up some monitoring of its RPKI output, alerting your operations staff for things like the number of authenticated RPKI objects dropping sharply? And to make sure any such monitoring is not taken offline during general systems maintenance events? On 17-Dec-2013, Mark Kosters writes: > Hi Jay > > Yes - we caught wind of this earlier today. We validate before publishing > to the community and this was a new twist. The manifest certificate on the > repository was invalid from 8:00AM EST until 2:05PM EST on 12/14. The > repository is generated 2x a day each with a 24hr expiry on the manifest > certificate. Unfortunately, we missed the evening run on 12/13 as we were > shutting non-public services down preparing for the database conversion. > It was "refreshed" the afternoon of 12/14 when the database was up and > certified to be healthy post-conversion. > > As for RPKI, this was an oversight on our part and we are reevaluating > having such a limited-duration cert. Other RIR's who have the same > characteristics are also looking at changing their manifest certs as well. > > Our apologies for any inconvenience that this may have caused. > > Regards, > Mark > > On 12/17/13, 12:56 PM, "Jay Borkenhagen" wrote: > > >Hi ARIN, > > > >I saw ARIN's announcements of scheduled maintenance for this past > >Saturday 14-December-2013, including a statement that RPKI Repository > >Services would be available during that outage. I also saw the 'all > >clear' message following the maintenance. > > > >But I have not yet seen any explanation of the hiccup in ARIN's RPKI > >that coincided with the maintenance. Can someone from ARIN please > >comment? > > > >Thank you. > > > > Jay B. > > > > > >_______________________________________________ > >arin-tech-discuss mailing list > >arin-tech-discuss at arin.net > >http://lists.arin.net/mailman/listinfo/arin-tech-discuss From brak at gameservers.com Wed Dec 18 10:12:00 2013 From: brak at gameservers.com (Brian Rak) Date: Wed, 18 Dec 2013 10:12:00 -0500 Subject: [arin-tech-discuss] IPv6 Detailed Reassign not automated? Message-ID: <52B1BB40.6010105@gameservers.com> Are IPv6 Detailed Reassignments a manual process right now? I've got 6 pending where I was reassigning to an existing ORGID rather then a new customer. They all appear to be pending review by ARIN. I've never seen any of our IPv4 reassignment requests pending like this, so I'm wondering what's going on. Submitted Date: 12-17-2013 15:45:27 Request Type: IPv6 Detailed Reassign Status: Pending Review Last Updated Date: 12-17-2013 15:45:27 Action Needed By: ARIN Or, is there something that I'm doing wrong that's leading to tickets being created rather then this happening automatically? From jonw at arin.net Wed Dec 18 16:32:36 2013 From: jonw at arin.net (Jon Worley) Date: Wed, 18 Dec 2013 21:32:36 +0000 Subject: [arin-tech-discuss] IPv6 Detailed Reassign not automated? In-Reply-To: <52B1BB40.6010105@gameservers.com> Message-ID: Hello Brian, Our software currently tickets IPv6 reassignments/reallocations for review when they either: 1) are for more than a /48, or 2) are for an Org ID that already has an IPv6 block (whether it's directly registered or a reassignment/reallocation) At the time the software was developed, there was a policy requirement that when a single end site requires an additional /48, it must provide documentation or materials that justify the request. You can find the relevant policy in section 6.5.4.2 of NRPM version 2011.3: https://www.arin.net/policy/archive/nrpm_20110727.pdf Ticketing these requests was necessary to ensure compliance with that policy. When policy 2011-3 ("Better IPv6 Allocations For ISPs") was implemented in January 2012, the requirement to supply documentation for multiple /48s was removed. It appears we overlooked this change at the time the new policy was implemented and didn't adjust our software to stop ticketing these for review. We'll report the problem to our engineering staff. If you have any further questions, comments, or concerns please respond to this message or contact me directly. Regards, Jon Worley Senior Resource Analyst ARIN Registration Services https://www.arin.net/ hostmaster at arin.net 703.227.0660 On 12/18/13 10:12 AM, "Brian Rak" wrote: >Are IPv6 Detailed Reassignments a manual process right now? I've got 6 >pending where I was reassigning to an existing ORGID rather then a new >customer. They all appear to be pending review by ARIN. I've never seen >any of our IPv4 reassignment requests pending like this, so I'm >wondering what's going on. > >Submitted Date: 12-17-2013 15:45:27 >Request Type: IPv6 Detailed Reassign >Status: Pending Review >Last Updated Date: 12-17-2013 15:45:27 >Action Needed By: ARIN > >Or, is there something that I'm doing wrong that's leading to tickets >being created rather then this happening automatically? >_______________________________________________ >arin-tech-discuss mailing list >arin-tech-discuss at arin.net >http://lists.arin.net/mailman/listinfo/arin-tech-discuss From brak at gameservers.com Wed Dec 18 16:57:59 2013 From: brak at gameservers.com (Brian Rak) Date: Wed, 18 Dec 2013 16:57:59 -0500 Subject: [arin-tech-discuss] IPv6 Detailed Reassign not automated? In-Reply-To: References: Message-ID: <52B21A67.2080006@gameservers.com> Ah those conditions make sense (as some of our requests went through ok, and others were ticketed). Do you have any sort of ETA as to when this will be corrected? On 12/18/2013 4:32 PM, Jon Worley wrote: > Hello Brian, > > Our software currently tickets IPv6 reassignments/reallocations for review > when they either: > > 1) are for more than a /48, or > 2) are for an Org ID that already has an IPv6 block (whether it's directly > registered or a reassignment/reallocation) > > At the time the software was developed, there was a policy requirement > that when a single end site requires an additional /48, it must provide > documentation or materials that justify the request. You can find the > relevant policy in section 6.5.4.2 of NRPM version 2011.3: > > https://www.arin.net/policy/archive/nrpm_20110727.pdf > > Ticketing these requests was necessary to ensure compliance with that > policy. > > When policy 2011-3 ("Better IPv6 Allocations For ISPs") was implemented in > January 2012, the requirement to supply documentation for multiple /48s > was removed. It appears we overlooked this change at the time the new > policy was implemented and didn't adjust our software to stop ticketing > these for review. We'll report the problem to our engineering staff. > > If you have any further questions, comments, or concerns please respond to > this message or contact me directly. > > Regards, > > Jon Worley > Senior Resource Analyst > ARIN Registration Services > https://www.arin.net/ > hostmaster at arin.net > 703.227.0660 > > > On 12/18/13 10:12 AM, "Brian Rak" wrote: > >> Are IPv6 Detailed Reassignments a manual process right now? I've got 6 >> pending where I was reassigning to an existing ORGID rather then a new >> customer. They all appear to be pending review by ARIN. I've never seen >> any of our IPv4 reassignment requests pending like this, so I'm >> wondering what's going on. >> >> Submitted Date: 12-17-2013 15:45:27 >> Request Type: IPv6 Detailed Reassign >> Status: Pending Review >> Last Updated Date: 12-17-2013 15:45:27 >> Action Needed By: ARIN >> >> Or, is there something that I'm doing wrong that's leading to tickets >> being created rather then this happening automatically? >> _______________________________________________ >> arin-tech-discuss mailing list >> arin-tech-discuss at arin.net >> http://lists.arin.net/mailman/listinfo/arin-tech-discuss From ecsd at transbay.net Tue Dec 31 11:54:30 2013 From: ecsd at transbay.net (Eric Dynamic) Date: Tue, 31 Dec 2013 08:54:30 -0800 Subject: [arin-tech-discuss] (1) Some serious issues with the REST service and (2) Strange inconsistencies in the database mappings Message-ID: <52C2F6C6.1090803@transbay.net> An HTML attachment was scrubbed... URL: From andy at arin.net Tue Dec 31 14:42:08 2013 From: andy at arin.net (Andy Newton) Date: Tue, 31 Dec 2013 19:42:08 +0000 Subject: [arin-tech-discuss] (1) Some serious issues with the REST service and (2) Strange inconsistencies in the database mappings In-Reply-To: <52C2F6C6.1090803@transbay.net> Message-ID: Eric, Thanks for your message and sorry about the confusion. First let me address your primary need, which is to gather data based on an IP address. While /ip/XXXX/orgs is not a supported query, /ip/XXXX/pft is and it will return the information you seek on both the organization and the POCs. Using the example address you gave, http://whois.arin.net/rest/ip/209.144.225.62/pft.txt will return the network registration, the associated organization, and the abuse contact you seek all in one query. Hopefully that will reduce the complexity of your code and address the bulk of your frustration. With regard to the issue of HTML being returned when JSON, TXT, or XML are requested, this is a known bug in the REST library upon which we rely. However, under these conditions our REST service does produce a proper HTTP status code of 404. When using curl, the ?f flag will produce a 404 and suppress the output, which should ease the burden on your script. Finally, our current plans for Whois information over HTTP/REST include the adoption of the RDAP protocol under development in the IETF. It is currently envisioned that all 5 RIRs and many domain registrars will deploy this service. When that time comes, programmatically retrieving data should be simpler and more uniform and thus reduce the complexity of your codebase. I hope this helps. Happy New Year! Andy Newton, Chief Engineer, ARIN From: Eric Dynamic > Date: Tuesday, December 31, 2013 11:54 AM To: "arin-tech-discuss at arin.net" > Subject: [arin-tech-discuss] (1) Some serious issues with the REST service and (2) Strange inconsistencies in the database mappings The REST info page: 4.3.3. MIME Types and File Extensions The following table lists the data types and their associated MIME types and file extensions: Data type Current Version MIME Type Version 1 MIME Type File Extension XML application/xml application/arin.whoisrws-v1+xml xml JSON application/json application/arin.whoisrws-v1+json json plain text text/plain txt HTML text/html html Problems: (1) formatting issue. http://whois.arin.net/rest/org/ATL-83/pocs.txt returns http://whois.arin.net/rest/poc/NOC3056-ARINhttp://whois.arin.net/rest/poc/NOC3056-ARINhttp://whois.arin.net/rest/poc/JCL171-ARINhttp://whois.arin.net/rest/poc/NOC3056-ARIN As opposed to what one can "swipe" from the web page: Abuse NOC3056-ARIN (NOC3056-ARIN) NOC NOC3056-ARIN (NOC3056-ARIN) Tech NOC3056-ARIN (NOC3056-ARIN) Admin JCL171-ARIN (JCL171-ARIN) So, the TXT output in this case has no line breaks and omits the function descriptions. Other similar TXT output more closely resembles what the web page shows. (2) missing information. Via the web, the URL http://whois.arin.net/rest/ip/209.144.225.62/orgs returns: NetRange 209.144.225.0 - 209.144.225.255 CIDR 209.144.225.0/24 Name SAVV-S218155-6 Handle NET-209-144-225-0-1 Parent SAVVIS4 (NET-209-144-0-0-1) Net Type Reallocated Origin AS Organization AppServe Technologies, LLC (ATL-83) Registration Date 2007-09-20 Last Updated 2007-09-20 Comments RESTful Link http://whois.arin.net/rest/net/NET-209-144-225-0-1 See Also Related POC records. See Also Related organization's POC records. See Also Related delegations. However, from the Unix command line curl -H "Accept: text/plain" http://whois.arin.net/rest/ip/209.144.225.62.txt returns this data: # # ARIN WHOIS data and services are subject to the Terms of Use # available at: https://www.arin.net/whois_tou.html # NetRange: 209.144.225.0 - 209.144.225.255 CIDR: 209.144.225.0/24 OriginAS: NetName: SAVV-S218155-6 NetHandle: NET-209-144-225-0-1 Parent: NET-209-144-0-0-1 NetType: Reallocated RegDate: 2007-09-20 Updated: 2007-09-20 Ref: http://whois.arin.net/rest/net/NET-209-144-225-0-1 # # ARIN WHOIS data and services are subject to the Terms of Use # available at: https://www.arin.net/whois_tou.html # Notice that there is no Organization Name. Ouch! for my purposes. (3A) Incorrect formats returned; (4A)Dead-end maze to get Organization Name due to apparent database mapping inconsistencies: I am trying to write a program which, given an IP address, returns * the associated NetRange, e.g. "209.144.225.0 - 209.144.225.255" * the Abuse POC associated with that netrange. Not getting the Organization Name in the command line call is making this a maze to accomplish, a dead-end as we'll see: If I do this: # curl -H "Accept: text/plain" http://whois.arin.net/rest/ip/209.144.225.62.txt # # ARIN WHOIS data and services are subject to the Terms of Use # available at: https://www.arin.net/whois_tou.html # NetRange: 209.144.225.0 - 209.144.225.255 CIDR: 209.144.225.0/24 OriginAS: NetName: SAVV-S218155-6 NetHandle: NET-209-144-225-0-1 Parent: NET-209-144-0-0-1 NetType: Reallocated RegDate: 2007-09-20 Updated: 2007-09-20 Ref: http://whois.arin.net/rest/net/NET-209-144-225-0-1 And then this: # curl -H "Accept: text/plain" http://whois.arin.net/rest/net/NET-209-144-225-0-1/pocs.txt http://whois.arin.net/rest/poc/JJ772-ARIN And then this: curl -H "Accept: text/plain" http://whois.arin.net/rest/poc/JJ772-ARIN/orgs.txt This is the output I get: > xmlns:pft="http://www.arin.net/whoisrws/pft/v1" xmlns:whois="http://www.arin.net/whoisrws/core/v1"> Whois-RWS