From markk at arin.net Wed Sep 15 13:43:45 2010 From: markk at arin.net (Mark Kosters) Date: Wed, 15 Sep 2010 17:43:45 +0000 Subject: [Arin-tech-discuss] Welcome Message-ID: Hi You are receiving this email since you were subscribed to one or more of the ARIN technical lists - arin-rpki-pilot, arin-whoisrws, or arin-regrws. Since the volume of each list is small, we decided to merge them into one list. ARIN's engineering team hangs out on this list to solicit opinions and answer questions on engineering topics as it relates to ARIN. We'll keep the individual archives of the retired lists up for those who want to look back to see what was said in the past. Of course, we'll have an archive for this new list as well. If you feel you have received this message in error, you do have a way to take care of the issue. The web page to manage your subscription is: http://lists.arin.net/mailman/listinfo/arin-tech-discuss There is also an email-based interface for users for this list; you can get info about using it by sending a message with just the word `help' as subject or in the body, to: arin-tech-discuss-request at arin.net Please address all questions to mailman-owner at arin.net. Regards, Mark Kosters ARIN CTO From anil.malyala at verizonbusiness.com Thu Sep 16 10:10:35 2010 From: anil.malyala at verizonbusiness.com (Malyala, Anil K) Date: Thu, 16 Sep 2010 14:10:35 +0000 Subject: [Arin-tech-discuss] issues with new RESTful bulk whois download Message-ID: Hello, I am running into errors pulling the file with the certificate. Options not to use certificate are not working as well. I removed the correct key in this post for security purposes. $ wget https://www.arin.net/public/rest/downloads/bulkwhois?apikey=API-Removed- B181 >test-arin-file --2010-09-15 18:23:26-- https://www.arin.net/public/rest/downloads/bulkwhois?apikey=API-Removed- B181 Resolving www.arin.net... 192.149.252.76, 192.149.252.75, 2001:500:4:13::81, ... Connecting to www.arin.net|192.149.252.76|:443... connected. ERROR: cannot verify www.arin.net's certificate, issued by `/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certificates.starfieldtech.com/repository/CN=Starfield Secure Certification Authority/serialNumber=10688435': Self-signed certificate encountered. To connect to www.arin.net insecurely, use `--no-check-certificate'. Unable to establish SSL connection. $ wget --no-check-certificate https//www.arin.net/publicrest/downloads/bulkwhois?apikey=API-59F9-195A- 4D60-B181 >test-arin-file --2010-09-15 18:23:17-- https://www.arin.net/public/rest/downloads/bulkwhois?apikey=API-Removed- B181 Resolving www.arin.net... 192.149.252.76, 192.149.252.75, 2001:500:4:13::81, ... Connecting to www.arin.net|192.149.252.76|:443... connected. WARNING: cannot verify www.arin.net's certificate, issued by `/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./OU=http://certificates.starfieldtech.com/repository/CN=Starfield Secure Certification Authority/serialNumber=10688435': Self-signed certificate encountered. HTTP request sent, awaiting response... 500 Internal Server Error 2010-09-15 18:23:18 ERROR 500: Internal Server Error. Thanks, Anil -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at arin.net Thu Sep 16 13:03:45 2010 From: andy at arin.net (Andy Newton) Date: Thu, 16 Sep 2010 17:03:45 +0000 Subject: [Arin-tech-discuss] issues with new RESTful bulk whois download In-Reply-To: References: Message-ID: <08B64425-CF6D-43AC-AD8D-987FEACD876A@arin.net> The 500 error may be received if the applied APIKEY does not have Bulk Whois access enabled. ARIN requires an Bulk Whois AUP to be signed before an APIKEY can be used to access Bulk Whois. We have in our issue tracker an issue to change the 500 to a more appropriate error for this type of error. Andy Newton Chief Engineer, ARIN From cayanaya at cheerful.com Fri Sep 17 20:56:39 2010 From: cayanaya at cheerful.com (cayanaya at cheerful.com) Date: Fri, 17 Sep 2010 20:56:39 -0400 Subject: [arin-tech-discuss] openend the website locked (password recovery) Message-ID: <8CD24E37CB91D47-D54-6DB7@web-mmc-m08.sysops.aol.com> morning, I have lost my website, locked. could you help me to opened the website. after I I have forgotten the password after i has change the password. this is my website : www.pancaindera.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes at ren-isac.net Tue Sep 21 15:37:21 2010 From: wes at ren-isac.net (Wes Young) Date: Tue, 21 Sep 2010 15:37:21 -0400 Subject: [arin-tech-discuss] bulkwhois libxml2 Message-ID: Figured this was a good place to start... i'm debugging an issue (or some "functionality") with libxml2 and perl (using the XML::LibXML::Reader interface), where it' seems to be stumbling on the series of: \n if I insert a linebreak; (\n\n) the libxml2 reader function rips through it no problem, if there's no line break, it views the as a blank element and then reads the rest of the file as garbage data (trying to do this stream like instead of DOM like). I'm assuming there isn't anything wrong with the way it's outputted (guessing most people are just java-nuts and it works like that), but i'm curious if anyone has gotten around this issue with libxml2 (or alike) by setting some sort of parsing flag, etc. I've tested it a few times with the first few set of asn elements you'll find in the data; and the line break pretty much makes it reproducible... jw if anyone has worked around that from the perl side... -- Wes http://claimid.com/wesyoung -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From andy at arin.net Wed Sep 22 09:51:39 2010 From: andy at arin.net (Andy Newton) Date: Wed, 22 Sep 2010 13:51:39 +0000 Subject: [arin-tech-discuss] bulkwhois libxml2 In-Reply-To: References: Message-ID: <1403BE18-7F30-4BDD-801C-473889489AB8@arin.net> On Sep 21, 2010, at 3:37 PM, Wes Young wrote: > Figured this was a good place to start... > > i'm debugging an issue (or some "functionality") with libxml2 and perl (using the XML::LibXML::Reader interface), where it' seems to be stumbling on the series of: > > \n > > if I insert a linebreak; (\n\n) the libxml2 reader function rips through it no problem, if there's no line break, it views the as a blank element and then reads the rest of the file as garbage data (trying to do this stream like instead of DOM like). > > I'm assuming there isn't anything wrong with the way it's outputted (guessing most people are just java-nuts and it works like that), but i'm curious if anyone has gotten around this issue with libxml2 (or alike) by setting some sort of parsing flag, etc. > > I've tested it a few times with the first few set of asn elements you'll find in the data; and the line break pretty much makes it reproducible... jw if anyone has worked around that from the perl side... Wes, The line break shouldn't be necessary. If you have validation turned on, perhaps that is causing some sort of problem. Also check other parsing options. I just ran last night's bulk whois through xmllint with the --stream option and it worked fine. xmllint uses libxml. Sorry I can be of further help here. -andy From wes at ren-isac.net Sat Sep 25 21:15:14 2010 From: wes at ren-isac.net (Wes Young) Date: Sat, 25 Sep 2010 21:15:14 -0400 Subject: [arin-tech-discuss] bulkwhois libxml2 In-Reply-To: <1403BE18-7F30-4BDD-801C-473889489AB8@arin.net> References: <1403BE18-7F30-4BDD-801C-473889489AB8@arin.net> Message-ID: gak. Ok, I was mis-reading the XML::LibXML::Reader implementation. As a followup here's what I came up with which seems to work (missing a few things, but people will get the idea). #!/usr/bin/perl use strict; use warnings; use XML::LibXML::Reader; use XML::LibXML::XPathContext; use XML::LibXML; my $cache = "/tmp/arin_db.xml"; my $reader = XML::LibXML::Reader->new('location' => $cache, load_ext_dtd => 0); my $xpc = XML::LibXML::XPathContext->new(); $xpc->registerNs('arin','http://www.arin.net/bulkwhois/core/v1'); my $x = 0; while ($reader->read()){ # prime the pump... next unless($reader->name() eq 'asn'); last; } do { # copy the node into memory (as a DOM for searching, probably leaky, but it works) my $node = $reader->copyCurrentNode(1); for(lc($reader->name())){ # select case for the various top-level nodes found within bulkwhois/ core/v1 if(/^asn$/){ # need to search via the namespace.... print $xpc->find('./arin:startAsNumber',$node)."\n"; } if(/^....$/) { ... } last; } # jump top level nodes, skipping children } while ($reader->nextSibling()); Some stats (within a vm): $> time perl arin_parse.pl real 3m28.060s user 3m9.504s sys 0m15.633s total mem wasted: 450meg (mostly cause it's perl, easy to mitigate leaking by using threads) cheers, On Sep 22, 2010, at 9:51 AM, Andy Newton wrote: > > On Sep 21, 2010, at 3:37 PM, Wes Young wrote: > >> Figured this was a good place to start... >> >> i'm debugging an issue (or some "functionality") with libxml2 and >> perl (using the XML::LibXML::Reader interface), where it' seems to >> be stumbling on the series of: >> >> \n >> >> if I insert a linebreak; (\n\n) the libxml2 reader >> function rips through it no problem, if there's no line break, it >> views the as a blank element and then reads the rest of the >> file as garbage data (trying to do this stream like instead of DOM >> like). >> >> I'm assuming there isn't anything wrong with the way it's outputted >> (guessing most people are just java-nuts and it works like that), >> but i'm curious if anyone has gotten around this issue with libxml2 >> (or alike) by setting some sort of parsing flag, etc. >> >> I've tested it a few times with the first few set of asn elements >> you'll find in the data; and the line break pretty much makes it >> reproducible... jw if anyone has worked around that from the perl >> side... > > Wes, > > The line break shouldn't be necessary. If you have validation turned > on, perhaps that is causing some sort of problem. Also check other > parsing options. > > I just ran last night's bulk whois through xmllint with the --stream > option and it worked fine. xmllint uses libxml. > > Sorry I can be of further help here. > > -andy > -- Wes http://claimid.com/wesyoung -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: