5221 2007-10-23 07:03:05 +0000 Implement SkipHost 2008-11-18 17:40:44 +0000 1 1 1 Unclassified LogValidator core unspecified All All RESOLVED FIXED http://lists.w3.org/Archives/Public/www-validator/2007Oct/0103.html P2 enhancement --- 1 ot ot martins oldest_to_newest 17350 0 ot 2007-10-23 07:03:05 +0000 As suggested by Martin Smith, a way for the log parser to skip certain log records if they came from specific IP (similar to the SkipHost [1] directive in awstats) would be useful. [1] http://awstats.sourceforge.net/docs/awstats_config.html#SkipHosts Ways to filter out on other criteria (referer, etc) also suggested. 17391 1 495 martins 2007-10-24 17:26:43 +0000 Created attachment 495 Patch to check for new ExcludeHosts configuration directive Patch to add new configuration behavior and corresponding configuration file entry: -- begin ## ExcludeHosts : records that should not be processed ## ## space seperated list of expressions ## regexp-like syntax for match on the remote host or address ## DEFAULT = None ## NOTE: This uses whatever is in your log file, so you may want to ask Apache to start/stop resolving ## remote addresses to host names if you want to do hostname-based matching # ExcludeHosts ^localhost ^127\.0\.0\.1$ ExcludeHosts www\.bebr\.ufl\.edu ^inp-susanf 17392 2 martins 2007-10-24 17:27:50 +0000 I chose to implement a directive with a name similar to that of ExcludeAreas: ## ExcludeHosts : records that should not be processed ## ## space seperated list of expressions ## regexp-like syntax for match on the remote host or address ## DEFAULT = None ## NOTE: This uses whatever is in your log file, so you may want to ask Apache to start/stop resolving ## remote addresses to host names if you want to do hostname-based matching # ExcludeHosts ^localhost ^127\.0\.0\.1$ 17393 3 martins 2007-10-24 17:29:24 +0000 (In reply to comment #1) By the way, my original comment had an extra line that I was using to test -- it can be removed in favor of the next one :) 22465 4 ot 2008-11-18 15:48:51 +0000 Hi Martin, Not sure why this disappeared from my radar for so long. I'm looking at your patch(es) now. 22468 5 ot 2008-11-18 17:15:08 +0000 OK, modulo one small issue with the excluding of records when a host/IP was matching, the patches were functional. I made a little fix and we're good to go (in CVS, and in CPAN by the end of the week). 22469 6 martins 2008-11-18 17:40:44 +0000 Thanks again Olivier. I'm glad it finally made it in :) 495 2007-10-24 17:26:43 +0000 2007-10-24 17:26:43 +0000 Patch to check for new ExcludeHosts configuration directive LogValidator.pm.diff text/plain 1820 martins MjYxYTI2MiwyNjMKPiAJbXkgJGFsbHNraXBob3N0cyA9ICgkY29uZmlne0xvZ1Byb2Nlc3Nvcn17 RXhjbHVkZUhvc3RzfSkgPyAkY29uZmlne0xvZ1Byb2Nlc3Nvcn17RXhjbHVkZUhvc3RzfSA6ICIi OyAjIGRlZmF1bHQgdG8gbm9uZQo+IAlteSBAc2tpcGhvc3RzcmVnZXggPSBzcGxpdCgiICIsICRh bGxza2lwaG9zdHMpOwoyNzZhMjc5LDI5MQo+IAkJCQkJbXkgJHRtcF9yZWNvcmRfcmVtb3RlX2Fk ZHIgPSAkc2VsZi0+ZmluZF9yZW1vdGVfYWRkcigkdG1wX3JlY29yZCwgJGxvZ3R5cGUpOwo+IAkJ CQkJaWYgKCR0bXBfcmVjb3JkX3JlbW90ZV9hZGRyKSAjIG5vdCBhIGJsYW5rIHJlbW90ZSBob3N0 IG9yIGFkZHJlc3MKPiAJCQkJCXsKPiAJCQkJCQlmb3JlYWNoIG15ICRza2lwZXhwcmVzc2lvbiAo QHNraXBob3N0c3JlZ2V4KQo+IAkJCQkJCXsKPiAJCQkJCQkgICAgIGlmKCAkdG1wX3JlY29yZF9y ZW1vdGVfYWRkciA9fiAvJHNraXBleHByZXNzaW9uLyApCj4gCQkJCQkJICAgICB7Cj4gCQkJCQkJ CXByaW50ICIgU2tpcHBpbmcgIiAuICR0bXBfcmVjb3JkX3JlbW90ZV9hZGRyIC4gIiBiZWNhdXNl IGl0IG1hdGNoZXMgdGhlIEV4Y2x1ZGVIb3N0cyBwYXR0ZXJuICIgLiAkc2tpcGV4cHJlc3Npb24u ICJcbiIgaWYgKCR2ZXJib3NlID4gMik7Cj4gCQkJCQkJCW5leHQ7Cj4gCQkJCQkJICAgICB9Cj4g CQkJCQkJfQo+IAkJCQkJfQo+IAozMzlhMzU1LDM4MAo+IHN1YiBmaW5kX3JlbW90ZV9hZGRyCj4g IyBmaW5kcyB0aGUgcmV0dXJuZWQgSFRUUCBjb2RlIGZyb20gYSBsb2cgcmVjb3JkLCBpZiBhdmFp bGFibGUKPiB7Cj4gICAgICAgICBteSAkc2VsZiA9IHNoaWZ0Owo+ICAgICAgICAgaWYgKEBfKQo+ ICAgICAgICAgewo+ICAgICAgICAgICAgICAgICBteSAkdG1wcmVjb3JkID0gc2hpZnQ7Cj4gICAg ICAgICAgICAgICAgIG15IEByZWNvcmRfYXJyeTsKPiAgICAgICAgICAgICAgICAgQHJlY29yZF9h cnJ5ID0gc3BsaXQoIiAiLCAkdG1wcmVjb3JkKTsKPiAgICAgICAgICAgICAgICAgIyBoYXJkY29k ZWQgdG8gbW9zdCBhcGFjaGUgbG9nIGZvcm1hdHMsIGluY2x1ZGVkIGNvbW1vbiBhbmQgY29tYmlu ZWQKPiAgICAgICAgICAgICAgICAgIyBmb3IgdGhlIG1vbWVudC4uLiBUT0RPCj4gICAgICAgICAg ICAgICAgIG15ICRsb2d0eXBlID0gc2hpZnQ7Cj4gICAgICAgICAgICAgICAgICMgcHJpbnQgImxv ZyB0eXBlICRsb2d0eXBlIiBpZiAoJHZlcmJvc2UgPiAyKTsKPiAgICAgICAgICAgICAgICAgaWYg KCRsb2d0eXBlIGVxICJwbGFpbiIpCj4gICAgICAgICAgICAgICAgIHsKPiAgICAgICAgICAgICAg ICAgICAgICAgICAkdG1wcmVjb3JkID0gIiI7Cj4gICAgICAgICAgICAgICAgIH0KPiAgICAgICAg ICAgICAgICAgZWxzZSAjY29tbW9uIGNvbWJpbmVkIGZ1bGwgb3IgdzNjCj4gICAgICAgICAgICAg ICAgIHsKPiAgICAgICAgICAgICAgICAgICAgICAgICAkdG1wcmVjb3JkID0gJHJlY29yZF9hcnJ5 WzBdOwo+ICAgICAgICAgICAgICAgICB9Cj4gICAgICAgICAjcHJpbnQgIlJlbW90ZSBBZGRyICR0 bXByZWNvcmQgXG4iIGlmICgoJHZlcmJvc2UgPiAyKSBhbmQgKCR0bXByZWNvcmQgbmUgIiIpKTsK PiAgICAgICAgIHJldHVybiAkdG1wcmVjb3JkOwo+ICAgICAgICAgfQo+IH0KPiAKNjYzYTcwNSw3 MDgKPiA9aXRlbSAkcHJvY2Vzc29yLT5maW5kX3JlbW90ZV9hZGRyCj4gCj4gR2l2ZW4gYSBsb2cg cmVjb3JkIGFuZCB0aGUgdHlwZSBvZiB0aGUgbG9nIChjb21tb24gbG9nIGZvcm1hdCwgZmxhdCBs aXN0IG9mIFVSSXMsIGV0YyksIGV4dHJhY3RzIHRoZSByZW1vdGUgaG9zdCBvciBpcAo+IAo=