6827 2009-04-16 14:01:08 +0000 Resource retrieval: the same URI may be retrieved more than once 2009-04-30 15:49:37 +0000 1 1 1 Unclassified mobileOK Basic checker Java Library unspecified Other All RESOLVED FIXED P2 enhancement --- 1 fd abel.rionda oldest_to_newest 24772 0 fd 2009-04-16 14:01:08 +0000 Description ----- The same URI may appear more than once in the list of resources to retrieve, because they may appear in different contexts (typically image and object). Although resource extractors try to re-use retrieval material as much as possible, two resources with the same URI may still have to be created with no retrieval information. The resources would both be retrieved, meaning that there will be two requests to the same URI. TODO ----- To ensure that URIs are retrieved only once, we may: 1. check that there exists retrieval information right before the retrieval process starts, but that would still leave open the possibility that the same URI are retrieved at once 2. introduce some kind of dependency between resources to postpone retrieval of a resource while a resource with the same URI is being retrieved and then re-use the retrieval material. 24973 1 fd 2009-04-30 15:49:37 +0000 The Preprocessor now maintains a hashtable of the URIs (fragment section excepted) to retrieve associated with a reference resource. Resources are only processed when they are the reference resource or when the reference resource associated with their URI has been retrieved. The Preprocessor.processResource method uses the retrieval information of the reference resource when available, and call Resource.retrieve if not. This was previously being done by extractors, but did not quite fit there anyway so I removed this bit from the extractors implementations.