Re: proposal on ISSUE-94 ("lossless paging")

Hi Ashok,

I think we once discussed this problem and this can be resolved/avoided if
the client is aware that complete representation has changed while it
is traversing the pages. For example, if the client gets an E-tag for the
page as well as an E-tag for complete representation, once it gets the page
2 it knows that the overall representation has changed. So it can try to
get the representation from the beginning.

This can be done efficiently using the 'If-None-Match' so that if the
client can know whether the change is in one of the pages that it already
got or in some other page that it is yet to get. For instance, in the
previous example if the change was in page 3 the client doesn't have to do
anything and can move on just adjusting the new E-tag for the complete
representation.

However, for reasons that I don't recall we didn't use this approach.

Best Regards,
Nandana

On Mon, Mar 24, 2014 at 10:10 PM, Ashok Malhotra
<ashok.malhotra@oracle.com>wrote:

> Hi Ted:
> The database situation is a bit different because it deals with committed
> data
> and uncommitted data.  In the LDP world the parallel might be cached data
> vs.
> rereading data from the server.  I tried to create examples where
> deletions and
> insertions cause problems.  These assume 10 items to a page and fixed size
> pages.
>
> Example 1:
> User reads page 1.  While he is working with the data some other user
> deletes
> item 5.  This causes item 11 to be moved to page 1.  Now, when he goes to
> page 2
> item 11 does not appear and is lost.  If he rereads page 1 from the cache,
> item 5 is
> still there but if he refreshes the cache he gets items 1 to 4 and 6 to
> 11.  That is,
> rereads can be inconsistent.
>
> Example 2:
> User reads page 1.  While he is working with the data some other user adds
> an item
> between items 5 and 6.   This causes item 10 to be moved to page 2.  If he
> goes to
> page 2 he sees item 10 again --- bug!  If he goes back to page 1 and
> refreshes the
> cache he gets items 1 to 5, new Item, items 6 to 9 --- inconsistent reread.
>
> Sandro's proposal that allows a variable number of items gets around some
> of these
> problems.
>
> All the best, Ashok
>
> On 3/24/2014 12:44 PM, Ted Thibodeau Jr wrote:
>
>> Sandro wrote:
>>
>>> PROPOSED: Change LDP paging to no longer be characterized as
>>> "lossy". Specifically, a client who pages all the way through
>>> (forward or backward) MUST be sent every triple in intersect
>>> (g0, g1, ... gn) and MAY be sent any triples in the union (g0,
>>> g1,  .. gn), where Gi is the paged graph at the time client
>>> makes request i.  Servers MAY refuse to serve any page because
>>> they don't want to implement this kind of paging in certain
>>> cases; if they do so, they MUST return a 410 GONE.
>>>
>>
>> I think Sandro's suggestion remains "lossy", in that insertions
>> *may* be made visible, but any deletions are to be concealed.
>> This bothers me.
>>
>> As I said in the con-call, I think the current paging thoughts
>> are a conflation of two concepts from the (R)DBMS world -
>> "(scrollable) cursors" and "(transaction) isolation".
>>
>> I strongly recommend some investigation of these concepts.
>> These wikipedia pages are imperfect but provide a good, if
>> basic, starting point.
>>
>>     <https://en.wikipedia.org/wiki/Isolation_%28database_systems%29>
>>
>>     <https://en.wikipedia.org/wiki/Cursor_%28databases%29>
>>
>> I recommend particular attention to the "Isolation Levels vs
>> Read Phenomena" table on the Isolation page.
>>
>> There are tradeoffs to each and every option -- and whether any
>> given tradeoff is more or less important depends on the use case --
>> and this is why the DBMS world mostly sees every engine implement
>> all cursors and isolation levels, and mostly allows clients to
>> specify which ones they want/need to use.
>>
>> I would suggest that we consider writing this version of the
>> LDP spec to only require implementation of the lowest-level,
>> least-effort options in each.
>>
>> For the cursor, this is the "FORWARD ONLY" a/k/a "non-scrollable"
>> cursor.  The scroll only goes from one end to the other, without
>> any reversing midway.  When working with an unORDERed result set,
>> you can only go in the arbitrary "forward"  direction.  When
>> working with an ORDERed result set, it is really "unidirectionally
>> scrollable" because it can be walked in either direction by simply
>> inverting the ORDER BY.
>>
>> For isolation, this is "READ UNCOMMITTED", which basically means
>> that the client can't rely on anything being the same *now* as it
>> was *then*, no matter how short the window between.  The client
>> gets a snapshot-in-time for each request, but has no way to tell
>> the server "act on *that* snapshot-in-time" nor how to act based
>> on *other* client requests received in the interim (or received
>> after mine, but based on the same state as my request was...)
>>
>> I think this leaves a clear path for future evolution (adding
>> the other cursors and/or isolation levels), while making the
>> current spec easier to write and early implementations easier
>> to produce.
>>
>> I hope this is more helpful than painful.
>>
>> Ted
>>
>>
>>
>>
>> --
>> A: Yes.                      http://www.guckes.net/faq/attribution.html
>> | Q: Are you sure?
>> | | A: Because it reverses the logical flow of conversation.
>> | | | Q: Why is top posting frowned upon?
>>
>> Ted Thibodeau, Jr.           //               voice +1-781-273-0900 x32
>> Senior Support & Evangelism  //        mailto:tthibodeau@openlinksw.com
>>                               //              http://twitter.com/TallTed
>> OpenLink Software, Inc.      //              http://www.openlinksw.com/
>>           10 Burlington Mall Road, Suite 265, Burlington MA 01803
>>       Weblog   -- http://www.openlinksw.com/blogs/
>>       LinkedIn -- http://www.linkedin.com/company/openlink-software/
>>       Twitter  -- http://twitter.com/OpenLink
>>       Google+  -- http://plus.google.com/100570109519069333827/
>>       Facebook -- http://www.facebook.com/OpenLinkSoftware
>> Universal Data Access, Integration, and Management Technology Providers
>>
>