I thought I was taking advantage of the "tuning" ability of a query's
results by setting the page size - however in the logs I kept seeing a
second SELECT for all of the rest of the results set - even though I was
just iterating over the first page. I was finally forced to figure it out
when PostgreSQL puked because the IN clause had more than its limit of
elements (defaults to 10,000). So I looked a little closer and found that
because I was using a JSTL tag to loop through the first page, I was
invoking the iterator on IncrementalFaultList which forces the retrieval of
the entire result set as objects. I was able to work-around by adding a
getter to return the current page of results as a sublist.
However, some times I would like to use the Iterator to process the result
set, but this always forces a full retrieval and in the case of large result
sets with PostgreSQL, a subsequent failure. Monkeying with the PostgreSQL
parameter is not an ideal solution. How about returning a custom Iterator
that resolves pages at a time as the first element in an unresolved page is
requested from the Iterator? This would space out the hits to the database
in page size chunks and should address the problem with PostgreSQL? I'd be
willing to code up such a modification to IncrementalFaultList if this
solution sounds acceptable.
Scott Finnerty
This archive was generated by hypermail 2.0.0 : Mon Sep 08 2003 - 11:05:32 EDT