On Monday, September 8, 2003, at 11:07 AM, Scott Finnerty wrote:
> However, some times I would like to use the Iterator to process the
> result
> set, but this always forces a full retrieval and in the case of large
> result
> sets with PostgreSQL, a subsequent failure. Monkeying with the
> PostgreSQL
> parameter is not an ideal solution. How about returning a custom
> Iterator
> that resolves pages at a time as the first element in an unresolved
> page is
> requested from the Iterator? This would space out the hits to the
> database
> in page size chunks and should address the problem with PostgreSQL?
> I'd be
> willing to code up such a modification to IncrementalFaultList if this
> solution sounds acceptable.
Hi Scott,
+1 for a lazy iterator. I guess I never cared too much about this in
the past, since I always used sublists. But your point is valid, and
your patch is very welcomed.
As for the huge IN query that happens if "resolveAll" is called on
IncrementalFaultList, I think we can patch it too, independently from
the Iterator. It may run a SELECT...WHERE...IN in chunks of reasonable
(configurable) size that should be bigger than a typical page size for
better performance. I'll open a bug for this second part, but will
probably mark it as "after 1.0".
Andrus
This archive was generated by hypermail 2.0.0 : Mon Sep 08 2003 - 11:29:43 EDT