Re: Caching query results

From: Andrus Adamchik (andru..bjectstyle.org)
Date: Wed Sep 27 2006 - 16:43:33 EDT

  • Next message: Andrus Adamchik: "Scratch Confluence space created [Was: Cayenne web site]"

    Hi Francesco,

    I see. Essentially DataObject -> Query set association would be an
    analogue of 3.0 cache groups (where group name == entity name).

    Actually in-memory checking procedure the way you describe it may not
    be as bad as I originally thought. Matching object state before and
    after commit against all cached queries for a given entity is
    proportional to the number of cached queries (including different
    parameter permutations of the same query), not the number of objects
    in the results. So Cayenne will not turn itself in a DB engine :-)

    If that works for you and you are willing to contribute it to Apache
    once it is polished, we may include it as a pluggable strategy...
    possibly a QueryCache decorator...

    http://svn.apache.org/repos/asf/incubator/cayenne/main/trunk/core/
    cayenne-jdk1.4/src/main/java/org/apache/cayenne/cache/QueryCache.java

    Andrus

    On Sep 26, 2006, at 9:14 AM, Francesco Fuzio wrote:

    > Hi Andrus,
    >
    > first of all thank you for the prompt support and your suggestions.
    >
    > As you correctly guessed I was talking about Cayenne 1.2.1
    >
    > I was thinking about this automatic (but based on custom
    > configuration) "invalidation algorithm":
    >
    > Configure somewhere a logic association DataObject--->"QueryData"[]
    > (ex Paintings DataObject has to be associated to the queries
    > [Select * from Paintings where year = 1300 | Select * from Artists,
    > Paintings where Paintings.year >1200 | Select * from Artists,
    > Paintings where Paintings.year >1200 order by Paintings.name]
    >
    > The "QueryData" object shoud contain, separately, information about
    > the "expression" (i.e. the "where" part ) and about ordering.
    > "Select * from Artists, Paintings where Paintings.year >1200 order
    > by Paintings.name" ---> { Paintings.year >1200 | order by
    > Paintings.name }
    >
    > If I modify (or create or delete) a DataObject I have to check the
    > ante and post modification version of the single DataObject against
    > the associated QueryData's
    > We could do this exploiting the Objects filtering capabilities
    > Expression (or optionally using third parties utilities <commons-
    > bean-utils??>) :
    >
    > Expression filter = Expression.fromString("Paintings.year >1200");
    > filter.filterObjects(objects);
    >
    > As a result we would have two Sets of Queries : those matching
    > before the modification and those matching after the modification.
    > For sure we have to invalidate all the query results that are not
    > in the intersection of the two sets.
    >
    > For the queries in the intersection:
    >
    > a)If they have NO ordering (Order by clause , paging limitation
    > etc) they are still valid
    >
    > b)If they have ordering: if ordering is on one of the modified
    > DataObject field, we have to invalidate the query result, otherwise
    > the query result is still valid.
    >
    > Of course this solution can lead to high computational resources
    > use, dependent on the number of queries it has to check.
    > But, for example in the project I am collaborating to, the Db is
    > the "under pressure"/bottleneck system and the Middleware has much
    > less load.
    > In such a situation "moving" load from the Db to the Mw is a
    > benefit for the Application as a whole.
    >
    > For "basic" queries (I made some tests) I think the algorithm
    > should work. Of course more systematic test cases should be
    > performed to completely validate the algorithm and/or find its
    > limitation.
    > Anyway I wanted to share it with you hoping it can be useful or can
    > be of some "inspiration" for a proper/more correct solution.
    >
    >
    > Francesco.



    This archive was generated by hypermail 2.0.0 : Wed Sep 27 2006 - 16:43:51 EDT