Re: ERXJDBCConnectionBroker Question

From: Brendan Duddridge (brenda..lickspace.com)
Date: Fri Aug 27 2010 - 16:55:13 UTC

  • Next message: Dov Rosenberg: "Re: ERXJDBCConnectionBroker Question"

    We use concurrent request handling and have a separate OSC per worker thread. Gives us good concurrency in our apps. But they're mostly read-only or in cases where it's not, we just make sure we refresh from the database when needed. But we definitely see multiple concurrent connections to the database in our apps.

    ____________________________________________________________________
    Brendan Duddridge | CTO | 403-277-5591 x24 | brenda..lickspace.com

    ClickSpace Interactive Inc.
    Suite L100, 239 - 10th Ave. SE
    Calgary, AB T2G 0V9

    http://www.clickspace.com

    On 2010-08-27, at 9:48 AM, Dov Rosenberg wrote:

    > I turned off the ERXJDBCConenctionBroker and set the
    > er.extensions.ERXObjectStoreCoordinatorPool.maxCoordinators = 10. When I run
    > my application under a load test I don't see any additional db connections
    > being made. I can see some of these messages in my logs:
    >
    > INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - setting
    > ERXEC.factory to MultiOSCFactory
    > INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - initializing Pool...
    > INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - initializing Pool
    > finished
    >
    > So I assume that I should see multiple db connections at some point. My load
    > test starts a new thread for each virtual user. Even with lazy loading I
    > would have expected to see at least 10 DB connections once I reached 15
    > virtual users.
    >
    > Am I missing a config param somewhere?
    >
    > Thanks in advance
    >
    > Dov Rosenberg
    >
    >
    >
    > On 8/27/10 1:11 AM, "Mike Schrag" <mschra..obox.com> wrote:
    >
    >> It's not wrong, I suppose, if you're doing direct custom access to the
    >> database. In that case, you would actually benefit from multiple database
    >> channels. But you're not going to benefit from EOF :)
    >>
    >> ms
    >>
    >> On Aug 26, 2010, at 11:55 PM, Chuck Hill wrote:
    >>
    >>> I remember those docs. I think they are just wrong.
    >>>
    >>> Chuck
    >>>
    >>>
    >>> On Aug 26, 2010, at 10:21 PM, Dov Rosenberg wrote:
    >>>
    >>>> Looking thru some old EOF documentation (v5.2) I found -
    >>>> http://developer.apple.com/legacy/mac/library/documentation/WebObjects/Enter
    >>>> prise_Objects/EnterpriseObjects.pdf
    >>>>
    >>>> The second step in instrumenting multithreading in an Enterprise Objects
    >>>> application is determining what
    >>>> kind of concurrency you need. This requires knowing (or predicting) the
    >>>> bottlenecks within your application.
    >>>> Do bottlenecks occur at the database level when multiple users attempt
    >>>> concurrent access to the data source
    >>>> so that adding more database channels alleviates the bottleneck? Do
    >>>> bottlenecks occur elsewhere in the
    >>>> access layer so that providing a separate access layer for each user
    >>>> alleviates the bottleneck?
    >>>> The answers to these questions help determine the mechanism you need to use
    >>>> to instrument concurrency
    >>>> within an Enterprise Objects application. Two common design patterns for
    >>>> concurrency within Enterprise
    >>>> Objects applications are to:
    >>>> ¡½ Provide each user with an independent access layer stack.
    >>>> ¡½ Provide multiple database channels on demand.
    >>>>
    >>>> The doc goes onto say that using multiple OSC introduces additional
    >>>> challenges with concurrency between OSC¡¦s.
    >>>>
    >>>> The doc even gives code samples on providing multiple OSC and adding
    >>>> additional database channels when a DatabaseChannelNeededNotification is
    >>>> received
    >>>>
    >>>> I am interested to hear your analysis of the Wonder code. I will keep
    >>>> experimenting
    >>>>
    >>>> Dov
    >>>>
    >>>> On 8/26/10 9:58 PM, "Mike Schrag" <mschra..obox.com> wrote:
    >>>>
    >>>>> I can't vouch for connectionbroker's accuracy of impl, but I would expect
    >>>>> all
    >>>>> your db access to be behind a single lock and therefore not benefit from
    >>>>> multiple channels. Of course most of the people who might challenge my
    >>>>> claims
    >>>>> are probably drunk in Montreal right now. I'll check source later tonight
    >>>>> and
    >>>>> see if I'm talking crazy.
    >>>>>
    >>>>> Sent from my iPhone
    >>>>>
    >>>>> On Aug 26, 2010, at 9:55 PM, Dov Rosenberg <DRosenber..nquira.Com> wrote:
    >>>>>
    >>>>>> Is it not a good idea to use the ERXJDBCAdaptor and ConnectionBroker
    >>>>>> together?
    >>>>>>
    >>>>>> I would have thought if I only used a single OSC it would have made sure
    >>>>>> that each transaction went to the correct connection.
    >>>>>>
    >>>>>> Should there only be a single db connection for each OSC?
    >>>>>>
    >>>>>> Thanks
    >>>>>>
    >>>>>> Dov Rosenberg
    >>>>>>
    >>>>>>
    >>>>>> On 8/26/10 9:46 PM, "Mike Schrag" <mschra..obox.com> wrote:
    >>>>>>
    >>>>>>> With one osc you saw activity on multiple connections concurrently? I
    >>>>>>> can't
    >>>>>>> imagine that scenario has a happy ending. I don't even know how I would
    >>>>>>> be
    >>>>>>> possible given that all your db access should be behind a dbc lock. You
    >>>>>>> might
    >>>>>>> see use of multiple connections, but that would probably explain your
    >>>>>>> failing
    >>>>>>> commits (inserting on conn 1, committing conn 2, maybe). Each osc should
    >>>>>>> end
    >>>>>>> up having its own conn and you should see parallel access that way.
    >>>>>>>
    >>>>>>> To answer more I think I need to not be on an iPhone :)
    >>>>>>>
    >>>>>>> Sent from my iPhone
    >>>>>>>
    >>>>>>> On Aug 26, 2010, at 9:37 PM, Dov Rosenberg <DRosenber..nquira.Com>
    >>>>>>> wrote:
    >>>>>>>
    >>>>>>>> Hmmm - I did a little jmeter test and definitely saw an improvement in
    >>>>>>>> page
    >>>>>>>> view performance and saw activity on multiple DB connections during the
    >>>>>>>> test. By simply adding the connection broker with a single OSC.
    >>>>>>>>
    >>>>>>>> If I set up object store pooling wont that increase issues with
    >>>>>>>> concurrency
    >>>>>>>> within my app? I.e. Trying to update data that has already been updated
    >>>>>>>> in >>>> a
    >>>>>>>> different OSC?
    >>>>>>>>
    >>>>>>>> I am using the Jgroups synchronizer between instances already.
    >>>>>>>>
    >>>>>>>> Would the following set of properties be consistent with each other:
    >>>>>>>>
    >>>>>>>> er.extensions.ERXObjectStoreCoordinatorPool.maxCoordinators = 10
    >>>>>>>> er.extensions.ERXJDBCAdaptor.className=er.extensions.jdbc.ERXJDBCAdaptor
    >>>>>>>> er.extensions.ERXJDBCAdaptor.useConnectionBroker = true
    >>>>>>>> er.extensions.remoteSynchronizer.enabled=true
    >>>>>>>> er.extensions.remoteSynchronizer=er.jgroups.ERJGroupsSynchronizer
    >>>>>>>> dbMinConnectionsGLOBAL=10
    >>>>>>>> dbMaxConnectionsGLOBAL=15
    >>>>>>>> er.extensions.ERXJDBCConnectionBroker.maxConnections=15
    >>>>>>>> er.extensions.ERXJDBCConnectionBroker.minConnections=10
    >>>>>>>>
    >>>>>>>> If I understand what is going on I should get 10 EOF stacks sharing a
    >>>>>>>> pool
    >>>>>>>> of 15 database connections. I assume there is some magic running behind
    >>>>>>>> the
    >>>>>>>> scenes to keep all of the OSC in sync with each other?
    >>>>>>>>
    >>>>>>>> Thanks again
    >>>>>>>>
    >>>>>>>> Dov Rosenberg
    >>>>>>>>
    >>>>>>>>
    >>>>>>>>
    >>>>>>>> On 8/26/10 9:00 PM, "Mike Schrag" <mschra..obox.com> wrote:
    >>>>>>>>
    >>>>>>>>> Connection pooling won't really do anything for you because each stack
    >>>>>>>>> is
    >>>>>>>>> single threaded. You want object store coordinator pooling.
    >>>>>>>>>
    >>>>>>>>> Sent from my iPhone
    >>>>>>>>>
    >>>>>>>>> On Aug 26, 2010, at 8:45 PM, Dov Rosenberg <DRosenber..nquira.Com>
    >>>>>>>>> wrote:
    >>>>>>>>>
    >>>>>>>>>> During some testing today I turned on support for connection pooling
    >>>>>>>>>> in
    >>>>>>>>>> my
    >>>>>>>>>> application using the ERXJDBCAdaptor and the ERXJDBCConnectionBroker.
    >>>>>>>>>> I
    >>>>>>>>>> could
    >>>>>>>>>> see the connections being used fine and all of the things that did
    >>>>>>>>>> fetches
    >>>>>>>>>> seemed to work without any issues. However when I tried doing
    >>>>>>>>>> something
    >>>>>>>>>> that
    >>>>>>>>>> generated an INSERT the transactions did not commit and no changes
    >>>>>>>>>> were
    >>>>>>>>>> made.
    >>>>>>>>>> I could see the SQL being generated as expected and the saveChanges()
    >>>>>>>>>> happened without throwing any exception ¡V just no data got written to
    >>>>>>>>>> the
    >>>>>>>>>> database. As soon as I disabled the use of the connection pool
    >>>>>>>>>> everything
    >>>>>>>>>> worked properly again.
    >>>>>>>>>>
    >>>>>>>>>> Any thoughts would be appreciated
    >>>>>>>>>>
    >>>>>>>>>> Dov Rosenberg
    >>>>>>>>>> _______________________________________________
    >>>>>>>>>> Do not post admin requests to the list. They will be ignored.
    >>>>>>>>>> Webobjects-dev mailing list (Webobjects-de..ists.apple.com)
    >>>>>>>>>> Help/Unsubscribe/Update your Subscription:
    >>>>>>>>>> http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40pobox.
    >>>>>>>>>> com
    >>>>>>>>>>
    >>>>>>>>>> This email sent to mschra..obox.com
    >>>>>>>>
    >>>>>>
    >>>> _______________________________________________
    >>>> Do not post admin requests to the list. They will be ignored.
    >>>> Webobjects-dev mailing list (Webobjects-de..ists.apple.com)
    >>>> Help/Unsubscribe/Update your Subscription:
    >>>> http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village
    >>>> .net
    >>>>
    >>>> This email sent to chil..lobal-village.net
    >>>
    >>> --
    >>> Chuck Hill Senior Consultant / VP Development
    >>>
    >>> Practical WebObjects - for developers who want to increase their overall
    >>> knowledge of WebObjects or who are trying to solve specific problems.
    >>> http://www.global-village.net/products/practical_webobjects
    >>>
    >>>
    >>>
    >>>
    >>>
    >>>
    >>>
    >>
    >



    This archive was generated by hypermail 2.0.0 : Fri Aug 27 2010 - 16:56:04 UTC