Re: ERXJDBCConnectionBroker Question

From: Dov Rosenberg (DRosenber..nquira.Com)
Date: Fri Aug 27 2010 - 17:13:14 UTC

  • Next message: Henrique Prange: "[ANN] Maven WOLifecycle Plug-in 2.2.1"

    How did you configure it to work? I seem to be missing something in my
    configuration.

    Can you provide your Properties file settings

    Thanks

    Dov Rosenberg


    On 8/27/10 12:55 PM, "Brendan Duddridge" <brendan@clickspace.com> wrote:

    > We use concurrent request handling and have a separate OSC per worker thread.
    > Gives us good concurrency in our apps. But they're mostly read-only or in
    > cases where it's not, we just make sure we refresh from the database when
    > needed. But we definitely see multiple concurrent connections to the database
    > in our apps.
    >
    > ____________________________________________________________________
    > Brendan Duddridge | CTO | 403-277-5591 x24 | brendan@clickspace.com
    >
    > ClickSpace Interactive Inc.
    > Suite L100, 239 - 10th Ave. SE
    > Calgary, AB T2G 0V9
    >
    > http://www.clickspace.com
    >
    > On 2010-08-27, at 9:48 AM, Dov Rosenberg wrote:
    >
    >> I turned off the ERXJDBCConenctionBroker and set the
    >> er.extensions.ERXObjectStoreCoordinatorPool.maxCoordinators = 10. When I run
    >> my application under a load test I don't see any additional db connections
    >> being made. I can see some of these messages in my logs:
    >>
    >> INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - setting
    >> ERXEC.factory to MultiOSCFactory
    >> INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - initializing Pool...
    >> INFO er.extensions.eof.ERXObjectStoreCoordinatorPool - initializing Pool
    >> finished
    >>
    >> So I assume that I should see multiple db connections at some point. My load
    >> test starts a new thread for each virtual user. Even with lazy loading I
    >> would have expected to see at least 10 DB connections once I reached 15
    >> virtual users.
    >>
    >> Am I missing a config param somewhere?
    >>
    >> Thanks in advance
    >>
    >> Dov Rosenberg
    >>
    >>
    >>
    >> On 8/27/10 1:11 AM, "Mike Schrag" <mschrag@pobox.com> wrote:
    >>
    >>> It's not wrong, I suppose, if you're doing direct custom access to the
    >>> database. In that case, you would actually benefit from multiple database
    >>> channels. But you're not going to benefit from EOF :)
    >>>
    >>> ms
    >>>
    >>> On Aug 26, 2010, at 11:55 PM, Chuck Hill wrote:
    >>>
    >>>> I remember those docs. I think they are just wrong.
    >>>>
    >>>> Chuck
    >>>>
    >>>>
    >>>> On Aug 26, 2010, at 10:21 PM, Dov Rosenberg wrote:
    >>>>
    >>>>> Looking thru some old EOF documentation (v5.2) I found -
    >>>>> http://developer.apple.com/legacy/mac/library/documentation/WebObjects/Ent
    >>>>> er
    >>>>> prise_Objects/EnterpriseObjects.pdf
    >>>>>
    >>>>> The second step in instrumenting multithreading in an Enterprise Objects
    >>>>> application is determining what
    >>>>> kind of concurrency you need. This requires knowing (or predicting) the
    >>>>> bottlenecks within your application.
    >>>>> Do bottlenecks occur at the database level when multiple users attempt
    >>>>> concurrent access to the data source
    >>>>> so that adding more database channels alleviates the bottleneck? Do
    >>>>> bottlenecks occur elsewhere in the
    >>>>> access layer so that providing a separate access layer for each user
    >>>>> alleviates the bottleneck?
    >>>>> The answers to these questions help determine the mechanism you need to
    >>>>> use
    >>>>> to instrument concurrency
    >>>>> within an Enterprise Objects application. Two common design patterns for
    >>>>> concurrency within Enterprise
    >>>>> Objects applications are to:
    >>>>> ¡½ Provide each user with an independent access layer stack.
    >>>>> ¡½ Provide multiple database channels on demand.
    >>>>>
    >>>>> The doc goes onto say that using multiple OSC introduces additional
    >>>>> challenges with concurrency between OSC¡¦s.
    >>>>>
    >>>>> The doc even gives code samples on providing multiple OSC and adding
    >>>>> additional database channels when a DatabaseChannelNeededNotification is
    >>>>> received
    >>>>>
    >>>>> I am interested to hear your analysis of the Wonder code. I will keep
    >>>>> experimenting
    >>>>>
    >>>>> Dov
    >>>>>
    >>>>> On 8/26/10 9:58 PM, "Mike Schrag" <mschrag@pobox.com> wrote:
    >>>>>
    >>>>>> I can't vouch for connectionbroker's accuracy of impl, but I would expect
    >>>>>> all
    >>>>>> your db access to be behind a single lock and therefore not benefit from
    >>>>>> multiple channels. Of course most of the people who might challenge my
    >>>>>> claims
    >>>>>> are probably drunk in Montreal right now. I'll check source later tonight
    >>>>>> and
    >>>>>> see if I'm talking crazy.
    >>>>>>
    >>>>>> Sent from my iPhone
    >>>>>>
    >>>>>> On Aug 26, 2010, at 9:55 PM, Dov Rosenberg <DRosenberg@Inquira.Com>
    >>>>>> wrote:
    >>>>>>
    >>>>>>> Is it not a good idea to use the ERXJDBCAdaptor and ConnectionBroker
    >>>>>>> together?
    >>>>>>>
    >>>>>>> I would have thought if I only used a single OSC it would have made sure
    >>>>>>> that each transaction went to the correct connection.
    >>>>>>>
    >>>>>>> Should there only be a single db connection for each OSC?
    >>>>>>>
    >>>>>>> Thanks
    >>>>>>>
    >>>>>>> Dov Rosenberg
    >>>>>>>
    >>>>>>>
    >>>>>>> On 8/26/10 9:46 PM, "Mike Schrag" <mschrag@pobox.com> wrote:
    >>>>>>>
    >>>>>>>> With one osc you saw activity on multiple connections concurrently? I
    >>>>>>>> can't
    >>>>>>>> imagine that scenario has a happy ending. I don't even know how I would
    >>>>>>>> be
    >>>>>>>> possible given that all your db access should be behind a dbc lock. You
    >>>>>>>> might
    >>>>>>>> see use of multiple connections, but that would probably explain your
    >>>>>>>> failing
    >>>>>>>> commits (inserting on conn 1, committing conn 2, maybe). Each osc
    >>>>>>>> should
    >>>>>>>> end
    >>>>>>>> up having its own conn and you should see parallel access that way.
    >>>>>>>>
    >>>>>>>> To answer more I think I need to not be on an iPhone :)
    >>>>>>>>
    >>>>>>>> Sent from my iPhone
    >>>>>>>>
    >>>>>>>> On Aug 26, 2010, at 9:37 PM, Dov Rosenberg <DRosenberg@Inquira.Com>
    >>>>>>>> wrote:
    >>>>>>>>
    >>>>>>>>> Hmmm - I did a little jmeter test and definitely saw an improvement in
    >>>>>>>>> page
    >>>>>>>>> view performance and saw activity on multiple DB connections during
    >>>>>>>>> the
    >>>>>>>>> test. By simply adding the connection broker with a single OSC.
    >>>>>>>>>
    >>>>>>>>> If I set up object store pooling wont that increase issues with
    >>>>>>>>> concurrency
    >>>>>>>>> within my app? I.e. Trying to update data that has already been
    >>>>>>>>> updated
    >>>>>>>>> in >>>> a
    >>>>>>>>> different OSC?
    >>>>>>>>>
    >>>>>>>>> I am using the Jgroups synchronizer between instances already.
    >>>>>>>>>
    >>>>>>>>> Would the following set of properties be consistent with each other:
    >>>>>>>>>
    >>>>>>>>> er.extensions.ERXObjectStoreCoordinatorPool.maxCoordinators = 10
    >>>>>>>>> er.extensions.ERXJDBCAdaptor.className=er.extensions.jdbc.ERXJDBCAdapt
    >>>>>>>>> or
    >>>>>>>>> er.extensions.ERXJDBCAdaptor.useConnectionBroker = true
    >>>>>>>>> er.extensions.remoteSynchronizer.enabled=true
    >>>>>>>>> er.extensions.remoteSynchronizer=er.jgroups.ERJGroupsSynchronizer
    >>>>>>>>> dbMinConnectionsGLOBAL=10
    >>>>>>>>> dbMaxConnectionsGLOBAL=15
    >>>>>>>>> er.extensions.ERXJDBCConnectionBroker.maxConnections=15
    >>>>>>>>> er.extensions.ERXJDBCConnectionBroker.minConnections=10
    >>>>>>>>>
    >>>>>>>>> If I understand what is going on I should get 10 EOF stacks sharing a
    >>>>>>>>> pool
    >>>>>>>>> of 15 database connections. I assume there is some magic running
    >>>>>>>>> behind
    >>>>>>>>> the
    >>>>>>>>> scenes to keep all of the OSC in sync with each other?
    >>>>>>>>>
    >>>>>>>>> Thanks again
    >>>>>>>>>
    >>>>>>>>> Dov Rosenberg
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>> On 8/26/10 9:00 PM, "Mike Schrag" <mschrag@pobox.com> wrote:
    >>>>>>>>>
    >>>>>>>>>> Connection pooling won't really do anything for you because each
    >>>>>>>>>> stack
    >>>>>>>>>> is
    >>>>>>>>>> single threaded. You want object store coordinator pooling.
    >>>>>>>>>>
    >>>>>>>>>> Sent from my iPhone
    >>>>>>>>>>
    >>>>>>>>>> On Aug 26, 2010, at 8:45 PM, Dov Rosenberg <DRosenberg@Inquira.Com>
    >>>>>>>>>> wrote:
    >>>>>>>>>>
    >>>>>>>>>>> During some testing today I turned on support for connection pooling
    >>>>>>>>>>> in
    >>>>>>>>>>> my
    >>>>>>>>>>> application using the ERXJDBCAdaptor and the
    >>>>>>>>>>> ERXJDBCConnectionBroker.
    >>>>>>>>>>> I
    >>>>>>>>>>> could
    >>>>>>>>>>> see the connections being used fine and all of the things that did
    >>>>>>>>>>> fetches
    >>>>>>>>>>> seemed to work without any issues. However when I tried doing
    >>>>>>>>>>> something
    >>>>>>>>>>> that
    >>>>>>>>>>> generated an INSERT the transactions did not commit and no changes
    >>>>>>>>>>> were
    >>>>>>>>>>> made.
    >>>>>>>>>>> I could see the SQL being generated as expected and the
    >>>>>>>>>>> saveChanges()
    >>>>>>>>>>> happened without throwing any exception ¡V just no data got written
    >>>>>>>>>>> to
    >>>>>>>>>>> the
    >>>>>>>>>>> database. As soon as I disabled the use of the connection pool
    >>>>>>>>>>> everything
    >>>>>>>>>>> worked properly again.
    >>>>>>>>>>>
    >>>>>>>>>>> Any thoughts would be appreciated
    >>>>>>>>>>>
    >>>>>>>>>>> Dov Rosenberg
    >>>>>>>>>>> _______________________________________________
    >>>>>>>>>>> Do not post admin requests to the list. They will be ignored.
    >>>>>>>>>>> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
    >>>>>>>>>>> Help/Unsubscribe/Update your Subscription:
    >>>>>>>>>>> http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40pobo
    >>>>>>>>>>> x.
    >>>>>>>>>>> com
    >>>>>>>>>>>
    >>>>>>>>>>> This email sent to mschrag@pobox.com
    >>>>>>>>>
    >>>>>>>
    >>>>> _______________________________________________
    >>>>> Do not post admin requests to the list. They will be ignored.
    >>>>> Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
    >>>>> Help/Unsubscribe/Update your Subscription:
    >>>>> http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-villa
    >>>>> ge
    >>>>> .net
    >>>>>
    >>>>> This email sent to chill@global-village.net
    >>>>
    >>>> --
    >>>> Chuck Hill Senior Consultant / VP Development
    >>>>
    >>>> Practical WebObjects - for developers who want to increase their overall
    >>>> knowledge of WebObjects or who are trying to solve specific problems.
    >>>> http://www.global-village.net/products/practical_webobjects
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>>
    >>>
    >>
    >
    >



    This archive was generated by hypermail 2.0.0 : Fri Aug 27 2010 - 17:13:59 UTC