[infinispan-dev] changes introduced by optimistic transactions

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] changes introduced by optimistic transactions

Mircea Markus
Hi,

These is a draft of the config changes I plan to add with ISPN-1131, can you please comment?

1.
<transaction lockingMode="optimistic"/>
The "lockingMode" attribute can have two values: optimistic(default) and pessimistic. If pessimistic is specified then this hase the same meaning as "useEagerLocking=true" (useEagerLocking to be deprecated).
The fluent API looks as follows:
        - config.fluent().transaction().lockingModeOptimistic();
        - config.fluent().transaction().lockingModePessimistic();
2.
An important restriction introduced by ISPN-1131 is the fact that a cache cannot be used in a mixed mode: i.e. all operations run within transactions or non does.
- by specifying a "transactionManagerLookup" the user marks the cache as transactional.
- there also is an "autoCommit" attribute: <transaction autoCommit="true"/>
        - if true (default), then user doesn't have to explicitly start and finish single-operation transactions, but a transaction is started under the hood on user's behalf
        - if false and a call is made out of a transaction's scope then an exception is thrown
        - if the cache is not transactional then autoCommit is ignored
- a cache that supports batch operations is implicitly transactional.  That's because batching is based on transactions.
and the fluent API:
config.fluent().transaction().autoCommit(true);

Cheers,
Mircea


   



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Sanne Grinovero-3
Hi Mircea,
what you propose has a strong impact on existing use cases.

Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.

Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?

If not, I'll have to start thinking about a different design.. not
happy about it since we got much testing from the community and I
would consider it quite stable now, but especially the performance
tests are not something I can easily reproduce on my own as it implies
big indexes, real clusters and real world use cases.

What we could do, is implement batching in the Lucene Directory itself
( a layer above Infinispan); that could work, but then again I don't
see why this could not be generalized and provide to everyone, keeping
the change less traumatic.. I'm likely not the only user of this API.

Sanne

2011/8/1 Mircea Markus <[hidden email]>:

> Hi,
>
> These is a draft of the config changes I plan to add with ISPN-1131, can you please comment?
>
> 1.
> <transaction lockingMode="optimistic"/>
> The "lockingMode" attribute can have two values: optimistic(default) and pessimistic. If pessimistic is specified then this hase the same meaning as "useEagerLocking=true" (useEagerLocking to be deprecated).
> The fluent API looks as follows:
>        - config.fluent().transaction().lockingModeOptimistic();
>        - config.fluent().transaction().lockingModePessimistic();
> 2.
> An important restriction introduced by ISPN-1131 is the fact that a cache cannot be used in a mixed mode: i.e. all operations run within transactions or non does.
> - by specifying a "transactionManagerLookup" the user marks the cache as transactional.
> - there also is an "autoCommit" attribute: <transaction autoCommit="true"/>
>        - if true (default), then user doesn't have to explicitly start and finish single-operation transactions, but a transaction is started under the hood on user's behalf
>        - if false and a call is made out of a transaction's scope then an exception is thrown
>        - if the cache is not transactional then autoCommit is ignored
> - a cache that supports batch operations is implicitly transactional.  That's because batching is based on transactions.
> and the fluent API:
> config.fluent().transaction().autoCommit(true);
>
> Cheers,
> Mircea
>
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Emmanuel Bernard
In reply to this post by Mircea Markus

On 1 août 2011, at 18:16, Mircea Markus wrote:

> Hi,
>
> These is a draft of the config changes I plan to add with ISPN-1131, can you please comment?
>
> 1.
> <transaction lockingMode="optimistic"/>
> The "lockingMode" attribute can have two values: optimistic(default) and pessimistic. If pessimistic is specified then this hase the same meaning as "useEagerLocking=true" (useEagerLocking to be deprecated).
> The fluent API looks as follows:
> - config.fluent().transaction().lockingModeOptimistic();
> - config.fluent().transaction().lockingModePessimistic();

config.fluent().transaction().lockingMode(OPTIMISTIC) //Enum are nice to see alternative options


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Sanne Grinovero-3
In reply to this post by Sanne Grinovero-3
2011/8/1 Sanne Grinovero <[hidden email]>:

> Hi Mircea,
> what you propose has a strong impact on existing use cases.
>
> Not having batching kills the Lucene performance, and using
> transactions is not an option unless the whole state of the index can
> fit in memory, which is not the use case we're targeting: I need to be
> able to use both on the same Cache.
>
> Isn't it possible to have a batching implementation which doesn't rely
> on transactions, or why are you needing to add this limitation?
>
> If not, I'll have to start thinking about a different design.. not
> happy about it since we got much testing from the community and I
> would consider it quite stable now, but especially the performance
> tests are not something I can easily reproduce on my own as it implies
> big indexes, real clusters and real world use cases.
>
> What we could do, is implement batching in the Lucene Directory itself
> ( a layer above Infinispan); that could work, but then again I don't
> see why this could not be generalized and provide to everyone, keeping
> the change less traumatic.. I'm likely not the only user of this API.

Sorry, not sure what I was thinking: I just realized that I can't
possibly implement batching on a different layer.

So I have no way to workaround this other than re-designing some core concepts:
please allow batching without transactions.

Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Mircea Markus
In reply to this post by Sanne Grinovero-3

On 1 Aug 2011, at 17:36, Sanne Grinovero wrote:

Hi Mircea,
what you propose has a strong impact on existing use cases.

Not having batching kills the Lucene performance, and using
transactions is not an option unless the whole state of the index can
fit in memory, which is not the use case we're targeting: I need to be
able to use both on the same Cache.
So you access the cache in two ways:
- through batching api and
- directly, outside the scope of a transaction
Am I correct?

Isn't it possible to have a batching implementation which doesn't rely
on transactions, or why are you needing to add this limitation?
Supporting a mixed way of accessing the cache might cause problems [1] (thanks to Paolo R. for this paper) and is also not consistent with the JSR 107's way of doing things, which doesn't go for  mixed cache.
I'd like to get a better grasp of your use case and let's catch up from there. IRC?


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Sanne Grinovero-3
2011/8/2 Mircea Markus <[hidden email]>:

>
> On 1 Aug 2011, at 17:36, Sanne Grinovero wrote:
>
> Hi Mircea,
> what you propose has a strong impact on existing use cases.
>
> Not having batching kills the Lucene performance, and using
> transactions is not an option unless the whole state of the index can
> fit in memory, which is not the use case we're targeting: I need to be
> able to use both on the same Cache.
>
> So you access the cache in two ways:
> - through batching api and
> - directly, outside the scope of a transaction
> Am I correct?
>
> Isn't it possible to have a batching implementation which doesn't rely
> on transactions, or why are you needing to add this limitation?
>
> Supporting a mixed way of accessing the cache might cause problems [1]
> (thanks to Paolo R. for this paper) and is also not consistent with the JSR
> 107's way of doing things, which doesn't go for  mixed cache.
> I'd like to get a better grasp of your use case and let's catch up from
> there. IRC?
> [1] http://www.cis.upenn.edu/acg/papers/cal06_atomic_semantics.pdf

I'm afraid we're mixing concerns, I didn't ask to allow both
transactional and non-transactional operations on the same cache, I
understand there are good reasons to not support that.

What I need is control for batching, i.e. be able to send a set of
writes on different keys at one specific point in time, combined with
the capability to avoid repeatable reads in the same cache: definitely
there are many ways to emulate a similar behaviour, I could collect
all writes in a map and then send async operations and block on them
all, or open transactions for the span of single operations when I
don't need to batch.
Consider as well that I might be pushing a thousand of packages of
10MB each, I'm not of the idea that keeping these in a distributed
transaction log is healthy for performance since I don't need rollback
capabilities, being this already handled by Lucene.

Definitely I have alternative options to explore, but it will take
more time and I don't think it would be reasonable to demand this for
5.1; also I think that batching should be supported by Infinispan
core. More importantly, we exposed batching before both
as a feature and as an API, I don't think we can take it away in a
minor release.

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Mircea Markus
In reply to this post by Emmanuel Bernard

On 1 Aug 2011, at 17:55, Emmanuel Bernard wrote:

>
> On 1 août 2011, at 18:16, Mircea Markus wrote:
>
>> Hi,
>>
>> These is a draft of the config changes I plan to add with ISPN-1131, can you please comment?
>>
>> 1.
>> <transaction lockingMode="optimistic"/>
>> The "lockingMode" attribute can have two values: optimistic(default) and pessimistic. If pessimistic is specified then this hase the same meaning as "useEagerLocking=true" (useEagerLocking to be deprecated).
>> The fluent API looks as follows:
>> - config.fluent().transaction().lockingModeOptimistic();
>> - config.fluent().transaction().lockingModePessimistic();
>
> config.fluent().transaction().lockingMode(OPTIMISTIC) //Enum are nice to see alternative options

Point taken - thanks!


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Mircea Markus
In reply to this post by Sanne Grinovero-3
I'm afraid we're mixing concerns, I didn't ask to allow both
transactional and non-transactional operations on the same cache, I
understand there are good reasons to not support that.

What I need is control for batching, i.e. be able to send a set of
writes on different keys at one specific point in time, combined with
the capability to avoid repeatable reads in the same cache: definitely
there are many ways to emulate a similar behaviour, I could collect
all writes in a map and then send async operations and block on them
all, or open transactions for the span of single operations when I
don't need to batch.
Consider as well that I might be pushing a thousand of packages of
10MB each, I'm not of the idea that keeping these in a distributed
transaction log is healthy for performance since I don't need rollback
capabilities, being this already handled by Lucene.
Right, after a long discussion and  brain storming here is the proposed solution:

Short version:  you'll have to set autoCommit=true and use1PcForInducedTransaction=true. Actually this will be the default value for the two attributes so you won't have to do any change to your existing code, nor configuration. The existing performance will be preserved, i.e. out-of-tx calls are handled within 1RPC.



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] changes introduced by optimistic transactions

Sanne Grinovero-3
2011/8/2 Mircea Markus <[hidden email]>:

> I'm afraid we're mixing concerns, I didn't ask to allow both
> transactional and non-transactional operations on the same cache, I
> understand there are good reasons to not support that.
>
> What I need is control for batching, i.e. be able to send a set of
> writes on different keys at one specific point in time, combined with
> the capability to avoid repeatable reads in the same cache: definitely
> there are many ways to emulate a similar behaviour, I could collect
> all writes in a map and then send async operations and block on them
> all, or open transactions for the span of single operations when I
> don't need to batch.
> Consider as well that I might be pushing a thousand of packages of
> 10MB each, I'm not of the idea that keeping these in a distributed
> transaction log is healthy for performance since I don't need rollback
> capabilities, being this already handled by Lucene.
>
> Right, after a long discussion and  brain storming here is the proposed
> solution:
> Short version:  you'll have to set autoCommit=true and
> use1PcForInducedTransaction=true. Actually this will be the default value
> for the two attributes so you won't have to do any change to your existing
> code, nor configuration. The existing performance will be preserved, i.e.
> out-of-tx calls are handled within 1RPC.
> Long version: https://issues.jboss.org/browse/ISPN-1297

Thanks! that sounds very good.
as always, we had very constructive discussions on irc.
(translation: was a though fight :D )

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev