[infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Galder Zamarreño
Hi,

Re: https://issues.jboss.org/browse/ISPN-841

The issue here is the fact that if you call a TreeCache operation passing flags, you want this flags to apply to all cache operations encompassing the tree cache op. Now, the thing to remember about flags is that they get cleared after each cache invocation, so we must somehow pass flags around to all methods that operate on the cache as a result of a treecache.put for example.

A rudimentary way to do so would be to pass Flag... to all methods involved which is not pretty and hard to maintain. An alternative would be to have some flags thread local that gets populated on start of tree cache operation and gets cleared in the end of the operation. Although this might work, isn't this very similar to what CacheDelegate does to maintain flags except that instead of keeping them for a cache invocation, it would keep them hanging around until the end of the operation? TreeCache operations are bounded by start/stop atomic calls that are essentially calls to start/stop batches. So, it seems to me that what this is asking for is for a wider functionality to keep flags for the duration of a transaction/batch, which would most likely be solved better in core/

Thoughts?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Manik Surtani

On 23 Dec 2010, at 16:48, Galder Zamarreño wrote:

> Hi,
>
> Re: https://issues.jboss.org/browse/ISPN-841
>
> The issue here is the fact that if you call a TreeCache operation passing flags, you want this flags to apply to all cache operations encompassing the tree cache op. Now, the thing to remember about flags is that they get cleared after each cache invocation, so we must somehow pass flags around to all methods that operate on the cache as a result of a treecache.put for example.
>
> A rudimentary way to do so would be to pass Flag... to all methods involved which is not pretty and hard to maintain. An alternative would be to have some flags thread local that gets populated on start of tree cache operation and gets cleared in the end of the operation. Although this might work, isn't this very similar to what CacheDelegate does to maintain flags except that instead of keeping them for a cache invocation, it would keep them hanging around until the end of the operation? TreeCache operations are bounded by start/stop atomic calls that are essentially calls to start/stop batches. So, it seems to me that what this is asking for is for a wider functionality to keep flags for the duration of a transaction/batch, which would most likely be solved better in core/

Hmm; flags are explicitly designed for per-invocation application.  If we change this to apply to an entire transaction in core (e.g., by storing the flags in a transaction context rather than an invocation context) it will break a lot of existing code.

I think it cannot/should not be solved in core but rather in the TreeCache.  A thread local makes sense.

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Sanne Grinovero
We had a similar issue in the past:
Even when using Flag.SKIP_STORE consistently on a specific type of
object, which I did want to absolutely prevent to be written to the
store, this could end up in the store during rehash operations as the
flag wouldn't be in the context at that moment, and for safety the new
owner will write to the store.
This forced me to change the Lucene Directory to use multiple caches,
to be able to configure the specific one without any store.

Maybe we could attach some flags to Entry, possibly of different type
than current Flag. For example it would be nice to have a
"NEEDS_STORE" which could be set to false as soon as it's written, and
so also have rehashing perform better as there's no need to rewrite to
the store for most of the data being received, as it's likely already
stored.
(I understand this would be quite broken in a store-per-node
configuration, but we can elaborate on it, or have it optional and
enable only for shared stores)

Sanne

2011/1/4 Manik Surtani <[hidden email]>:

>
> On 23 Dec 2010, at 16:48, Galder Zamarreño wrote:
>
>> Hi,
>>
>> Re: https://issues.jboss.org/browse/ISPN-841
>>
>> The issue here is the fact that if you call a TreeCache operation passing flags, you want this flags to apply to all cache operations encompassing the tree cache op. Now, the thing to remember about flags is that they get cleared after each cache invocation, so we must somehow pass flags around to all methods that operate on the cache as a result of a treecache.put for example.
>>
>> A rudimentary way to do so would be to pass Flag... to all methods involved which is not pretty and hard to maintain. An alternative would be to have some flags thread local that gets populated on start of tree cache operation and gets cleared in the end of the operation. Although this might work, isn't this very similar to what CacheDelegate does to maintain flags except that instead of keeping them for a cache invocation, it would keep them hanging around until the end of the operation? TreeCache operations are bounded by start/stop atomic calls that are essentially calls to start/stop batches. So, it seems to me that what this is asking for is for a wider functionality to keep flags for the duration of a transaction/batch, which would most likely be solved better in core/
>
> Hmm; flags are explicitly designed for per-invocation application.  If we change this to apply to an entire transaction in core (e.g., by storing the flags in a transaction context rather than an invocation context) it will break a lot of existing code.
>
> I think it cannot/should not be solved in core but rather in the TreeCache.  A thread local makes sense.
>
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Manik Surtani

On 4 Jan 2011, at 17:20, Sanne Grinovero wrote:

> We had a similar issue in the past:
> Even when using Flag.SKIP_STORE consistently on a specific type of
> object, which I did want to absolutely prevent to be written to the
> store, this could end up in the store during rehash operations as the
> flag wouldn't be in the context at that moment, and for safety the new
> owner will write to the store.
> This forced me to change the Lucene Directory to use multiple caches,
> to be able to configure the specific one without any store.
>
> Maybe we could attach some flags to Entry, possibly of different type
> than current Flag. For example it would be nice to have a
> "NEEDS_STORE" which could be set to false as soon as it's written, and
> so also have rehashing perform better as there's no need to rewrite to
> the store for most of the data being received, as it's likely already
> stored.

Rehashing should bypass writing to a CacheStore anyway, if the store is shared.  If this is not the case, then that is a bug.  

> (I understand this would be quite broken in a store-per-node
> configuration, but we can elaborate on it, or have it optional and
> enable only for shared stores)
>
> Sanne
>
> 2011/1/4 Manik Surtani <[hidden email]>:
>>
>> On 23 Dec 2010, at 16:48, Galder Zamarreño wrote:
>>
>>> Hi,
>>>
>>> Re: https://issues.jboss.org/browse/ISPN-841
>>>
>>> The issue here is the fact that if you call a TreeCache operation passing flags, you want this flags to apply to all cache operations encompassing the tree cache op. Now, the thing to remember about flags is that they get cleared after each cache invocation, so we must somehow pass flags around to all methods that operate on the cache as a result of a treecache.put for example.
>>>
>>> A rudimentary way to do so would be to pass Flag... to all methods involved which is not pretty and hard to maintain. An alternative would be to have some flags thread local that gets populated on start of tree cache operation and gets cleared in the end of the operation. Although this might work, isn't this very similar to what CacheDelegate does to maintain flags except that instead of keeping them for a cache invocation, it would keep them hanging around until the end of the operation? TreeCache operations are bounded by start/stop atomic calls that are essentially calls to start/stop batches. So, it seems to me that what this is asking for is for a wider functionality to keep flags for the duration of a transaction/batch, which would most likely be solved better in core/
>>
>> Hmm; flags are explicitly designed for per-invocation application.  If we change this to apply to an entire transaction in core (e.g., by storing the flags in a transaction context rather than an invocation context) it will break a lot of existing code.
>>
>> I think it cannot/should not be solved in core but rather in the TreeCache.  A thread local makes sense.
>>
>> --
>> Manik Surtani
>> [hidden email]
>> twitter.com/maniksurtani
>>
>> Lead, Infinispan
>> http://www.infinispan.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Sanne Grinovero
2011/1/4 Manik Surtani <[hidden email]>:

>
> On 4 Jan 2011, at 17:20, Sanne Grinovero wrote:
>
>> We had a similar issue in the past:
>> Even when using Flag.SKIP_STORE consistently on a specific type of
>> object, which I did want to absolutely prevent to be written to the
>> store, this could end up in the store during rehash operations as the
>> flag wouldn't be in the context at that moment, and for safety the new
>> owner will write to the store.
>> This forced me to change the Lucene Directory to use multiple caches,
>> to be able to configure the specific one without any store.
>>
>> Maybe we could attach some flags to Entry, possibly of different type
>> than current Flag. For example it would be nice to have a
>> "NEEDS_STORE" which could be set to false as soon as it's written, and
>> so also have rehashing perform better as there's no need to rewrite to
>> the store for most of the data being received, as it's likely already
>> stored.
>
> Rehashing should bypass writing to a CacheStore anyway, if the store is shared.  If this is not the case, then that is a bug.
>

Good to know, opened ISPN-861 to track this.

Sanne

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] TreeCache needs Flag(s) to be maintained for the duration of a batch/tx

Galder Zamarreño
In reply to this post by Manik Surtani

On Jan 4, 2011, at 5:48 PM, Manik Surtani wrote:

>
> On 23 Dec 2010, at 16:48, Galder Zamarreño wrote:
>
>> Hi,
>>
>> Re: https://issues.jboss.org/browse/ISPN-841
>>
>> The issue here is the fact that if you call a TreeCache operation passing flags, you want this flags to apply to all cache operations encompassing the tree cache op. Now, the thing to remember about flags is that they get cleared after each cache invocation, so we must somehow pass flags around to all methods that operate on the cache as a result of a treecache.put for example.
>>
>> A rudimentary way to do so would be to pass Flag... to all methods involved which is not pretty and hard to maintain. An alternative would be to have some flags thread local that gets populated on start of tree cache operation and gets cleared in the end of the operation. Although this might work, isn't this very similar to what CacheDelegate does to maintain flags except that instead of keeping them for a cache invocation, it would keep them hanging around until the end of the operation? TreeCache operations are bounded by start/stop atomic calls that are essentially calls to start/stop batches. So, it seems to me that what this is asking for is for a wider functionality to keep flags for the duration of a transaction/batch, which would most likely be solved better in core/
>
> Hmm; flags are explicitly designed for per-invocation application.  If we change this to apply to an entire transaction in core (e.g., by storing the flags in a transaction context rather than an invocation context) it will break a lot of existing code.

I was not advocating for changing the meaning of flags, but rather adding to it. IOW, have the ability to set flags on a per invocation basis is definitely needed, but maybe core could also benefit from having the ability to set a flag for a tx, so have the two options available somehow. I'm just saying this out loud in case there's any other such use cases elsewhere apart from tree cache.

>
> I think it cannot/should not be solved in core but rather in the TreeCache.  A thread local makes sense.

Ok. If some other component needs a similar thing, we'd then think about it again.

>
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...