[infinispan-dev] https://issues.jboss.org/browse/ISPN-977

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Mircea Markus
Hi,

The basic problem behind this is that I need to be notified when a new consistent hash is installed. 

ATM there isn't any support (of which I know)  for  a "@ConsistenHashChangeListener".

I'm thinking to add such notifications either:
a) internally: Observer pattern on DistributionManager or even on DistributionManagerImpl
b) more generically, as a fully flagged listener.

I favor a) and then if more people ask for it we will expose it as a fully flagged listener.

Suggestions?

Cheers,
Mircea


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Sanne Grinovero
First thing I thought when reading your email was "OMG do we support
on-the-fly hash implementation changes? crazy!"

That's obviously not the case, but if you name it as
@ConsistenHashChangeListene that's what I would think.

Wouldn't it be better to change the exact timing of the viewchange
event? I don't see why Infinispan users might be more interested in
knowing about the topology details according to the transport than
what they are about the actual Infinispan hashing topology - I would
expect that when I receive which notification the new view is already
installed and ready to go; actually I thought that was the case since
ever.

What would be the use cases to get the notification *before* the new
hash is installed?

Cheers,
Sanne

2011/5/11 Mircea Markus <[hidden email]>:

> Hi,
> The basic problem behind this is that I need to be notified when a new
> consistent hash is installed.
>
> ATM there isn't any support (of which I know)  for  a
> "@ConsistenHashChangeListener".
>
> I'm thinking to add such notifications either:
> a) internally: Observer pattern on DistributionManager or even on
> DistributionManagerImpl
> b) more generically, as a fully flagged listener.
>
> I favor a) and then if more people ask for it we will expose it as a fully
> flagged listener.
>
> Suggestions?
>
> Cheers,
> Mircea
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Dan Berindei
With the new push-based rebalancing code, we install the new CH before
doing any rebalancing work, so the inconsistency window should be much
smaller.
In theory we also block new transactions, maybe you could use that to
stop generating new keys while rehashing.

On the other hand, in 5.0 you should be able to implement
KeyAffinityService on top of Pete's grouping work for
https://issues.jboss.org/browse/ISPN-312 and then you won't need any
listeners.

Dan


On Wed, May 11, 2011 at 3:29 PM, Sanne Grinovero
<[hidden email]> wrote:

> First thing I thought when reading your email was "OMG do we support
> on-the-fly hash implementation changes? crazy!"
>
> That's obviously not the case, but if you name it as
> @ConsistenHashChangeListene that's what I would think.
>
> Wouldn't it be better to change the exact timing of the viewchange
> event? I don't see why Infinispan users might be more interested in
> knowing about the topology details according to the transport than
> what they are about the actual Infinispan hashing topology - I would
> expect that when I receive which notification the new view is already
> installed and ready to go; actually I thought that was the case since
> ever.
>
> What would be the use cases to get the notification *before* the new
> hash is installed?
>
> Cheers,
> Sanne
>
> 2011/5/11 Mircea Markus <[hidden email]>:
>> Hi,
>> The basic problem behind this is that I need to be notified when a new
>> consistent hash is installed.
>>
>> ATM there isn't any support (of which I know)  for  a
>> "@ConsistenHashChangeListener".
>>
>> I'm thinking to add such notifications either:
>> a) internally: Observer pattern on DistributionManager or even on
>> DistributionManagerImpl
>> b) more generically, as a fully flagged listener.
>>
>> I favor a) and then if more people ask for it we will expose it as a fully
>> flagged listener.
>>
>> Suggestions?
>>
>> Cheers,
>> Mircea
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Mircea Markus
In reply to this post by Sanne Grinovero
On 11 May 2011, at 13:29, Sanne Grinovero wrote:
> First thing I thought when reading your email was "OMG do we support
> on-the-fly hash implementation changes? crazy!"
>
> That's obviously not the case, but if you name it as
> @ConsistenHashChangeListene that's what I would think.
good point :-)
ConsistentHashMembershipChange perhaps?

>
> Wouldn't it be better to change the exact timing of the viewchange
> event? I don't see why Infinispan users might be more interested in
> knowing about the topology details according to the transport than
> what they are about the actual Infinispan hashing topology - I would
> expect that when I receive which notification the new view is already
> installed and ready to go; actually I thought that was the case since
> ever.
>
> What would be the use cases to get the notification *before* the new
> hash is installed?
@ViewChanged  listener is a CacheManager listener, so it's independent of the caches in that are in use. It is possible for the CacheManager to wait for all its distributed caches to install the view first, and only then dispatch the @ViewChange event,
Not sure that's a good idea in the general case, though.

>
> Cheers,
> Sanne
>
> 2011/5/11 Mircea Markus <[hidden email]>:
>> Hi,
>> The basic problem behind this is that I need to be notified when a new
>> consistent hash is installed.
>>
>> ATM there isn't any support (of which I know)  for  a
>> "@ConsistenHashChangeListener".
>>
>> I'm thinking to add such notifications either:
>> a) internally: Observer pattern on DistributionManager or even on
>> DistributionManagerImpl
>> b) more generically, as a fully flagged listener.
>>
>> I favor a) and then if more people ask for it we will expose it as a fully
>> flagged listener.
>>
>> Suggestions?
>>
>> Cheers,
>> Mircea
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Mircea Markus
In reply to this post by Dan Berindei

On 11 May 2011, at 13:57, Dan Berindei wrote:

> With the new push-based rebalancing code, we install the new CH before
> doing any rebalancing work, so the inconsistency window should be much
> smaller.
> In theory we also block new transactions, maybe you could use that to
> stop generating new keys while rehashing.
that's sounds interesting, I'll take a look!
>
> On the other hand, in 5.0 you should be able to implement
> KeyAffinityService on top of Pete's grouping work for
> https://issues.jboss.org/browse/ISPN-312 and then you won't need any
> listeners.
Not sure how could that work? KAS generates pseudo-random keys that map to a certain nice. It's up to the user to place them in cache at some point, if ever.

>
> Dan
>
>
> On Wed, May 11, 2011 at 3:29 PM, Sanne Grinovero
> <[hidden email]> wrote:
>> First thing I thought when reading your email was "OMG do we support
>> on-the-fly hash implementation changes? crazy!"
>>
>> That's obviously not the case, but if you name it as
>> @ConsistenHashChangeListene that's what I would think.
>>
>> Wouldn't it be better to change the exact timing of the viewchange
>> event? I don't see why Infinispan users might be more interested in
>> knowing about the topology details according to the transport than
>> what they are about the actual Infinispan hashing topology - I would
>> expect that when I receive which notification the new view is already
>> installed and ready to go; actually I thought that was the case since
>> ever.
>>
>> What would be the use cases to get the notification *before* the new
>> hash is installed?
>>
>> Cheers,
>> Sanne
>>
>> 2011/5/11 Mircea Markus <[hidden email]>:
>>> Hi,
>>> The basic problem behind this is that I need to be notified when a new
>>> consistent hash is installed.
>>>
>>> ATM there isn't any support (of which I know)  for  a
>>> "@ConsistenHashChangeListener".
>>>
>>> I'm thinking to add such notifications either:
>>> a) internally: Observer pattern on DistributionManager or even on
>>> DistributionManagerImpl
>>> b) more generically, as a fully flagged listener.
>>>
>>> I favor a) and then if more people ask for it we will expose it as a fully
>>> flagged listener.
>>>
>>> Suggestions?
>>>
>>> Cheers,
>>> Mircea
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Sanne Grinovero
In reply to this post by Mircea Markus
2011/5/11 Mircea Markus <[hidden email]>:

> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>> First thing I thought when reading your email was "OMG do we support
>> on-the-fly hash implementation changes? crazy!"
>>
>> That's obviously not the case, but if you name it as
>> @ConsistenHashChangeListene that's what I would think.
> good point :-)
> ConsistentHashMembershipChange perhaps?
>>
>> Wouldn't it be better to change the exact timing of the viewchange
>> event? I don't see why Infinispan users might be more interested in
>> knowing about the topology details according to the transport than
>> what they are about the actual Infinispan hashing topology - I would
>> expect that when I receive which notification the new view is already
>> installed and ready to go; actually I thought that was the case since
>> ever.
>>
>> What would be the use cases to get the notification *before* the new
>> hash is installed?
> @ViewChanged  listener is a CacheManager listener, so it's independent of the caches in that are in use. It is possible for the CacheManager to wait for all its distributed caches to install the view first, and only then dispatch the @ViewChange event,
> Not sure that's a good idea in the general case, though.

let's talk about use cases. As an application developer building
something cool on top of Infinispan, what's the point in receiving an
event about new nodes being joined, if they're not ready to be used?
I'd rather have you notify me when new nodes joined and I can actually
start using them; the rest is low-level detail that an application
shouldn't worry about - I think - but I'm ready to take it back if you
find an example for which that's not true.

Sanne

>>
>> Cheers,
>> Sanne
>>
>> 2011/5/11 Mircea Markus <[hidden email]>:
>>> Hi,
>>> The basic problem behind this is that I need to be notified when a new
>>> consistent hash is installed.
>>>
>>> ATM there isn't any support (of which I know)  for  a
>>> "@ConsistenHashChangeListener".
>>>
>>> I'm thinking to add such notifications either:
>>> a) internally: Observer pattern on DistributionManager or even on
>>> DistributionManagerImpl
>>> b) more generically, as a fully flagged listener.
>>>
>>> I favor a) and then if more people ask for it we will expose it as a fully
>>> flagged listener.
>>>
>>> Suggestions?
>>>
>>> Cheers,
>>> Mircea
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Mircea Markus

On 11 May 2011, at 15:53, Sanne Grinovero wrote:

> 2011/5/11 Mircea Markus <[hidden email]>:
>> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>>> First thing I thought when reading your email was "OMG do we support
>>> on-the-fly hash implementation changes? crazy!"
>>>
>>> That's obviously not the case, but if you name it as
>>> @ConsistenHashChangeListene that's what I would think.
>> good point :-)
>> ConsistentHashMembershipChange perhaps?
>>>
>>> Wouldn't it be better to change the exact timing of the viewchange
>>> event? I don't see why Infinispan users might be more interested in
>>> knowing about the topology details according to the transport than
>>> what they are about the actual Infinispan hashing topology - I would
>>> expect that when I receive which notification the new view is already
>>> installed and ready to go; actually I thought that was the case since
>>> ever.
>>>
>>> What would be the use cases to get the notification *before* the new
>>> hash is installed?
>> @ViewChanged  listener is a CacheManager listener, so it's independent of the caches in that are in use. It is possible for the CacheManager to wait for all its distributed caches to install the view first, and only then dispatch the @ViewChange event,
>> Not sure that's a good idea in the general case, though.
>
> let's talk about use cases. As an application developer building
> something cool on top of Infinispan, what's the point in receiving an
> event about new nodes being joined, if they're not ready to be used?
> I'd rather have you notify me when new nodes joined and I can actually
> start using them; the rest is low-level detail that an application
> shouldn't worry about - I think - but I'm ready to take it back if you
> find an example for which that's not true.
TBH I can't find a convincing-enough use case to write down :-)
There is a small semantic (again theory!) difference between @ViewChanged and @NodeJoined (which is a really cool replacement-name for ConsistentHashMembershipChange btw).
ViewChange happens sooner than NodeJoined. This can be a matter of nanos or even seconds (timeout is by default 10 secs)  - depending on how long the state transfer takes.
I'd love to see other opinions around this as well.


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Dan Berindei
In reply to this post by Mircea Markus
On Wed, May 11, 2011 at 5:51 PM, Mircea Markus <[hidden email]> wrote:

>
> On 11 May 2011, at 13:57, Dan Berindei wrote:
>
>> With the new push-based rebalancing code, we install the new CH before
>> doing any rebalancing work, so the inconsistency window should be much
>> smaller.
>> In theory we also block new transactions, maybe you could use that to
>> stop generating new keys while rehashing.
> that's sounds interesting, I'll take a look!
>>
>> On the other hand, in 5.0 you should be able to implement
>> KeyAffinityService on top of Pete's grouping work for
>> https://issues.jboss.org/browse/ISPN-312 and then you won't need any
>> listeners.
> Not sure how could that work? KAS generates pseudo-random keys that map to a certain nice. It's up to the user to place them in cache at some point, if ever.

I was thinking of wrapping the keys generated by the user's
KeyGenerator together with a node address in our own MagicKey class
and set the @Group annotation on the address field.
We already have a MagicKey class for testing, the only problem with it
is that it only uses the address for hashing so you'd get lots of
collisions. With Pete's changes MagicKey becomes something we can
actually expose to the users (I was thinking through the
KeyAffinityService API).

I didn't think this through however, because MagicKey<K> would have to
extend K in order to satisfy the KeyAffinityService API (and even to
be able to store the key in the Cache<K,V> later).

Your question does raise an interesting point, though: a new node can
join between the user call to
KeyAffinityService.getKeyForAddress(address) and
Cache.put(generatedKey), making generatedKey map to the new node. So
there is no way for us to completely eliminate the issue of "stale"
keys.


>>
>> Dan
>>
>>
>> On Wed, May 11, 2011 at 3:29 PM, Sanne Grinovero
>> <[hidden email]> wrote:
>>> First thing I thought when reading your email was "OMG do we support
>>> on-the-fly hash implementation changes? crazy!"
>>>
>>> That's obviously not the case, but if you name it as
>>> @ConsistenHashChangeListene that's what I would think.
>>>
>>> Wouldn't it be better to change the exact timing of the viewchange
>>> event? I don't see why Infinispan users might be more interested in
>>> knowing about the topology details according to the transport than
>>> what they are about the actual Infinispan hashing topology - I would
>>> expect that when I receive which notification the new view is already
>>> installed and ready to go; actually I thought that was the case since
>>> ever.
>>>
>>> What would be the use cases to get the notification *before* the new
>>> hash is installed?
>>>
>>> Cheers,
>>> Sanne
>>>
>>> 2011/5/11 Mircea Markus <[hidden email]>:
>>>> Hi,
>>>> The basic problem behind this is that I need to be notified when a new
>>>> consistent hash is installed.
>>>>
>>>> ATM there isn't any support (of which I know)  for  a
>>>> "@ConsistenHashChangeListener".
>>>>
>>>> I'm thinking to add such notifications either:
>>>> a) internally: Observer pattern on DistributionManager or even on
>>>> DistributionManagerImpl
>>>> b) more generically, as a fully flagged listener.
>>>>
>>>> I favor a) and then if more people ask for it we will expose it as a fully
>>>> flagged listener.
>>>>
>>>> Suggestions?
>>>>
>>>> Cheers,
>>>> Mircea
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Erik Salter
Wouldn't any rehash affect the locality of these generated keys, or am I missing something?

Erik

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Dan Berindei
Sent: Wednesday, May 11, 2011 12:47 PM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

On Wed, May 11, 2011 at 5:51 PM, Mircea Markus <[hidden email]> wrote:

>
> On 11 May 2011, at 13:57, Dan Berindei wrote:
>
>> With the new push-based rebalancing code, we install the new CH
>> before doing any rebalancing work, so the inconsistency window should
>> be much smaller.
>> In theory we also block new transactions, maybe you could use that to
>> stop generating new keys while rehashing.
> that's sounds interesting, I'll take a look!
>>
>> On the other hand, in 5.0 you should be able to implement
>> KeyAffinityService on top of Pete's grouping work for
>> https://issues.jboss.org/browse/ISPN-312 and then you won't need any
>> listeners.
> Not sure how could that work? KAS generates pseudo-random keys that map to a certain nice. It's up to the user to place them in cache at some point, if ever.

I was thinking of wrapping the keys generated by the user's KeyGenerator together with a node address in our own MagicKey class and set the @Group annotation on the address field.
We already have a MagicKey class for testing, the only problem with it is that it only uses the address for hashing so you'd get lots of collisions. With Pete's changes MagicKey becomes something we can actually expose to the users (I was thinking through the KeyAffinityService API).

I didn't think this through however, because MagicKey<K> would have to extend K in order to satisfy the KeyAffinityService API (and even to be able to store the key in the Cache<K,V> later).

Your question does raise an interesting point, though: a new node can join between the user call to
KeyAffinityService.getKeyForAddress(address) and Cache.put(generatedKey), making generatedKey map to the new node. So there is no way for us to completely eliminate the issue of "stale"
keys.


>>
>> Dan
>>
>>
>> On Wed, May 11, 2011 at 3:29 PM, Sanne Grinovero
>> <[hidden email]> wrote:
>>> First thing I thought when reading your email was "OMG do we support
>>> on-the-fly hash implementation changes? crazy!"
>>>
>>> That's obviously not the case, but if you name it as
>>> @ConsistenHashChangeListene that's what I would think.
>>>
>>> Wouldn't it be better to change the exact timing of the viewchange
>>> event? I don't see why Infinispan users might be more interested in
>>> knowing about the topology details according to the transport than
>>> what they are about the actual Infinispan hashing topology - I would
>>> expect that when I receive which notification the new view is
>>> already installed and ready to go; actually I thought that was the
>>> case since ever.
>>>
>>> What would be the use cases to get the notification *before* the new
>>> hash is installed?
>>>
>>> Cheers,
>>> Sanne
>>>
>>> 2011/5/11 Mircea Markus <[hidden email]>:
>>>> Hi,
>>>> The basic problem behind this is that I need to be notified when a
>>>> new consistent hash is installed.
>>>>
>>>> ATM there isn't any support (of which I know)  for  a
>>>> "@ConsistenHashChangeListener".
>>>>
>>>> I'm thinking to add such notifications either:
>>>> a) internally: Observer pattern on DistributionManager or even on
>>>> DistributionManagerImpl
>>>> b) more generically, as a fully flagged listener.
>>>>
>>>> I favor a) and then if more people ask for it we will expose it as
>>>> a fully flagged listener.
>>>>
>>>> Suggestions?
>>>>
>>>> Cheers,
>>>> Mircea
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Dan Berindei
On Wed, May 11, 2011 at 8:47 PM, Erik Salter <[hidden email]> wrote:
> Wouldn't any rehash affect the locality of these generated keys, or am I missing something?
>

Indeed, the moment a new node joins the cluster you have no guarantees
that the keys that used to be on the same node are still stored
together.

Cheers
Dan


>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Dan Berindei
> Sent: Wednesday, May 11, 2011 12:47 PM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977
>
> On Wed, May 11, 2011 at 5:51 PM, Mircea Markus <[hidden email]> wrote:
>>
>> On 11 May 2011, at 13:57, Dan Berindei wrote:
>>
>>> With the new push-based rebalancing code, we install the new CH
>>> before doing any rebalancing work, so the inconsistency window should
>>> be much smaller.
>>> In theory we also block new transactions, maybe you could use that to
>>> stop generating new keys while rehashing.
>> that's sounds interesting, I'll take a look!
>>>
>>> On the other hand, in 5.0 you should be able to implement
>>> KeyAffinityService on top of Pete's grouping work for
>>> https://issues.jboss.org/browse/ISPN-312 and then you won't need any
>>> listeners.
>> Not sure how could that work? KAS generates pseudo-random keys that map to a certain nice. It's up to the user to place them in cache at some point, if ever.
>
> I was thinking of wrapping the keys generated by the user's KeyGenerator together with a node address in our own MagicKey class and set the @Group annotation on the address field.
> We already have a MagicKey class for testing, the only problem with it is that it only uses the address for hashing so you'd get lots of collisions. With Pete's changes MagicKey becomes something we can actually expose to the users (I was thinking through the KeyAffinityService API).
>
> I didn't think this through however, because MagicKey<K> would have to extend K in order to satisfy the KeyAffinityService API (and even to be able to store the key in the Cache<K,V> later).
>
> Your question does raise an interesting point, though: a new node can join between the user call to
> KeyAffinityService.getKeyForAddress(address) and Cache.put(generatedKey), making generatedKey map to the new node. So there is no way for us to completely eliminate the issue of "stale"
> keys.
>
>
>>>
>>> Dan
>>>
>>>
>>> On Wed, May 11, 2011 at 3:29 PM, Sanne Grinovero
>>> <[hidden email]> wrote:
>>>> First thing I thought when reading your email was "OMG do we support
>>>> on-the-fly hash implementation changes? crazy!"
>>>>
>>>> That's obviously not the case, but if you name it as
>>>> @ConsistenHashChangeListene that's what I would think.
>>>>
>>>> Wouldn't it be better to change the exact timing of the viewchange
>>>> event? I don't see why Infinispan users might be more interested in
>>>> knowing about the topology details according to the transport than
>>>> what they are about the actual Infinispan hashing topology - I would
>>>> expect that when I receive which notification the new view is
>>>> already installed and ready to go; actually I thought that was the
>>>> case since ever.
>>>>
>>>> What would be the use cases to get the notification *before* the new
>>>> hash is installed?
>>>>
>>>> Cheers,
>>>> Sanne
>>>>
>>>> 2011/5/11 Mircea Markus <[hidden email]>:
>>>>> Hi,
>>>>> The basic problem behind this is that I need to be notified when a
>>>>> new consistent hash is installed.
>>>>>
>>>>> ATM there isn't any support (of which I know)  for  a
>>>>> "@ConsistenHashChangeListener".
>>>>>
>>>>> I'm thinking to add such notifications either:
>>>>> a) internally: Observer pattern on DistributionManager or even on
>>>>> DistributionManagerImpl
>>>>> b) more generically, as a fully flagged listener.
>>>>>
>>>>> I favor a) and then if more people ask for it we will expose it as
>>>>> a fully flagged listener.
>>>>>
>>>>> Suggestions?
>>>>>
>>>>> Cheers,
>>>>> Mircea
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> [hidden email]
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] https://issues.jboss.org/browse/ISPN-977

Manik Surtani
In reply to this post by Sanne Grinovero

On 11 May 2011, at 13:29, Sanne Grinovero wrote:

> First thing I thought when reading your email was "OMG do we support
> on-the-fly hash implementation changes? crazy!"
>
> That's obviously not the case, but if you name it as
> @ConsistenHashChangeListene that's what I would think.
>
> Wouldn't it be better to change the exact timing of the viewchange
> event? I don't see why Infinispan users might be more interested in
> knowing about the topology details according to the transport than
> what they are about the actual Infinispan hashing topology - I would
> expect that when I receive which notification the new view is already
> installed and ready to go; actually I thought that was the case since
> ever.
>
> What would be the use cases to get the notification *before* the new
> hash is installed?

A lot of code makes use of view changes so as to update the consistent hash.  :-)

Also view changes indicate a lower-level change (it is propagated from JGroups) and any service that, for example, relies on a coordinator would need to know if this coordinator changes (such as SingletonCacheStore, etc).
--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977

Manik Surtani
In reply to this post by Erik Salter

On 11 May 2011, at 18:47, Erik Salter wrote:

> Wouldn't any rehash affect the locality of these generated keys, or am I missing something?

It would.  And hence ISPN-977, to address that.  Or is your concern keys already generated before the rehash?  The latter would certainly be a problem.  Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around?  For other apps, move the "session" to the appropriate node?

Cheers
Manik
--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] New events to reflect topology change Was: https://issues.jboss.org/browse/ISPN-977

Manik Surtani
In reply to this post by Sanne Grinovero
So this is related to ISPN-360 which is currently titled "create NodeJoinedEvent".

Leaving names out for now, essentially, what we need is a notification to inform listeners that (a) a JGroups ViewChange has been detected (b) a rehash has started and (c) a rehash has completed.  Possibly, being more fine-grained, that (d) a new ConsistentHash view has been installed.  Does this cover it all?  If so, how about:

@ViewChanged (same as we have now)
@RehashStarted
@RehashComplete
@TopologyChanged // to reflect a new CH?

Thoughts?

Cheers
Manik



On 11 May 2011, at 13:29, Sanne Grinovero wrote:

> First thing I thought when reading your email was "OMG do we support
> on-the-fly hash implementation changes? crazy!"
>
> That's obviously not the case, but if you name it as
> @ConsistenHashChangeListene that's what I would think.
>
> Wouldn't it be better to change the exact timing of the viewchange
> event? I don't see why Infinispan users might be more interested in
> knowing about the topology details according to the transport than
> what they are about the actual Infinispan hashing topology - I would
> expect that when I receive which notification the new view is already
> installed and ready to go; actually I thought that was the case since
> ever.
>
> What would be the use cases to get the notification *before* the new
> hash is installed?
>
> Cheers,
> Sanne
>
> 2011/5/11 Mircea Markus <[hidden email]>:
>> Hi,
>> The basic problem behind this is that I need to be notified when a new
>> consistent hash is installed.
>>
>> ATM there isn't any support (of which I know)  for  a
>> "@ConsistenHashChangeListener".
>>
>> I'm thinking to add such notifications either:
>> a) internally: Observer pattern on DistributionManager or even on
>> DistributionManagerImpl
>> b) more generically, as a fully flagged listener.
>>
>> I favor a) and then if more people ask for it we will expose it as a fully
>> flagged listener.
>>
>> Suggestions?
>>
>> Cheers,
>> Mircea
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] New events to reflect topology change Was: https://issues.jboss.org/browse/ISPN-977

Michal Linhard
On 05/13/2011 11:28 AM, Manik Surtani wrote:

> So this is related to ISPN-360 which is currently titled "create NodeJoinedEvent".
>
> Leaving names out for now, essentially, what we need is a notification to inform listeners that (a) a JGroups ViewChange has been detected (b) a rehash has started and (c) a rehash has completed.  Possibly, being more fine-grained, that (d) a new ConsistentHash view has been installed.  Does this cover it all?  If so, how about:
>
> @ViewChanged (same as we have now)
> @RehashStarted
> @RehashComplete
> @TopologyChanged // to reflect a new CH?
>
> Thoughts?
Covers my requirements. With RehashComplete we would be able build a
monitoring component that would enable us to wait until rehash is
complete during the cluster scale-in phase.

m.

> Cheers
> Manik
>
>
>
> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>
>> First thing I thought when reading your email was "OMG do we support
>> on-the-fly hash implementation changes? crazy!"
>>
>> That's obviously not the case, but if you name it as
>> @ConsistenHashChangeListene that's what I would think.
>>
>> Wouldn't it be better to change the exact timing of the viewchange
>> event? I don't see why Infinispan users might be more interested in
>> knowing about the topology details according to the transport than
>> what they are about the actual Infinispan hashing topology - I would
>> expect that when I receive which notification the new view is already
>> installed and ready to go; actually I thought that was the case since
>> ever.
>>
>> What would be the use cases to get the notification *before* the new
>> hash is installed?
>>
>> Cheers,
>> Sanne
>>
>> 2011/5/11 Mircea Markus<[hidden email]>:
>>> Hi,
>>> The basic problem behind this is that I need to be notified when a new
>>> consistent hash is installed.
>>>
>>> ATM there isn't any support (of which I know)  for  a
>>> "@ConsistenHashChangeListener".
>>>
>>> I'm thinking to add such notifications either:
>>> a) internally: Observer pattern on DistributionManager or even on
>>> DistributionManagerImpl
>>> b) more generically, as a fully flagged listener.
>>>
>>> I favor a) and then if more people ask for it we will expose it as a fully
>>> flagged listener.
>>>
>>> Suggestions?
>>>
>>> Cheers,
>>> Mircea
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Michal Linhard
Quality Assurance Engineer

Red Hat Czech s.r.o.
Purkynova 99 612 45 Brno, Czech Republic
phone: +420 532 294 320 ext. 62320
mobile: +420 728 626 363

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977

Erik Salter
In reply to this post by Manik Surtani
Yes, collocation of all keys is a large concern of my application(s).

Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code.   What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node.

Erik

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Manik Surtani
Sent: Friday, May 13, 2011 5:25 AM
To: infinispan -Dev List
Subject: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977


On 11 May 2011, at 18:47, Erik Salter wrote:

> Wouldn't any rehash affect the locality of these generated keys, or am I missing something?

It would.  And hence ISPN-977, to address that.  Or is your concern keys already generated before the rehash?  The latter would certainly be a problem.  Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around?  For other apps, move the "session" to the appropriate node?

Cheers
Manik
--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977

Dan Berindei
On Fri, May 13, 2011 at 6:38 PM, Erik Salter <[hidden email]> wrote:
> Yes, collocation of all keys is a large concern of my application(s).
>
> Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code.   What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node.
>

By playing around with the hash code do you mean you set the hashcode
for all the keys you want on a certain server to the same value? I
imagine that would degrade performance quite a bit, because we have
HashMaps everywhere and your keys will always end up in the same hash
bucket.

ISPN-312 seems to me like a much better fit for your use case than the
KeyAffinityService. Even if you added a listener to change your keys
when the topology changes, that would mean on a rehash the keys could
get moved to the new server and then back to the old server, whereas
with ISPN-312 they will either all stay on the old node or they will
all move to the new node.

Cheers
Dan


> Erik
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Manik Surtani
> Sent: Friday, May 13, 2011 5:25 AM
> To: infinispan -Dev List
> Subject: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977
>
>
> On 11 May 2011, at 18:47, Erik Salter wrote:
>
>> Wouldn't any rehash affect the locality of these generated keys, or am I missing something?
>
> It would.  And hence ISPN-977, to address that.  Or is your concern keys already generated before the rehash?  The latter would certainly be a problem.  Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around?  For other apps, move the "session" to the appropriate node?
>
> Cheers
> Manik
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977

Erik Salter
Hi Dan,

I don't necessarily care about which server it's on, as long as the keys for my set of caches all remain collocated.  I understand they will all end up in the same bucket, but for one hash code, that's at most 200 keys.  I must acquire a lock for a subset of them during a transaction -- so I make liberal use of the "eagerLockSingleNode" option and redirecting my calling application to execute a transaction on the local node.  Acquiring cluster-wide locks is an absolute throughput killer.

I took a look at the KeyAffinityService a while ago (when it came out) and quickly realized it would not be suitable for my purposes.  I was wondering if ISPN-977 would allow me to use it.  But you're right.  What I ultimately want is ISPN-312.

Erik

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Dan Berindei
Sent: Friday, May 13, 2011 12:58 PM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977

On Fri, May 13, 2011 at 6:38 PM, Erik Salter <[hidden email]> wrote:
> Yes, collocation of all keys is a large concern of my application(s).
>
> Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code.   What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node.
>

By playing around with the hash code do you mean you set the hashcode for all the keys you want on a certain server to the same value? I imagine that would degrade performance quite a bit, because we have HashMaps everywhere and your keys will always end up in the same hash bucket.


ISPN-312 seems to me like a much better fit for your use case than the KeyAffinityService. Even if you added a listener to change your keys when the topology changes, that would mean on a rehash the keys could get moved to the new server and then back to the old server, whereas with ISPN-312 they will either all stay on the old node or they will all move to the new node.

Cheers
Dan


> Erik
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Manik
> Surtani
> Sent: Friday, May 13, 2011 5:25 AM
> To: infinispan -Dev List
> Subject: [infinispan-dev] Generated keys affected by rehash Was:
> https://issues.jboss.org/browse/ISPN-977
>
>
> On 11 May 2011, at 18:47, Erik Salter wrote:
>
>> Wouldn't any rehash affect the locality of these generated keys, or am I missing something?
>
> It would.  And hence ISPN-977, to address that.  Or is your concern keys already generated before the rehash?  The latter would certainly be a problem.  Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around?  For other apps, move the "session" to the appropriate node?
>
> Cheers
> Manik
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] New events to reflect topology change Was: https://issues.jboss.org/browse/ISPN-977

Mircea Markus
In reply to this post by Manik Surtani

On 13 May 2011, at 10:28, Manik Surtani wrote:

> So this is related to ISPN-360 which is currently titled "create NodeJoinedEvent".
>
> Leaving names out for now, essentially, what we need is a notification to inform listeners that (a) a JGroups ViewChange has been detected (b) a rehash has started and (c) a rehash has completed.  Possibly, being more fine-grained, that (d) a new ConsistentHash view has been installed.  Does this cover it all?  If so, how about:
>
> @ViewChanged (same as we have now)
> @RehashStarted
> @RehashComplete
> @TopologyChanged // to reflect a new CH?
>
> Thoughts?
Sounds good to me.

>
> Cheers
> Manik
>
>
>
> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>
>> First thing I thought when reading your email was "OMG do we support
>> on-the-fly hash implementation changes? crazy!"
>>
>> That's obviously not the case, but if you name it as
>> @ConsistenHashChangeListene that's what I would think.
>>
>> Wouldn't it be better to change the exact timing of the viewchange
>> event? I don't see why Infinispan users might be more interested in
>> knowing about the topology details according to the transport than
>> what they are about the actual Infinispan hashing topology - I would
>> expect that when I receive which notification the new view is already
>> installed and ready to go; actually I thought that was the case since
>> ever.
>>
>> What would be the use cases to get the notification *before* the new
>> hash is installed?
>>
>> Cheers,
>> Sanne
>>
>> 2011/5/11 Mircea Markus <[hidden email]>:
>>> Hi,
>>> The basic problem behind this is that I need to be notified when a new
>>> consistent hash is installed.
>>>
>>> ATM there isn't any support (of which I know)  for  a
>>> "@ConsistenHashChangeListener".
>>>
>>> I'm thinking to add such notifications either:
>>> a) internally: Observer pattern on DistributionManager or even on
>>> DistributionManagerImpl
>>> b) more generically, as a fully flagged listener.
>>>
>>> I favor a) and then if more people ask for it we will expose it as a fully
>>> flagged listener.
>>>
>>> Suggestions?
>>>
>>> Cheers,
>>> Mircea
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] New events to reflect topology change Was: https://issues.jboss.org/browse/ISPN-977

Sanne Grinovero
Are @ViewChanged and @RehashStarted different?
If they are, are they different in any way useful to applications or
our own modules?

For consistency we could also consider the "Event.isPre() " to return
before/after reashing - tough I don't feel inclined to like this
proposal myself.

Sanne

2011/5/16 Mircea Markus <[hidden email]>:

>
> On 13 May 2011, at 10:28, Manik Surtani wrote:
>
>> So this is related to ISPN-360 which is currently titled "create NodeJoinedEvent".
>>
>> Leaving names out for now, essentially, what we need is a notification to inform listeners that (a) a JGroups ViewChange has been detected (b) a rehash has started and (c) a rehash has completed.  Possibly, being more fine-grained, that (d) a new ConsistentHash view has been installed.  Does this cover it all?  If so, how about:
>>
>> @ViewChanged (same as we have now)
>> @RehashStarted
>> @RehashComplete
>> @TopologyChanged // to reflect a new CH?
>>
>> Thoughts?
> Sounds good to me.
>>
>> Cheers
>> Manik
>>
>>
>>
>> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>>
>>> First thing I thought when reading your email was "OMG do we support
>>> on-the-fly hash implementation changes? crazy!"
>>>
>>> That's obviously not the case, but if you name it as
>>> @ConsistenHashChangeListene that's what I would think.
>>>
>>> Wouldn't it be better to change the exact timing of the viewchange
>>> event? I don't see why Infinispan users might be more interested in
>>> knowing about the topology details according to the transport than
>>> what they are about the actual Infinispan hashing topology - I would
>>> expect that when I receive which notification the new view is already
>>> installed and ready to go; actually I thought that was the case since
>>> ever.
>>>
>>> What would be the use cases to get the notification *before* the new
>>> hash is installed?
>>>
>>> Cheers,
>>> Sanne
>>>
>>> 2011/5/11 Mircea Markus <[hidden email]>:
>>>> Hi,
>>>> The basic problem behind this is that I need to be notified when a new
>>>> consistent hash is installed.
>>>>
>>>> ATM there isn't any support (of which I know)  for  a
>>>> "@ConsistenHashChangeListener".
>>>>
>>>> I'm thinking to add such notifications either:
>>>> a) internally: Observer pattern on DistributionManager or even on
>>>> DistributionManagerImpl
>>>> b) more generically, as a fully flagged listener.
>>>>
>>>> I favor a) and then if more people ask for it we will expose it as a fully
>>>> flagged listener.
>>>>
>>>> Suggestions?
>>>>
>>>> Cheers,
>>>> Mircea
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> [hidden email]
>> twitter.com/maniksurtani
>>
>> Lead, Infinispan
>> http://www.infinispan.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] New events to reflect topology change Was: https://issues.jboss.org/browse/ISPN-977

Manik Surtani-2
ViewChanged just means a change in membership has been detected. Not all nodes may be involved in a corresponding rehash.

I agree about using the isPre flag. It isn't the prettiest but at least it's consistent.

Anyway I have pushed a patch for this, please have a look and comment.

Sent from my mobile phone

On 16 May 2011, at 18:33, Sanne Grinovero <[hidden email]> wrote:

> Are @ViewChanged and @RehashStarted different?
> If they are, are they different in any way useful to applications or
> our own modules?
>
> For consistency we could also consider the "Event.isPre() " to return
> before/after reashing - tough I don't feel inclined to like this
> proposal myself.
>
> Sanne
>
> 2011/5/16 Mircea Markus <[hidden email]>:
>>
>> On 13 May 2011, at 10:28, Manik Surtani wrote:
>>
>>> So this is related to ISPN-360 which is currently titled "create NodeJoinedEvent".
>>>
>>> Leaving names out for now, essentially, what we need is a notification to inform listeners that (a) a JGroups ViewChange has been detected (b) a rehash has started and (c) a rehash has completed.  Possibly, being more fine-grained, that (d) a new ConsistentHash view has been installed.  Does this cover it all?  If so, how about:
>>>
>>> @ViewChanged (same as we have now)
>>> @RehashStarted
>>> @RehashComplete
>>> @TopologyChanged // to reflect a new CH?
>>>
>>> Thoughts?
>> Sounds good to me.
>>>
>>> Cheers
>>> Manik
>>>
>>>
>>>
>>> On 11 May 2011, at 13:29, Sanne Grinovero wrote:
>>>
>>>> First thing I thought when reading your email was "OMG do we support
>>>> on-the-fly hash implementation changes? crazy!"
>>>>
>>>> That's obviously not the case, but if you name it as
>>>> @ConsistenHashChangeListene that's what I would think.
>>>>
>>>> Wouldn't it be better to change the exact timing of the viewchange
>>>> event? I don't see why Infinispan users might be more interested in
>>>> knowing about the topology details according to the transport than
>>>> what they are about the actual Infinispan hashing topology - I would
>>>> expect that when I receive which notification the new view is already
>>>> installed and ready to go; actually I thought that was the case since
>>>> ever.
>>>>
>>>> What would be the use cases to get the notification *before* the new
>>>> hash is installed?
>>>>
>>>> Cheers,
>>>> Sanne
>>>>
>>>> 2011/5/11 Mircea Markus <[hidden email]>:
>>>>> Hi,
>>>>> The basic problem behind this is that I need to be notified when a new
>>>>> consistent hash is installed.
>>>>>
>>>>> ATM there isn't any support (of which I know)  for  a
>>>>> "@ConsistenHashChangeListener".
>>>>>
>>>>> I'm thinking to add such notifications either:
>>>>> a) internally: Observer pattern on DistributionManager or even on
>>>>> DistributionManagerImpl
>>>>> b) more generically, as a fully flagged listener.
>>>>>
>>>>> I favor a) and then if more people ask for it we will expose it as a fully
>>>>> flagged listener.
>>>>>
>>>>> Suggestions?
>>>>>
>>>>> Cheers,
>>>>> Mircea
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> [hidden email]
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Manik Surtani
>>> [hidden email]
>>> twitter.com/maniksurtani
>>>
>>> Lead, Infinispan
>>> http://www.infinispan.org
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12