[infinispan-dev] JoinTask.broadcastNewCh

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] JoinTask.broadcastNewCh

Mircea Markus
Hi,

This is something Bela has brought up while working on integrating RELAY in infinispan.
Right now it's the joiner (JoinTask) that broadcast the new CH to the cluster members. In the case of RELAY, this wouldn't be that good as the joiner might be on a remote site and this means a potentially costly RPC.
Isn't it possible for all the existing members to determine the new CH internally on the @viewChanged call and not wait for new joiner's broadcast? 
The change I've made in the code to do that[1] caused lots of intermittent failure, so before digging further - is there's something I miss which makes this approach not to work?

Cheers,
Mircea



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] JoinTask.broadcastNewCh

Bela Ban


On 2/10/11 1:07 PM, Mircea Markus wrote:
> Hi,
>
> This is something Bela has brought up while working on integrating RELAY in infinispan.
> Right now it's the joiner (JoinTask) that broadcast the new CH to the cluster members. In the case of RELAY, this wouldn't be that good as the joiner might be on a remote site and this means a potentially costly RPC.


On top of that, IMO it would be simpler for every member to compute its
new CH *itself*, rather than relying on a new joiner to do that.


> Isn't it possible for all the existing members to determine the new CH internally on the @viewChanged call and not wait for new joiner's broadcast?


+1


Let me give a bit of background. In RELAY I can have members A in one
local cluster and B in a different cluster. RELAY connects both to form
a virtual view {A,B}.

So, when A has been started and B starts up, what happens is that B will
get view {B} first, before getting {A,B}.

When B gets {B}, it becomes the coordinator and doesn't start a JoinTask
because it has the correct view. When B gets the new view {A,B}, it
handleView() is does *not* broadcast the CH either, because the view is
not a merge view. This results in the CH on B having a membership of
{A,B}, but A's CH only has a membership of {A}, thus A will *not*
distribute its data to B !

I think the simplest solution here would be to simply compute the CH on
a viewChange(); this way, we don't even have to have communication
across the cluster to determine the new CH.

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] JoinTask.broadcastNewCh

Mircea Markus

On 10 Feb 2011, at 13:12, Bela Ban wrote:

>
>
> On 2/10/11 1:07 PM, Mircea Markus wrote:
>> Hi,
>>
>> This is something Bela has brought up while working on integrating RELAY in infinispan.
>> Right now it's the joiner (JoinTask) that broadcast the new CH to the cluster members. In the case of RELAY, this wouldn't be that good as the joiner might be on a remote site and this means a potentially costly RPC.
>
>
> On top of that, IMO it would be simpler for every member to compute its
> new CH *itself*, rather than relying on a new joiner to do that.
>
>
>> Isn't it possible for all the existing members to determine the new CH internally on the @viewChanged call and not wait for new joiner's broadcast?
>
>
> +1
>
>
> Let me give a bit of background. In RELAY I can have members A in one
> local cluster and B in a different cluster. RELAY connects both to form
> a virtual view {A,B}.
>
> So, when A has been started and B starts up, what happens is that B will
> get view {B} first, before getting {A,B}.
>
> When B gets {B}, it becomes the coordinator and doesn't start a JoinTask
> because it has the correct view. When B gets the new view {A,B}, it
> handleView() is does *not* broadcast the CH either, because the view is
> not a merge view.

Thinking some more not sure that the "install ch on @viewChanged" solution is the right one for your problem.
What if there was actually a split brain between A and B? then you'd want a merge and not to join.  The starting node should be able to differentiate between a merge and a join.



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] JoinTask.broadcastNewCh

Bela Ban


On 2/10/11 2:57 PM, Mircea Markus wrote:

>
> On 10 Feb 2011, at 13:12, Bela Ban wrote:
>
>>
>>
>> On 2/10/11 1:07 PM, Mircea Markus wrote:
>>> Hi,
>>>
>>> This is something Bela has brought up while working on integrating RELAY in infinispan.
>>> Right now it's the joiner (JoinTask) that broadcast the new CH to the cluster members. In the case of RELAY, this wouldn't be that good as the joiner might be on a remote site and this means a potentially costly RPC.
>>
>>
>> On top of that, IMO it would be simpler for every member to compute its
>> new CH *itself*, rather than relying on a new joiner to do that.
>>
>>
>>> Isn't it possible for all the existing members to determine the new CH internally on the @viewChanged call and not wait for new joiner's broadcast?
>>
>>
>> +1
>>
>>
>> Let me give a bit of background. In RELAY I can have members A in one
>> local cluster and B in a different cluster. RELAY connects both to form
>> a virtual view {A,B}.
>>
>> So, when A has been started and B starts up, what happens is that B will
>> get view {B} first, before getting {A,B}.
>>
>> When B gets {B}, it becomes the coordinator and doesn't start a JoinTask
>> because it has the correct view. When B gets the new view {A,B}, it
>> handleView() is does *not* broadcast the CH either, because the view is
>> not a merge view.
>
> Thinking some more not sure that the "install ch on @viewChanged" solution is the right one for your problem.
> What if there was actually a split brain between A and B? then you'd want a merge and not to join.  The starting node should be able to differentiate between a merge and a join.

Yes, but this differentiation could be done in handleView(). The code
knows whether it's getting a View or a MergeView

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev