[infinispan-dev] Conflict Manager and Partition Handling Blog

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Conflict Manager and Partition Handling Blog

Ryan Emerson-2
Hi Everyone,

Here's a blog post on the introduction of ConflictManager and the recent changes to partition handling.

http://blog.infinispan.org/2017/07/conflict-management-and-partition.html

Cheers
Ryan
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Galder Zamarreño
Hey Ryan,

Very detailed blog post! Great work on both the post and the feature! :D

While reading, the following question came to my mind: how does Infinispan determine there's a conflict? Does it rely on .equals() based equality?

A follow up would be: whether in the future this could be pluggable, e.g. when comparing a version field is enough to realise there's a conflict. As opposed of relying in .equals(), if that's what's being used inside :)

Cheers,
--
Galder Zamarreño
Infinispan, Red Hat

> On 17 Jul 2017, at 14:16, Ryan Emerson <[hidden email]> wrote:
>
> Hi Everyone,
>
> Here's a blog post on the introduction of ConflictManager and the recent changes to partition handling.
>
> http://blog.infinispan.org/2017/07/conflict-management-and-partition.html
>
> Cheers
> Ryan
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Galder Zamarreño
One more thing: have you thought how we could have a simple tutorial on this feature?

It'd be great to find a simple, reduced, example to show it off :)

Cheers,
--
Galder Zamarreño
Infinispan, Red Hat

> On 25 Jul 2017, at 16:54, Galder Zamarreño <[hidden email]> wrote:
>
> Hey Ryan,
>
> Very detailed blog post! Great work on both the post and the feature! :D
>
> While reading, the following question came to my mind: how does Infinispan determine there's a conflict? Does it rely on .equals() based equality?
>
> A follow up would be: whether in the future this could be pluggable, e.g. when comparing a version field is enough to realise there's a conflict. As opposed of relying in .equals(), if that's what's being used inside :)
>
> Cheers,
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 17 Jul 2017, at 14:16, Ryan Emerson <[hidden email]> wrote:
>>
>> Hi Everyone,
>>
>> Here's a blog post on the introduction of ConflictManager and the recent changes to partition handling.
>>
>> http://blog.infinispan.org/2017/07/conflict-management-and-partition.html
>>
>> Cheers
>> Ryan
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Galder Zamarreño
Oh, if we can't find a simple tutorial for it, there's always https://github.com/infinispan-demos :)

--
Galder Zamarreño
Infinispan, Red Hat

> On 25 Jul 2017, at 17:11, Galder Zamarreño <[hidden email]> wrote:
>
> One more thing: have you thought how we could have a simple tutorial on this feature?
>
> It'd be great to find a simple, reduced, example to show it off :)
>
> Cheers,
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 25 Jul 2017, at 16:54, Galder Zamarreño <[hidden email]> wrote:
>>
>> Hey Ryan,
>>
>> Very detailed blog post! Great work on both the post and the feature! :D
>>
>> While reading, the following question came to my mind: how does Infinispan determine there's a conflict? Does it rely on .equals() based equality?
>>
>> A follow up would be: whether in the future this could be pluggable, e.g. when comparing a version field is enough to realise there's a conflict. As opposed of relying in .equals(), if that's what's being used inside :)
>>
>> Cheers,
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>> On 17 Jul 2017, at 14:16, Ryan Emerson <[hidden email]> wrote:
>>>
>>> Hi Everyone,
>>>
>>> Here's a blog post on the introduction of ConflictManager and the recent changes to partition handling.
>>>
>>> http://blog.infinispan.org/2017/07/conflict-management-and-partition.html
>>>
>>> Cheers
>>> Ryan
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Ryan Emerson-2
Hi Galder,

Thanks for the feedback.

Conflicts are detected by applying a predicate to the returned Map<Address, CacheEntry> for each cache entry. Currently this predicate utilises Stream::distinct (so .equals), to check for multiple versions of an entry. So implementing pluggable strategies for defining a conflict should be trivial :)

Good idea about a simple tutorial/demo, I'll look into it when I get a chance.

Cheers
Ryan

----- Original Message -----

> Oh, if we can't find a simple tutorial for it, there's always
> https://github.com/infinispan-demos :)
>
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
> > On 25 Jul 2017, at 17:11, Galder Zamarreño <[hidden email]> wrote:
> >
> > One more thing: have you thought how we could have a simple tutorial on
> > this feature?
> >
> > It'd be great to find a simple, reduced, example to show it off :)
> >
> > Cheers,
> > --
> > Galder Zamarreño
> > Infinispan, Red Hat
> >
> >> On 25 Jul 2017, at 16:54, Galder Zamarreño <[hidden email]> wrote:
> >>
> >> Hey Ryan,
> >>
> >> Very detailed blog post! Great work on both the post and the feature! :D
> >>
> >> While reading, the following question came to my mind: how does Infinispan
> >> determine there's a conflict? Does it rely on .equals() based equality?
> >>
> >> A follow up would be: whether in the future this could be pluggable, e.g.
> >> when comparing a version field is enough to realise there's a conflict.
> >> As opposed of relying in .equals(), if that's what's being used inside :)
> >>
> >> Cheers,
> >> --
> >> Galder Zamarreño
> >> Infinispan, Red Hat
> >>
> >>> On 17 Jul 2017, at 14:16, Ryan Emerson <[hidden email]> wrote:
> >>>
> >>> Hi Everyone,
> >>>
> >>> Here's a blog post on the introduction of ConflictManager and the recent
> >>> changes to partition handling.
> >>>
> >>> http://blog.infinispan.org/2017/07/conflict-management-and-partition.html
> >>>
> >>> Cheers
> >>> Ryan
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> [hidden email]
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Sanne Grinovero-3
Great job! I love to see this improved and being described in detail.

+1 to add some practical examples, as I'm afraid we only notice
limitations in features like this when thinking about specific
use-cases.

The option `REMOVE_ALL` seems sensible for the disposable Cache use
case. One question though: if one partition has a defined value for a
key, while the other partition has no value (null) for this same key,
is it considered a conflict?
I think you need to clarify if a "null" in a subset of partitions
causes the conflict merge to be triggered or not. I think it should:
for example having the cache use case in mind, an explicit
invalidation needs to be propagated safely.

Thanks,
Sanne


On 26 July 2017 at 13:41, Ryan Emerson <[hidden email]> wrote:

> Hi Galder,
>
> Thanks for the feedback.
>
> Conflicts are detected by applying a predicate to the returned Map<Address, CacheEntry> for each cache entry. Currently this predicate utilises Stream::distinct (so .equals), to check for multiple versions of an entry. So implementing pluggable strategies for defining a conflict should be trivial :)
>
> Good idea about a simple tutorial/demo, I'll look into it when I get a chance.
>
> Cheers
> Ryan
>
> ----- Original Message -----
>> Oh, if we can't find a simple tutorial for it, there's always
>> https://github.com/infinispan-demos :)
>>
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>> > On 25 Jul 2017, at 17:11, Galder Zamarreño <[hidden email]> wrote:
>> >
>> > One more thing: have you thought how we could have a simple tutorial on
>> > this feature?
>> >
>> > It'd be great to find a simple, reduced, example to show it off :)
>> >
>> > Cheers,
>> > --
>> > Galder Zamarreño
>> > Infinispan, Red Hat
>> >
>> >> On 25 Jul 2017, at 16:54, Galder Zamarreño <[hidden email]> wrote:
>> >>
>> >> Hey Ryan,
>> >>
>> >> Very detailed blog post! Great work on both the post and the feature! :D
>> >>
>> >> While reading, the following question came to my mind: how does Infinispan
>> >> determine there's a conflict? Does it rely on .equals() based equality?
>> >>
>> >> A follow up would be: whether in the future this could be pluggable, e.g.
>> >> when comparing a version field is enough to realise there's a conflict.
>> >> As opposed of relying in .equals(), if that's what's being used inside :)
>> >>
>> >> Cheers,
>> >> --
>> >> Galder Zamarreño
>> >> Infinispan, Red Hat
>> >>
>> >>> On 17 Jul 2017, at 14:16, Ryan Emerson <[hidden email]> wrote:
>> >>>
>> >>> Hi Everyone,
>> >>>
>> >>> Here's a blog post on the introduction of ConflictManager and the recent
>> >>> changes to partition handling.
>> >>>
>> >>> http://blog.infinispan.org/2017/07/conflict-management-and-partition.html
>> >>>
>> >>> Cheers
>> >>> Ryan
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> [hidden email]
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> >
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Ryan Emerson-2
> The option `REMOVE_ALL` seems sensible for the disposable Cache use
> case. One question though: if one partition has a defined value for a
> key, while the other partition has no value (null) for this same key,
> is it considered a conflict?
> I think you need to clarify if a "null" in a subset of partitions
> causes the conflict merge to be triggered or not. I think it should:
> for example having the cache use case in mind, an explicit
> invalidation needs to be propagated safely.

Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Cheers
Ryan
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Sanne Grinovero-3
On 1 August 2017 at 12:19, Ryan Emerson <[hidden email]> wrote:

>> The option `REMOVE_ALL` seems sensible for the disposable Cache use
>> case. One question though: if one partition has a defined value for a
>> key, while the other partition has no value (null) for this same key,
>> is it considered a conflict?
>> I think you need to clarify if a "null" in a subset of partitions
>> causes the conflict merge to be triggered or not. I think it should:
>> for example having the cache use case in mind, an explicit
>> invalidation needs to be propagated safely.
>
> Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Thanks, looks great. Would you mind clarifying the docs about this?

Sanne

>
> Cheers
> Ryan
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Wayne Wang
In reply to this post by Ryan Emerson-2
Hi Ryan,

I understand that infinispan can be used by many platforms/applications. In addition, I was not able to find more information as to how infinispan cache manager actually work.

What I described is the infinispan embedded in wildfly 10.1.10.Final. This observation happened in a cluster of two wildfly 10.1.0.Final instances

Basically, the console output of the wildfly instance that actually made the modification of an object printed out update statement (expected). Later on, if a user revisit the same object (hotel), the console will not print out any query statement. This is expected since the latest data was cached.

For the other wildfly instance(cluster member), if a user visit the same object (hotel) first time, a query to select for a specific object(hotel) was printed out. This is expected since the cache should be invalidated and query needs to be executed to retrieve latest data.  However, when the user repeatedly revisit the same object (hotel), the query got printed out again. This will happen until a period of time (may be expiration setting).

This observation indicated that the wildfly instance which actually made the modification was able retrieve the latest data and cache it again. But the wildfly instance that receives the signal (to invalidate cache) indeed invalidated the cache, however, it did just that without cache the latest data.

Is this behavior the intended design or some other configuration I need to make to ensure the other cluster members can also cache the latest data?

Thanks,

Wayne


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Ryan Emerson
Sent: Tuesday, August 1, 2017 5:20 AM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

> The option `REMOVE_ALL` seems sensible for the disposable Cache use
> case. One question though: if one partition has a defined value for a
> key, while the other partition has no value (null) for this same key,
> is it considered a conflict?
> I think you need to clarify if a "null" in a subset of partitions
> causes the conflict merge to be triggered or not. I think it should:
> for example having the cache use case in mind, an explicit
> invalidation needs to be propagated safely.

Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Cheers
Ryan
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Ryan Emerson-2
In reply to this post by Sanne Grinovero-3
No problem, my intention is to issue a PR later this week.

----- Original Message -----

> On 1 August 2017 at 12:19, Ryan Emerson <[hidden email]> wrote:
> >> The option `REMOVE_ALL` seems sensible for the disposable Cache use
> >> case. One question though: if one partition has a defined value for a
> >> key, while the other partition has no value (null) for this same key,
> >> is it considered a conflict?
> >> I think you need to clarify if a "null" in a subset of partitions
> >> causes the conflict merge to be triggered or not. I think it should:
> >> for example having the cache use case in mind, an explicit
> >> invalidation needs to be propagated safely.
> >
> > Yes a combination of null/non-null entries is detected as a conflict. So in
> > the use-case you describe, utilising the REMOVE_ALL strategy would result
> > in the cache entry being removed from the cache on merge.
>
> Thanks, looks great. Would you mind clarifying the docs about this?
>
> Sanne
>
> >
> > Cheers
> > Ryan
> > _______________________________________________
> > infinispan-dev mailing list
> > [hidden email]
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

Ryan Emerson-2
In reply to this post by Wayne Wang
Hi Wayne,

I see that you have raised your query in the Infinispan forum [1], which is the correct place for these kinds of queries, not this mailing list. This mailing list is for technical discussions regarding the development of Infinispan. Furthermore, in the future please try to avoid hijacking existing threads on mailing lists if they are not directly related to your issue/query.

[1] https://developer.jboss.org/thread/275678

Thanks
Ryan

----- Original Message -----

> Hi Ryan,
>
> I understand that infinispan can be used by many platforms/applications. In
> addition, I was not able to find more information as to how infinispan cache
> manager actually work.
>
> What I described is the infinispan embedded in wildfly 10.1.10.Final. This
> observation happened in a cluster of two wildfly 10.1.0.Final instances
>
> Basically, the console output of the wildfly instance that actually made the
> modification of an object printed out update statement (expected). Later on,
> if a user revisit the same object (hotel), the console will not print out
> any query statement. This is expected since the latest data was cached.
>
> For the other wildfly instance(cluster member), if a user visit the same
> object (hotel) first time, a query to select for a specific object(hotel)
> was printed out. This is expected since the cache should be invalidated and
> query needs to be executed to retrieve latest data.  However, when the user
> repeatedly revisit the same object (hotel), the query got printed out again.
> This will happen until a period of time (may be expiration setting).
>
> This observation indicated that the wildfly instance which actually made the
> modification was able retrieve the latest data and cache it again. But the
> wildfly instance that receives the signal (to invalidate cache) indeed
> invalidated the cache, however, it did just that without cache the latest
> data.
>
> Is this behavior the intended design or some other configuration I need to
> make to ensure the other cluster members can also cache the latest data?
>
> Thanks,
>
> Wayne
>
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Ryan Emerson
> Sent: Tuesday, August 1, 2017 5:20 AM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Conflict Manager and Partition Handling Blog
>
> > The option `REMOVE_ALL` seems sensible for the disposable Cache use
> > case. One question though: if one partition has a defined value for a
> > key, while the other partition has no value (null) for this same key,
> > is it considered a conflict?
> > I think you need to clarify if a "null" in a subset of partitions
> > causes the conflict merge to be triggered or not. I think it should:
> > for example having the cache use case in mind, an explicit
> > invalidation needs to be propagated safely.
>
> Yes a combination of null/non-null entries is detected as a conflict. So in
> the use-case you describe, utilising the REMOVE_ALL strategy would result in
> the cache entry being removed from the cache on merge.
>
> Cheers
> Ryan
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...