[infinispan-dev] ISPN-863 proposal

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] ISPN-863 proposal

Vladimir Blagojevic

Dave,

I applaud you for this effort and your implementation and wiki are indeed admirable as you have followed the goals outlined in this JIRA. However, now that I have had an opportunity to look at the final solution I have one concern. We went through a great lengths to design a container that has a low lock contention [1] while maintaining high precision of a sophisticated eviction algorithm. Goals outlined in this JIRA have lead you in a direction to redesign the original proposal. I was wondering if it is possible to use your MemoryMonitor to make a decision whether or not to grow BCHM rather than to check if eviction should be done upon each put invocation? Currently we preset the size of BCHM and we never grow it (i.e. Segment#rehash is never invoked). Maybe this is a place where MemoryMonitor comes as a natural fit. We can keep the current design as outlined in [1] while reaping the benefits of a memory bound resizeable container.


WDYT?


Regards,
Vladimir



[1] http://infinispan.blogspot.com/2010/03/infinispan-eviction-batching-updates.html


Dave said:

It took me some time to figure out how to use Git and add some more tests. I have pushed my changes to a topic branch located at https://github.com/dlmarion/infinispan/tree/ISPN-863-master.

Please let me know if you have any questions or concerns. I have put up some documentation at https://github.com/dlmarion/infinispan/wiki/ISPN-863-Implementation

 

 

-- Dave Marion
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 proposal

david marion
 
  As a user I'm not tied to any specific implementation (I just want the feature), but I 'll have to look at the code to fully understand your suggestion. I don't have the code in front of me at the moment (I'm at work). However, I don't believe I lock anything specifically. It's very possible that I am doing something that is causing lock contention, can you point me to where this is occurring? I am accessing the same per segment lock free queue that the article mentions.
 
-- Dave Marion
 

Date: Mon, 31 Jan 2011 11:57:53 -0300
From: [hidden email]
To: [hidden email]
Subject: [infinispan-dev] ISPN-863 proposal

Dave,

I applaud you for this effort and your implementation and wiki are indeed admirable as you have followed the goals outlined in this JIRA. However, now that I have had an opportunity to look at the final solution I have one concern. We went through a great lengths to design a container that has a low lock contention [1] while maintaining high precision of a sophisticated eviction algorithm. Goals outlined in this JIRA have lead you in a direction to redesign the original proposal. I was wondering if it is possible to use your MemoryMonitor to make a decision whether or not to grow BCHM rather than to check if eviction should be done upon each put invocation? Currently we preset the size of BCHM and we never grow it (i.e. Segment#rehash is never invoked). Maybe this is a place where MemoryMonitor comes as a natural fit. We can keep the current design as outlined in [1] while reaping the benefits of a memory bound resizeable container.


WDYT?


Regards,
Vladimir



[1] http://infinispan.blogspot.com/2010/03/infinispan-eviction-batching-updates.html


Dave said:

It took me some time to figure out how to use Git and add some more tests. I have pushed my changes to a topic branch located at https://github.com/dlmarion/infinispan/tree/ISPN-863-master.

Please let me know if you have any questions or concerns. I have put up some documentation at https://github.com/dlmarion/infinispan/wiki/ISPN-863-Implementation

 

 

-- Dave Marion
_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 proposal

Vladimir Blagojevic
Yes you are accessing the lock-free queue but you are also locking a segment repeatedly to remove each entry selected for eviction (MemoryGuard#evict). While this might not be the end of the world it will cause lock contention.

Regards,
Vladimir
On 11-01-31 12:37 PM, david marion wrote:
 
  As a user I'm not tied to any specific implementation (I just want the feature), but I 'll have to look at the code to fully understand your suggestion. I don't have the code in front of me at the moment (I'm at work). However, I don't believe I lock anything specifically. It's very possible that I am doing something that is causing lock contention, can you point me to where this is occurring? I am accessing the same per segment lock free queue that the article mentions.
 
-- Dave Marion
 

Date: Mon, 31 Jan 2011 11:57:53 -0300
From: [hidden email]
To: [hidden email]
Subject: [infinispan-dev] ISPN-863 proposal

Dave,

I applaud you for this effort and your implementation and wiki are indeed admirable as you have followed the goals outlined in this JIRA. However, now that I have had an opportunity to look at the final solution I have one concern. We went through a great lengths to design a container that has a low lock contention [1] while maintaining high precision of a sophisticated eviction algorithm. Goals outlined in this JIRA have lead you in a direction to redesign the original proposal. I was wondering if it is possible to use your MemoryMonitor to make a decision whether or not to grow BCHM rather than to check if eviction should be done upon each put invocation? Currently we preset the size of BCHM and we never grow it (i.e. Segment#rehash is never invoked). Maybe this is a place where MemoryMonitor comes as a natural fit. We can keep the current design as outlined in [1] while reaping the benefits of a memory bound resizeable container.


WDYT?


Regards,
Vladimir



[1] http://infinispan.blogspot.com/2010/03/infinispan-eviction-batching-updates.html


Dave said:

It took me some time to figure out how to use Git and add some more tests. I have pushed my changes to a topic branch located at https://github.com/dlmarion/infinispan/tree/ISPN-863-master.

Please let me know if you have any questions or concerns. I have put up some documentation at https://github.com/dlmarion/infinispan/wiki/ISPN-863-Implementation

 

 

-- Dave Marion
_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev