[infinispan-dev] ISPN-863

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] ISPN-863

david marion

Manik,

 

I am on the list. I will start getting my dev environment setup according to instructions on site.

 

Dave Marion

 
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863

Manik Surtani
<base href="x-msg://56/">Excellent, Dave.

You should probably come up with a design first - or maybe a prototype on your fork of the project on GitHub - and post it here for comment.

Cheers
Manik

On 8 Jan 2011, at 00:34, david marion wrote:

Manik,
 
I am on the list. I will start getting my dev environment setup according to instructions on site.
 
Dave Marion
  _______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Poor man's size-based eviction Re: ISPN-863

Manik Surtani
Hi Dave

Just saw that you signed the contributor agreement.

So yeah, what you may want to look into is the EvictionManager [1] which will periodically have its processEviction() [2] method called.  This doesn't actually do any evictions here (named so for legacy reasons) since the current data container is self-sizing/evicting.  However it is still periodically invoked to purge expired entries.  

You can piggyback on the same invocation to test upper and lower bounds of memory utilisation (if enabled), but the bit that I still think you need to figure out is deciding which entries to evict, and how many of them to evict.

Cheers
Manik


On 10 Jan 2011, at 11:08, Manik Surtani wrote:

Excellent, Dave.

You should probably come up with a design first - or maybe a prototype on your fork of the project on GitHub - and post it here for comment.

Cheers
Manik

On 8 Jan 2011, at 00:34, david marion wrote:

Manik,
 
I am on the list. I will start getting my dev environment setup according to instructions on site.
 
Dave Marion
  _______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

david marion

I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.

 

Eviction currently appears to happen in two different ways:

 

1.       If an entry is expired, then it can be evicted.

2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.

 

Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.

 

  1. Modify configuration in some way so that the following can be specified:
    1. The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
    2. The number of items that should be evicted when memory reaches this threshold
  2. Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
  3. Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
  4. Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
  5. Create an instance of the new Eviction class in each segment.
  6. Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.

 

Questions:

 

  1. What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?

 

Thoughts

 

  1. This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.

 

 

-- Dave Marion


 

From: [hidden email]
Date: Tue, 11 Jan 2011 11:46:40 +0000
To: [hidden email]
Subject: [infinispan-dev] Poor man's size-based eviction Re: ISPN-863

Hi Dave

Just saw that you signed the contributor agreement.

So yeah, what you may want to look into is the EvictionManager [1] which will periodically have its processEviction() [2] method called.  This doesn't actually do any evictions here (named so for legacy reasons) since the current data container is self-sizing/evicting.  However it is still periodically invoked to purge expired entries.  

You can piggyback on the same invocation to test upper and lower bounds of memory utilisation (if enabled), but the bit that I still think you need to figure out is deciding which entries to evict, and how many of them to evict.

Cheers
Manik


On 10 Jan 2011, at 11:08, Manik Surtani wrote:

Excellent, Dave.

You should probably come up with a design first - or maybe a prototype on your fork of the project on GitHub - and post it here for comment.

Cheers
Manik

On 8 Jan 2011, at 00:34, david marion wrote:

Manik,
 
I am on the list. I will start getting my dev environment setup according to instructions on site.
 
Dave Marion
  _______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
Excellent, memory based eviction is very important ! This has been on
the todo list for years !

One thing you might want to think about is - *in addition* to what you
say below - whether you want to add an OOME handler, which evits on OOME
caused by putXXX() methods.

For example, you could wrap all methods which add to the map to catch
OOMEs. When this is the case, kick off the eviction handler to evict
elements immediately. Kind of like a second line of defense against OOMEs...


WDYT ?



On 1/14/11 4:22 AM, david marion wrote:

>
> I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
>
> Eviction currently appears to happen in two different ways:
>
> 1.       If an entry is expired, then it can be evicted.
> 2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
>
> Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
>
>
> Modify configuration in some way so that the following can be specified:
>
> The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
> The number of items that should be evicted when memory reaches this threshold
> Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
> Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
> Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
> Create an instance of the new Eviction class in each segment.
> Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
>
> Questions:
>
>
> What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
>
> Thoughts
>
>
> This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Manik Surtani
Nice, I like the "2nd level of defence" here.

On 14 Jan 2011, at 08:40, Bela Ban wrote:

> Excellent, memory based eviction is very important ! This has been on
> the todo list for years !
>
> One thing you might want to think about is - *in addition* to what you
> say below - whether you want to add an OOME handler, which evits on OOME
> caused by putXXX() methods.
>
> For example, you could wrap all methods which add to the map to catch
> OOMEs. When this is the case, kick off the eviction handler to evict
> elements immediately. Kind of like a second line of defense against OOMEs...
>
>
> WDYT ?
>
>
>
> On 1/14/11 4:22 AM, david marion wrote:
>>
>> I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
>>
>> Eviction currently appears to happen in two different ways:
>>
>> 1.       If an entry is expired, then it can be evicted.
>> 2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
>>
>> Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
>>
>>
>> Modify configuration in some way so that the following can be specified:
>>
>> The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
>> The number of items that should be evicted when memory reaches this threshold
>> Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
>> Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
>> Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
>> Create an instance of the new Eviction class in each segment.
>> Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
>>
>> Questions:
>>
>>
>> What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
>>
>> Thoughts
>>
>>
>> This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Manik Surtani
In reply to this post by david marion
<base href="x-msg://29/">I'll let Vladimir comment more as he wrote most of this code originally, but it sounds OK in principle.  

Would you need to check the memory usage on every put and replace?  Wouldn't this be expensive?  Couldn't this happen periodically?

On 14 Jan 2011, at 03:22, david marion wrote:

I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
 
Eviction currently appears to happen in two different ways:
 
1.       If an entry is expired, then it can be evicted.
2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
 
Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
 
  1. Modify configuration in some way so that the following can be specified:
    1. The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
    2. The number of items that should be evicted when memory reaches this threshold
  2. Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
  3. Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
  4. Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
  5. Create an instance of the new Eviction class in each segment.
  6. Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
 
Questions:
 
  1. What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
 
Thoughts
 
  1. This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
 
 
-- Dave Marion

 

From: [hidden email]
Date: Tue, 11 Jan 2011 11:46:40 +0000
To: [hidden email]
Subject: [infinispan-dev] Poor man's size-based eviction Re: ISPN-863

Hi Dave

Just saw that you signed the contributor agreement.

So yeah, what you may want to look into is the EvictionManager [1] which will periodically have its processEviction() [2] method called.  This doesn't actually do any evictions here (named so for legacy reasons) since the current data container is self-sizing/evicting.  However it is still periodically invoked to purge expired entries.  

You can piggyback on the same invocation to test upper and lower bounds of memory utilisation (if enabled), but the bit that I still think you need to figure out is deciding which entries to evict, and how many of them to evict.

Cheers
Manik


On 10 Jan 2011, at 11:08, Manik Surtani wrote:

Excellent, Dave.

You should probably come up with a design first - or maybe a prototype on your fork of the project on GitHub - and post it here for comment.

Cheers
Manik

On 8 Jan 2011, at 00:34, david marion wrote:

Manik,
 
I am on the list. I will start getting my dev environment setup according to instructions on site.
 
Dave Marion
  _______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Galder Zamarreno
In reply to this post by Bela Ban
-1

If you get an OOME, chances are the system is a pretty bad shape and you can't really do much else at that point.

On Jan 14, 2011, at 9:40 AM, Bela Ban wrote:

> Excellent, memory based eviction is very important ! This has been on
> the todo list for years !
>
> One thing you might want to think about is - *in addition* to what you
> say below - whether you want to add an OOME handler, which evits on OOME
> caused by putXXX() methods.
>
> For example, you could wrap all methods which add to the map to catch
> OOMEs. When this is the case, kick off the eviction handler to evict
> elements immediately. Kind of like a second line of defense against OOMEs...
>
>
> WDYT ?
>
>
>
> On 1/14/11 4:22 AM, david marion wrote:
>>
>> I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
>>
>> Eviction currently appears to happen in two different ways:
>>
>> 1.       If an entry is expired, then it can be evicted.
>> 2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
>>
>> Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
>>
>>
>> Modify configuration in some way so that the following can be specified:
>>
>> The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
>> The number of items that should be evicted when memory reaches this threshold
>> Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
>> Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
>> Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
>> Create an instance of the new Eviction class in each segment.
>> Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
>>
>> Questions:
>>
>>
>> What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
>>
>> Thoughts
>>
>>
>> This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

david marion
In reply to this post by Bela Ban
We could definitely try that. I'm not sure what happens internally when an OOM is raised by the VM.
 
> Date: Fri, 14 Jan 2011 09:40:37 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [infinispan-dev] ISPN-863 - Thoughts / Questions
>
> Excellent, memory based eviction is very important ! This has been on
> the todo list for years !
>
> One thing you might want to think about is - *in addition* to what you
> say below - whether you want to add an OOME handler, which evits on OOME
> caused by putXXX() methods.
>
> For example, you could wrap all methods which add to the map to catch
> OOMEs. When this is the case, kick off the eviction handler to evict
> elements immediately. Kind of like a second line of defense against OOMEs...
>
>
> WDYT ?
>
>
>
> On 1/14/11 4:22 AM, david marion wrote:
> >
> > I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
> >
> > Eviction currently appears to happen in two different ways:
> >
> > 1. If an entry is expired, then it can be evicted.
> > 2. If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
> >
> > Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
> >
> >
> > Modify configuration in some way so that the following can be specified:
> >
> > The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
> > The number of items that should be evicted when memory reaches this threshold
> > Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
> > Create a new Eviction class, a subclass of LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
> > Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
> > Create an instance of the new Eviction class in each segment.
> > Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
> >
> > Questions:
> >
> >
> > What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
> >
> > Thoughts
> >
> >
> > This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
In reply to this post by Galder Zamarreno


On 1/14/11 3:08 PM, Galder Zamarreño wrote:
> -1
>
> If you get an OOME, chances are the system is a pretty bad shape and you can't really do much else at that point.


If a put() is the culprit leading to the OOME, we can definitely do
something !

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
In reply to this post by david marion
Very simple: when a malloc() fails, an OOME (runtime exception) is
thrown. We can catch it and see if we can help reducing memory by
evicting elements from the cache. Note that OOMEs are also thrown when
the stack is exhausted (IIRC) and when a new thread cannot be created.

Definitely something worth trying out...


On 1/14/11 3:26 PM, david marion wrote:

>
> We could definitely try that. I'm not sure what happens internally when an OOM is raised by the VM.
>
>> Date: Fri, 14 Jan 2011 09:40:37 +0100
>> From: [hidden email]
>> To: [hidden email]
>> Subject: Re: [infinispan-dev] ISPN-863 - Thoughts / Questions
>>
>> Excellent, memory based eviction is very important ! This has been on
>> the todo list for years !
>>
>> One thing you might want to think about is - *in addition* to what you
>> say below - whether you want to add an OOME handler, which evits on OOME
>> caused by putXXX() methods.
>>
>> For example, you could wrap all methods which add to the map to catch
>> OOMEs. When this is the case, kick off the eviction handler to evict
>> elements immediately. Kind of like a second line of defense against OOMEs...
>>
>>
>> WDYT ?
>>
>>
>>
>> On 1/14/11 4:22 AM, david marion wrote:
>>>
>>> I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
>>>
>>> Eviction currently appears to happen in two different ways:
>>>
>>> 1. If an entry is expired, then it can be evicted.
>>> 2. If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
>>>
>>> Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
>>>
>>>
>>> Modify configuration in some way so that the following can be specified:
>>>
>>> The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
>>> The number of items that should be evicted when memory reaches this threshold
>>> Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
>>> Create a new Eviction class, a subclass of LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
>>> Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
>>> Create an instance of the new Eviction class in each segment.
>>> Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
>>>
>>> Questions:
>>>
>>>
>>> What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
>>>
>>> Thoughts
>>>
>>>
>>> This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
>>
>> --
>> Bela Ban
>> Lead JGroups / Clustering Team
>> JBoss
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>    
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

david marion
In reply to this post by Manik Surtani
 
  I agree. There are two ways to determine what the memory usage is, polling or notifications. The notification mechanism is not guaranteed, so we could have a background thread that polls to determine whether or not the threshold has been passed. Then in the put and replace methods, it would be the cost of checking a boolean value.
 

From: [hidden email]
Date: Fri, 14 Jan 2011 11:45:29 +0000
To: [hidden email]
Subject: Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

I'll let Vladimir comment more as he wrote most of this code originally, but it sounds OK in principle.  

Would you need to check the memory usage on every put and replace?  Wouldn't this be expensive?  Couldn't this happen periodically?

On 14 Jan 2011, at 03:22, david marion wrote:

I have been looking at the code and think I have a starting point on the design. Remember, the goal here is to be able to create a cache that uses expiration but has no defined cache size limit. Reading through the code, if ‘-1’ is specified for the eviction maxEntries, then a ConcurrentHashMap is created and used as the data container and no eviction is performed. Otherwise a BoundedConcurrentHashMap is created and used as the data container and eviction is performed according to the specified strategy.
 
Eviction currently appears to happen in two different ways:
 
1.       If an entry is expired, then it can be evicted.
2.       If the number of entries in a segment of the cache contains more entries than “allowed” by the configuration.
 
Here are the modifications that I think need to be made. I have not made any code mods yet, so I don’t know for sure if this will work. Definitely looking for feedback.
 
  1. Modify configuration in some way so that the following can be specified:
    1. The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
    2. The number of items that should be evicted when memory reaches this threshold
  2. Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
  3. Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
  4. Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap
  5. Create an instance of the new Eviction class in each segment.
  6. Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.
 
Questions:
 
  1. What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?
 
Thoughts
 
  1. This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.
 
 
-- Dave Marion

 

From: [hidden email]
Date: Tue, 11 Jan 2011 11:46:40 +0000
To: [hidden email]
Subject: [infinispan-dev] Poor man's size-based eviction Re: ISPN-863

Hi Dave

Just saw that you signed the contributor agreement.

So yeah, what you may want to look into is the EvictionManager [1] which will periodically have its processEviction() [2] method called.  This doesn't actually do any evictions here (named so for legacy reasons) since the current data container is self-sizing/evicting.  However it is still periodically invoked to purge expired entries.  

You can piggyback on the same invocation to test upper and lower bounds of memory utilisation (if enabled), but the bit that I still think you need to figure out is deciding which entries to evict, and how many of them to evict.

Cheers
Manik


On 10 Jan 2011, at 11:08, Manik Surtani wrote:

Excellent, Dave.

You should probably come up with a design first - or maybe a prototype on your fork of the project on GitHub - and post it here for comment.

Cheers
Manik

On 8 Jan 2011, at 00:34, david marion wrote:

Manik,
 
I am on the list. I will start getting my dev environment setup according to instructions on site.
 
Dave Marion
  _______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev



_______________________________________________ infinispan-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Manik Surtani
In reply to this post by Bela Ban

On 14 Jan 2011, at 14:29, Bela Ban wrote:

>
>
> On 1/14/11 3:08 PM, Galder Zamarreño wrote:
>> -1
>>
>> If you get an OOME, chances are the system is a pretty bad shape and you can't really do much else at that point.
>
>
> If a put() is the culprit leading to the OOME, we can definitely do
> something !

Galder is right though, even if the put() is the culprit, concurrent threads elsewhere (out of our control) would also be seeing OOMs and even if we free up memory by evicting, other threads may be in an unstable state.

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
In reply to this post by david marion


On 1/14/11 3:32 PM, david marion wrote:
>
>
>    I agree. There are two ways to determine what the memory usage is, polling or notifications. The notification mechanism is not guaranteed


Are you referring to MemoryMXBean / MemoryPoolMXBean ? What do you mean
by not guaranteed ? Are you suggesting the notfications are not
implemented by all JVM vendors ?


> so we could have a background thread that polls to determine whether or not the threshold has been passed. Then in the put and replace methods, it would be the cost of checking a boolean value.

When notifications are available, we should use them, otherwise polling
would be fine, I guess.


--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

david marion
I should have been more specific, notification delivery is not guaranteed. From http://download.oracle.com/javase/6/docs/api/java/lang/management/MemoryPoolMXBean.html:
 
"There is no guarantee about when the MemoryMXBean will emit a threshold notification and when the notification will be delivered. "
 
> Date: Fri, 14 Jan 2011 15:46:02 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [infinispan-dev] ISPN-863 - Thoughts / Questions
>
>
>
> On 1/14/11 3:32 PM, david marion wrote:
> >
> >
> > I agree. There are two ways to determine what the memory usage is, polling or notifications. The notification mechanism is not guaranteed
>
>
> Are you referring to MemoryMXBean / MemoryPoolMXBean ? What do you mean
> by not guaranteed ? Are you suggesting the notfications are not
> implemented by all JVM vendors ?
>
>
> > so we could have a background thread that polls to determine whether or not the threshold has been passed. Then in the put and replace methods, it would be the cost of checking a boolean value.
>
> When notifications are available, we should use them, otherwise polling
> would be fine, I guess.
>
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
In reply to this post by Manik Surtani


On 1/14/11 3:35 PM, Manik Surtani wrote:

>> If a put() is the culprit leading to the OOME, we can definitely do
>> something !
>
> Galder is right though, even if the put() is the culprit, concurrent threads elsewhere (out of our control) would also be seeing OOMs and even if we free up memory by evicting, other threads may be in an unstable state.

JGroups tries to handle temporary OOMEs, and if Infinispan does that,
too, then it's only the app left which can cause OOMEs.

But at least the middleware (JGroups, Infinispan)  should go to great
lengths to make sure it handles OOMEs.

I see a big issue coming when we don't do this: as long as we don't have
a more or less even key/value distribution in Infinispan, there is a
chance we end up with some JVMs lightly loaded (in terms of memory) and
others almost full.

Even if we come up with a better distribution function, we cannot
prevent an application from using objects with uneven sizes that they
stick into the cache, e.g. thousands of smallish values and a few very
large values. This will also cause an uneven memory distribution, even
if the key distribution is even.

One thing we could do on the receiving side is to check memory only if
we receive a value that bigger than a certain size. Problem is what
happens when someone sets a value...

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Bela Ban
In reply to this post by david marion


On 1/14/11 3:48 PM, david marion wrote:
>
> I should have been more specific, notification delivery is not guaranteed. From http://download.oracle.com/javase/6/docs/api/java/lang/management/MemoryPoolMXBean.html:
>
> "There is no guarantee about when the MemoryMXBean will emit a threshold notification and when the notification will be delivered."


Interesting. I suggest we try it out to see what the behavior is. If
this is unreliable, I guess we have to switch to polling

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Manik Surtani
In reply to this post by Bela Ban
Yeah, I do agree that we should do this (best effort); just that we shouldn't fool ourselves into thinking that this is going to solve all problems caused by an OOM in Infinispan.

Also, re: middleware, this is often more than just JGroups + Infinispan (think JBoss AS, maybe SEAM, some web fwk, or Spring, and a whole bunch of user code that could be left unstable).

On 14 Jan 2011, at 14:54, Bela Ban wrote:

>
>
> On 1/14/11 3:35 PM, Manik Surtani wrote:
>
>>> If a put() is the culprit leading to the OOME, we can definitely do
>>> something !
>>
>> Galder is right though, even if the put() is the culprit, concurrent threads elsewhere (out of our control) would also be seeing OOMs and even if we free up memory by evicting, other threads may be in an unstable state.
>
> JGroups tries to handle temporary OOMEs, and if Infinispan does that,
> too, then it's only the app left which can cause OOMEs.
>
> But at least the middleware (JGroups, Infinispan)  should go to great
> lengths to make sure it handles OOMEs.
>
> I see a big issue coming when we don't do this: as long as we don't have
> a more or less even key/value distribution in Infinispan, there is a
> chance we end up with some JVMs lightly loaded (in terms of memory) and
> others almost full.
>
> Even if we come up with a better distribution function, we cannot
> prevent an application from using objects with uneven sizes that they
> stick into the cache, e.g. thousands of smallish values and a few very
> large values. This will also cause an uneven memory distribution, even
> if the key distribution is even.
>
> One thing we could do on the receiving side is to check memory only if
> we receive a value that bigger than a certain size. Problem is what
> happens when someone sets a value...
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Vladimir Blagojevic
In reply to this post by david marion
Hey David,

First of all kudos for taking on such a non-trivial task!

On 11-01-14 12:22 AM, david marion wrote:
  1. Modify configuration in some way so that the following can be specified:
    1. The percentage value of used JVM memory (i.e. 95) at which entries should be evicted to try and avoid an OOM error.
    2. The number of items that should be evicted when memory reaches this threshold
  2. Modify LRU and LIRS Eviction class so that the accessQueue member can be accessed by the new Eviction class so that two access queues don’t have to be maintained.
  3. Create a new Eviction class, a subclass of  LIRS, where the accessQueue is used from the Eviction strategy the user specifies and the for loop in the execute method is exited when the evicted set equals value from 1.b above.
  4. Modify DataContainerFactory.construct() to call DefaultDataContainer.boundedDataContainer() regardless of eviction policy. This will always create a BoundedConcurrentHashMap

I don't think you have to extend LRU and LIRS, as long as you implement EvictionPolicy you are fullfilling the contract. What extending LRU and LIRS can do is give you some order in selecting proper elements for eviction. I think the precise technical term in research literature is "eviction precision".


  1. Create an instance of the new Eviction class in each segment.
  2. Modify BoundedConcurrentHashMap.Segment put and replace methods such that when new values are going to be put into the Segment, the memory usage is checked and the execute method is called on the new Eviction class.

Sounds right!


 

Questions:

 

  1. What are the implications of using a BoundedConcurrentHashMap instead of a ConcurrentHashMap when maxEntries is set to -1?

I think none except you turn on eviction by using BoundedConcurrentHashMap.


 

Thoughts

 

  1. This will not guarantee that an OOM error does not occur. It will attempt to guard against an OOM caused by putting new values into the cache. This will probably be more effective when the cache is being used in client/server mode, and less effective when used in embedded mode as to other code running in the JVM.

 

 

-- Dave Marion



Cheers,
Vladimir

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] ISPN-863 - Thoughts / Questions

Galder Zamarreno
In reply to this post by Manik Surtani

On Jan 14, 2011, at 3:35 PM, Manik Surtani wrote:

>
> On 14 Jan 2011, at 14:29, Bela Ban wrote:
>
>>
>>
>> On 1/14/11 3:08 PM, Galder Zamarreño wrote:
>>> -1
>>>
>>> If you get an OOME, chances are the system is a pretty bad shape and you can't really do much else at that point.
>>
>>
>> If a put() is the culprit leading to the OOME, we can definitely do
>> something !
>
> Galder is right though, even if the put() is the culprit, concurrent threads elsewhere (out of our control) would also be seeing OOMs and even if we free up memory by evicting, other threads may be in an unstable state.

As a side thing, IMO, rather than trying to salvage an OOME, it's better to analise what when wrong adding these two flags:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/java_heap

>
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12