Quantcast

[infinispan-dev] Video for DevNation talk on auto-scaling Infinispan with Kubernetes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Video for DevNation talk on auto-scaling Infinispan with Kubernetes

Galder Zamarreño
Hi all,

More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/

Enjoy!
--
Galder Zamarreño
Infinispan, Red Hat


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Video for DevNation talk on auto-scaling Infinispan with Kubernetes

Sebastian Laskawiec
Thanks Galder for this finding!

Just a small comment on things which are not obvious - Ray created his own custom autoscaler implementation. Both OpenShift and Kubernetes have support for so called Custom Metrics [1][2] which you can use for autoscaling. 

However, trying to scale based on the memory consumption is a tricky business [3][4]. I've been talking with Andrew about this and he says that settings `-Xms` to some low value and `-Xmx` reasonably high might do the trick. When you add a node to the cluster (assuming only DIST caches) the total amount of used memory in the Pods will (possibly) drop after some time. This however might not work for CMS GC. 

Anyway - those are the things which are on my plate for this and next week. Once I have some results I would be glad to share them with you. 

And as always - any help with this would be highly appreciated.

Thanks
Sebastian



On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarreño <[hidden email]> wrote:
Hi all,

More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/

Enjoy!
--
Galder Zamarreño
Infinispan, Red Hat


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Video for DevNation talk on auto-scaling Infinispan with Kubernetes

Galder Zamarreño
Hey Sebastian,

Yeah, as Ray rightly says in the video, his approach is based on old Kubernetes versions, and so the current status might be different.

What about you write up a blog with a link to the video and you explain the best practices or more recent approaches that we're using these days? A comparison might be handy for our users to better understand the concepts.

WDYT?

Cheers,
--
Galder Zamarreño
Infinispan, Red Hat

> On 3 Aug 2016, at 11:08, Sebastian Laskawiec <[hidden email]> wrote:
>
> Thanks Galder for this finding!
>
> Just a small comment on things which are not obvious - Ray created his own custom autoscaler implementation. Both OpenShift and Kubernetes have support for so called Custom Metrics [1][2] which you can use for autoscaling.
>
> However, trying to scale based on the memory consumption is a tricky business [3][4]. I've been talking with Andrew about this and he says that settings `-Xms` to some low value and `-Xmx` reasonably high might do the trick. When you add a node to the cluster (assuming only DIST caches) the total amount of used memory in the Pods will (possibly) drop after some time. This however might not work for CMS GC.
>
> Anyway - those are the things which are on my plate for this and next week. Once I have some results I would be glad to share them with you.
>
> And as always - any help with this would be highly appreciated.
>
> Thanks
> Sebastian
>
> [1] https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md
> [2] https://github.com/openshift/origin-metrics
> [3] https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/
> [4] https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/
>
>
> On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarreño <[hidden email]> wrote:
> Hi all,
>
> More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
> http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/
>
> Enjoy!
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Video for DevNation talk on auto-scaling Infinispan with Kubernetes

Sebastian Laskawiec
That's a good idea, but let me first figure out how to do it properly (and make sure it works as I expect)...

On Wed, Aug 3, 2016 at 11:16 AM, Galder Zamarreño <[hidden email]> wrote:
Hey Sebastian,

Yeah, as Ray rightly says in the video, his approach is based on old Kubernetes versions, and so the current status might be different.

What about you write up a blog with a link to the video and you explain the best practices or more recent approaches that we're using these days? A comparison might be handy for our users to better understand the concepts.

WDYT?

Cheers,
--
Galder Zamarreño
Infinispan, Red Hat

> On 3 Aug 2016, at 11:08, Sebastian Laskawiec <[hidden email]> wrote:
>
> Thanks Galder for this finding!
>
> Just a small comment on things which are not obvious - Ray created his own custom autoscaler implementation. Both OpenShift and Kubernetes have support for so called Custom Metrics [1][2] which you can use for autoscaling.
>
> However, trying to scale based on the memory consumption is a tricky business [3][4]. I've been talking with Andrew about this and he says that settings `-Xms` to some low value and `-Xmx` reasonably high might do the trick. When you add a node to the cluster (assuming only DIST caches) the total amount of used memory in the Pods will (possibly) drop after some time. This however might not work for CMS GC.
>
> Anyway - those are the things which are on my plate for this and next week. Once I have some results I would be glad to share them with you.
>
> And as always - any help with this would be highly appreciated.
>
> Thanks
> Sebastian
>
> [1] https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md
> [2] https://github.com/openshift/origin-metrics
> [3] https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/
> [4] https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/
>
>
> On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarreño <[hidden email]> wrote:
> Hi all,
>
> More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
> http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/
>
> Enjoy!
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...