[infinispan-dev] Maintenance of OpenShift templates

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Maintenance of OpenShift templates

Galder Zamarreno
Hi,

Looking at [1] and I'm wondering why the templates have to maintain a
different XML file for OpenShift?

We already ship an XML in the server called `cloud.xml`, that should
just work. Having a separate XML file in the templates means we're
duplicating the maintainance of XML files.

Also, users can now create caches programmatically. This is by far the
most common tweak that had to be done to the config. So, I see the
urgency to change XML files less immediate.

Sure, there will always be people who modify/tweak things and that's
fine. We should however show the people how to do that in a way that
doesn't require us to duplicate our maintanence work.

Also, if we want to show the users how to use a custom XML file, I don't
think we should show them how to embedd it in the template as JSON
[2]. It's quite a pain. Instead, the XML should be kept as a separate
file and the JSON file reference it.

Cheers,

[1]
https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
[2] https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Sebastian Laskawiec
Hey Galder,

Comments inlined.

Thanks,
Seb

On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño <[hidden email]> wrote:
Hi,

Looking at [1] and I'm wondering why the templates have to maintain a
different XML file for OpenShift?

We already ship an XML in the server called `cloud.xml`, that should
just work. Having a separate XML file in the templates means we're
duplicating the maintainance of XML files.

Also, users can now create caches programmatically. This is by far the
most common tweak that had to be done to the config. So, I see the
urgency to change XML files less immediate.

So just to give you guys a bit more context - the templates were created pretty long time ago when we didn't have admin capabilities in Hot Rod and REST. The main argument for putting the whole configuration into a ConfigMap was to make configuration changes easier for the users. With ConfigMap approach they can log into OpenShift UI, go to Resources -> ConfigMaps and edit everything using UI. That's super convenient for hacking in my opinion. Of course, you don't need to do that at all if you don't want. You can just spin up a new Infinispan cluster using `oc new-app`.

There are at least two other ways for changing the configuration that I can think of. The first one is S2I [1][2] (long story short, you need to put your configuration into a git repository and tell OpenShift to build an image based on it). Even though it may seem very convenient, it's OpenShift only solution (and there are no easy (out of the box) options to get this running on raw Kubernetes). I'm not judging whether it's good or bad here, just telling you how it works. The other option would be to tell the users to do exactly the same things we do in our templates themselves. In other words we would remove configuration from the templates and provide a manual for the users how to deal with configuration. I believe this is exactly what Galder is suggesting, right?

Recently we implemented admin commands in the Hot Rod. Assuming that caches created this way are not wiped out during restart (that needs to be checked), we could remove the configuration from the templates and tell the users to create their caches over Hot Rod and REST. However we still need to have a back door for modifying configuration manually since there are some changes that can not be done via admin API.

 

Sure, there will always be people who modify/tweak things and that's
fine. We should however show the people how to do that in a way that
doesn't require us to duplicate our maintanence work.

If we think about further maintenance, I believe we should take more things into consideration. During the last planning meeting Tristan mentioned about bringing the project and the product closer together. On the Cloud Enablement side of things there are ongoing experiments to get a community images out. 

If we decided to take this direction (the CE way), our templates would need to be deprecated or will change drastically. The image will react on different set of variables and configuration options.

Also, if we want to show the users how to use a custom XML file, I don't
think we should show them how to embedd it in the template as JSON
[2]. It's quite a pain. Instead, the XML should be kept as a separate
file and the JSON file reference it.

I'm still struggling to understand why this is a pain. Could you please explain it a bit more? If you look into the maintenance guide [3], there are only a few steps. For me it takes no longer than 15 minutes to do the upgrade. You also mentioned on IRC that this approach is a pain for our users (I believe you mentioned something about Ray). I also can not understand why, could you please explain it a bit more?

 

Cheers,

[1]
https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
[2] https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Galder Zamarreno
Sebastian Laskawiec <[hidden email]> writes:

> Hey Galder,
>
> Comments inlined.
>
> Thanks,
> Seb
>
> On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Hi,
>
>     Looking at [1] and I'm wondering why the templates have to
>     maintain a
>     different XML file for OpenShift?
>
>     We already ship an XML in the server called `cloud.xml`, that
>     should
>     just work. Having a separate XML file in the templates means we're
>     duplicating the maintainance of XML files.
>
>     Also, users can now create caches programmatically. This is by far
>     the
>     most common tweak that had to be done to the config. So, I see the
>     urgency to change XML files less immediate.
>
> So just to give you guys a bit more context - the templates were
> created pretty long time ago when we didn't have admin capabilities in
> Hot Rod and REST. The main argument for putting the whole
> configuration into a ConfigMap was to make configuration changes
> easier for the users. With ConfigMap approach they can log into
> OpenShift UI, go to Resources -> ConfigMaps and edit everything using
> UI. That's super convenient for hacking in my opinion. Of course, you
> don't need to do that at all if you don't want. You can just spin up a
> new Infinispan cluster using `oc new-app`.

I agree with the usability of the ConfigMap. However, the duplication is
very annoying. Would it be possible for the ConfigMap to be created on
the fly out of the cloud.xml that's shipped by Infinispan Server? That
way we'd still have a ConfigMap without having to duplicate XML.

> There are at least two other ways for changing the configuration that
> I can think of. The first one is S2I [1][2] (long story short, you
> need to put your configuration into a git repository and tell
> OpenShift to build an image based on it). Even though it may seem very
> convenient, it's OpenShift only solution (and there are no easy (out
> of the box) options to get this running on raw Kubernetes). I'm not
> judging whether it's good or bad here, just telling you how it works.
> The other option would be to tell the users to do exactly the same
> things we do in our templates themselves. In other words we would
> remove configuration from the templates and provide a manual for the
> users how to deal with configuration. I believe this is exactly what
> Galder is suggesting, right?

What we do in the templates right now to show users how to tweak their
config is in convoluted.

Ideally, adding their own custom configuration should be just a matter
of:

1. Creating a ConfigMap yaml pointing to an XML.
2. Ask users to put their XML in a separate file pointed by the ConfigMap.
3. Deploy ConfigMap and XML.
4. Trigger a new Infinispan redeployment.

Not sure how doable this is with the current template approach, or we
could explain how to do this for an already up and running application
that has Infinispan created out of the default template?

>
> Recently we implemented admin commands in the Hot Rod. Assuming that
> caches created this way are not wiped out during restart (that needs
> to be checked), we could remove the configuration from the templates
> and tell the users to create their caches over Hot Rod and REST.
> However we still need to have a back door for modifying configuration
> manually since there are some changes that can not be done via admin
> API.
>
> [1] https://github.com/openshift/source-to-image
> [2]
> https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>
>
>     Sure, there will always be people who modify/tweak things and
>     that's
>     fine. We should however show the people how to do that in a way
>     that
>     doesn't require us to duplicate our maintanence work.
>
> If we think about further maintenance, I believe we should take more
> things into consideration. During the last planning meeting Tristan
> mentioned about bringing the project and the product closer together.
> On the Cloud Enablement side of things there are ongoing experiments
> to get a community images out.
>
> If we decided to take this direction (the CE way), our templates would
> need to be deprecated or will change drastically. The image will react
> on different set of variables and configuration options.
>
>     Also, if we want to show the users how to use a custom XML file, I
>     don't
>     think we should show them how to embedd it in the template as JSON
>     [2]. It's quite a pain. Instead, the XML should be kept as a
>     separate
>     file and the JSON file reference it.
>
> I'm still struggling to understand why this is a pain. Could you
> please explain it a bit more? If you look into the maintenance guide
> [3], there are only a few steps. For me it takes no longer than 15
> minutes to do the upgrade. You also mentioned on IRC that this
> approach is a pain for our users (I believe you mentioned something
> about Ray). I also can not understand why, could you please explain it
> a bit more?
>
> [3]
> https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>
>
>     Cheers,
>
>     [1]
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>
>     [2]
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>
>     _______________________________________________
>     infinispan-dev mailing list
>     [hidden email]
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Sebastian Laskawiec


On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <[hidden email]> wrote:
Sebastian Laskawiec <[hidden email]> writes:

> Hey Galder,
>
> Comments inlined.
>
> Thanks,
> Seb
>
> On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Hi,
>
>     Looking at [1] and I'm wondering why the templates have to
>     maintain a
>     different XML file for OpenShift?
>
>     We already ship an XML in the server called `cloud.xml`, that
>     should
>     just work. Having a separate XML file in the templates means we're
>     duplicating the maintainance of XML files.
>
>     Also, users can now create caches programmatically. This is by far
>     the
>     most common tweak that had to be done to the config. So, I see the
>     urgency to change XML files less immediate.
>
> So just to give you guys a bit more context - the templates were
> created pretty long time ago when we didn't have admin capabilities in
> Hot Rod and REST. The main argument for putting the whole
> configuration into a ConfigMap was to make configuration changes
> easier for the users. With ConfigMap approach they can log into
> OpenShift UI, go to Resources -> ConfigMaps and edit everything using
> UI. That's super convenient for hacking in my opinion. Of course, you
> don't need to do that at all if you don't want. You can just spin up a
> new Infinispan cluster using `oc new-app`.

I agree with the usability of the ConfigMap. However, the duplication is
very annoying. Would it be possible for the ConfigMap to be created on
the fly out of the cloud.xml that's shipped by Infinispan Server? That
way we'd still have a ConfigMap without having to duplicate XML.

Probably not. This would require special permissions to call Kubernetes API from the Pod. In other words, I can't think about any other way that would work in OpenShift Online for the instance.
 

> There are at least two other ways for changing the configuration that
> I can think of. The first one is S2I [1][2] (long story short, you
> need to put your configuration into a git repository and tell
> OpenShift to build an image based on it). Even though it may seem very
> convenient, it's OpenShift only solution (and there are no easy (out
> of the box) options to get this running on raw Kubernetes). I'm not
> judging whether it's good or bad here, just telling you how it works.
> The other option would be to tell the users to do exactly the same
> things we do in our templates themselves. In other words we would
> remove configuration from the templates and provide a manual for the
> users how to deal with configuration. I believe this is exactly what
> Galder is suggesting, right?

What we do in the templates right now to show users how to tweak their
config is in convoluted.

Ideally, adding their own custom configuration should be just a matter
of:

1. Creating a ConfigMap yaml pointing to an XML.
2. Ask users to put their XML in a separate file pointed by the ConfigMap.
3. Deploy ConfigMap and XML.
4. Trigger a new Infinispan redeployment.

That would probably need to be a new deployment. Most of the StatefulSet spec is immutable.
 

Not sure how doable this is with the current template approach, or we
could explain how to do this for an already up and running application
that has Infinispan created out of the default template?

I've been thinking about this for a while and this is what I think we should do:
  1. Wait a couple of weeks and review the community image created by the CE Team. See if this is a good fit for us. If it is, I would focus on adopting this approach and adjust our templates to handle it.
  2. Whether or not we adopt the CE community work, we could put all necessary stuff into cloud.xml or services.xml configuration. We could do one step forward and merge them together. 
  3. Make sure that dynamically created caches are persisted (this is super important!!)
  4. Once #3 is verified we should have a decision whether or not we are adopting the CE way. At this point we could document how to use custom configuration with a ConfigMap and drop it from the templates.
WDYT? Does this plan makes sense to you?
 

>
> Recently we implemented admin commands in the Hot Rod. Assuming that
> caches created this way are not wiped out during restart (that needs
> to be checked), we could remove the configuration from the templates
> and tell the users to create their caches over Hot Rod and REST.
> However we still need to have a back door for modifying configuration
> manually since there are some changes that can not be done via admin
> API.
>
> [1] https://github.com/openshift/source-to-image
> [2]
> https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>
>
>     Sure, there will always be people who modify/tweak things and
>     that's
>     fine. We should however show the people how to do that in a way
>     that
>     doesn't require us to duplicate our maintanence work.
>
> If we think about further maintenance, I believe we should take more
> things into consideration. During the last planning meeting Tristan
> mentioned about bringing the project and the product closer together.
> On the Cloud Enablement side of things there are ongoing experiments
> to get a community images out.
>
> If we decided to take this direction (the CE way), our templates would
> need to be deprecated or will change drastically. The image will react
> on different set of variables and configuration options.
>
>     Also, if we want to show the users how to use a custom XML file, I
>     don't
>     think we should show them how to embedd it in the template as JSON
>     [2]. It's quite a pain. Instead, the XML should be kept as a
>     separate
>     file and the JSON file reference it.
>
> I'm still struggling to understand why this is a pain. Could you
> please explain it a bit more? If you look into the maintenance guide
> [3], there are only a few steps. For me it takes no longer than 15
> minutes to do the upgrade. You also mentioned on IRC that this
> approach is a pain for our users (I believe you mentioned something
> about Ray). I also can not understand why, could you please explain it
> a bit more?
>
> [3]
> https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>
>
>     Cheers,
>
>     [1]
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>
>     [2]
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>
>     _______________________________________________
>     infinispan-dev mailing list
>     [hidden email]
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Galder Zamarreno
Sebastian Laskawiec <[hidden email]> writes:

> On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Sebastian Laskawiec <[hidden email]> writes:
>
>     > Hey Galder,
>     >
>     > Comments inlined.
>     >
>     > Thanks,
>     > Seb
>     >
>     > On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño
>     <[hidden email]>
>     > wrote:
>     >
>     > Hi,
>     >
>     > Looking at [1] and I'm wondering why the templates have to
>     > maintain a
>     > different XML file for OpenShift?
>     >
>     > We already ship an XML in the server called `cloud.xml`, that
>     > should
>     > just work. Having a separate XML file in the templates means
>     we're
>     > duplicating the maintainance of XML files.
>     >
>     > Also, users can now create caches programmatically. This is by
>     far
>     > the
>     > most common tweak that had to be done to the config. So, I see
>     the
>     > urgency to change XML files less immediate.
>     >
>     > So just to give you guys a bit more context - the templates were
>     > created pretty long time ago when we didn't have admin
>     capabilities in
>     > Hot Rod and REST. The main argument for putting the whole
>     > configuration into a ConfigMap was to make configuration changes
>     > easier for the users. With ConfigMap approach they can log into
>     > OpenShift UI, go to Resources -> ConfigMaps and edit everything
>     using
>     > UI. That's super convenient for hacking in my opinion. Of
>     course, you
>     > don't need to do that at all if you don't want. You can just
>     spin up a
>     > new Infinispan cluster using `oc new-app`.
>
>     I agree with the usability of the ConfigMap. However, the
>     duplication is
>     very annoying. Would it be possible for the ConfigMap to be
>     created on
>     the fly out of the cloud.xml that's shipped by Infinispan Server?
>     That
>     way we'd still have a ConfigMap without having to duplicate XML.
>
> Probably not. This would require special permissions to call
> Kubernetes API from the Pod. In other words, I can't think about any
> other way that would work in OpenShift Online for the instance.
>
>     > There are at least two other ways for changing the configuration
>     that
>     > I can think of. The first one is S2I [1][2] (long story short,
>     you
>     > need to put your configuration into a git repository and tell
>     > OpenShift to build an image based on it). Even though it may
>     seem very
>     > convenient, it's OpenShift only solution (and there are no easy
>     (out
>     > of the box) options to get this running on raw Kubernetes). I'm
>     not
>     > judging whether it's good or bad here, just telling you how it
>     works.
>     > The other option would be to tell the users to do exactly the
>     same
>     > things we do in our templates themselves. In other words we
>     would
>     > remove configuration from the templates and provide a manual for
>     the
>     > users how to deal with configuration. I believe this is exactly
>     what
>     > Galder is suggesting, right?
>
>     What we do in the templates right now to show users how to tweak
>     their
>     config is in convoluted.
>
>     Ideally, adding their own custom configuration should be just a
>     matter
>     of:
>
>     1. Creating a ConfigMap yaml pointing to an XML.
>     2. Ask users to put their XML in a separate file pointed by the
>     ConfigMap.
>     3. Deploy ConfigMap and XML.
>     4. Trigger a new Infinispan redeployment.
>
> That would probably need to be a new deployment. Most of the
> StatefulSet spec is immutable.
>
>     Not sure how doable this is with the current template approach, or
>     we
>     could explain how to do this for an already up and running
>     application
>     that has Infinispan created out of the default template?
>
> I've been thinking about this for a while and this is what I think we
> should do:
>
> 1 Wait a couple of weeks and review the community image created by the
>   CE Team. See if this is a good fit for us. If it is, I would focus
>   on adopting this approach and adjust our templates to handle it.
> 2 Whether or not we adopt the CE community work, we could put all
>   necessary stuff into cloud.xml or services.xml configuration. We
>   could do one step forward and merge them together.
> 3 Make sure that dynamically created caches are persisted (this is
>   super important!!)
> 4 Once #3 is verified we should have a decision whether or not we are
>   adopting the CE way. At this point we could document how to use
>   custom configuration with a ConfigMap and drop it from the
>   templates.
>
> WDYT? Does this plan makes sense to you?

Sounds good

>
>     >
>     > Recently we implemented admin commands in the Hot Rod. Assuming
>     that
>     > caches created this way are not wiped out during restart (that
>     needs
>     > to be checked), we could remove the configuration from the
>     templates
>     > and tell the users to create their caches over Hot Rod and REST.
>     > However we still need to have a back door for modifying
>     configuration
>     > manually since there are some changes that can not be done via
>     admin
>     > API.
>     >
>     > [1] https://github.com/openshift/source-to-image
>     > [2]
>     >
>     https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>    
>     >
>     >
>     > Sure, there will always be people who modify/tweak things and
>     > that's
>     > fine. We should however show the people how to do that in a way
>     > that
>     > doesn't require us to duplicate our maintanence work.
>     >
>     > If we think about further maintenance, I believe we should take
>     more
>     > things into consideration. During the last planning meeting
>     Tristan
>     > mentioned about bringing the project and the product closer
>     together.
>     > On the Cloud Enablement side of things there are ongoing
>     experiments
>     > to get a community images out.
>     >
>     > If we decided to take this direction (the CE way), our templates
>     would
>     > need to be deprecated or will change drastically. The image will
>     react
>     > on different set of variables and configuration options.
>     >
>     > Also, if we want to show the users how to use a custom XML file,
>     I
>     > don't
>     > think we should show them how to embedd it in the template as
>     JSON
>     > [2]. It's quite a pain. Instead, the XML should be kept as a
>     > separate
>     > file and the JSON file reference it.
>     >
>     > I'm still struggling to understand why this is a pain. Could you
>     > please explain it a bit more? If you look into the maintenance
>     guide
>     > [3], there are only a few steps. For me it takes no longer than
>     15
>     > minutes to do the upgrade. You also mentioned on IRC that this
>     > approach is a pain for our users (I believe you mentioned
>     something
>     > about Ray). I also can not understand why, could you please
>     explain it
>     > a bit more?
>     >
>     > [3]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>    
>     >
>     >
>     > Cheers,
>     >
>     > [1]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>    
>     >
>     > [2]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>    
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Sebastian Laskawiec
Just to follow up on this subject - a new toolkit called Cekit has been released [1] (Cekit is a replacement for Concreate). It supports ODCS repositories so it should be possible to build a community image from it. 

IMO, we should start looking at it now or after GA is released. Even though the second approach (after GA) makes much more sense, the release cycle will be much longer since then.

Thanks,

On Wed, Mar 7, 2018 at 12:14 PM Galder Zamarreño <[hidden email]> wrote:
Sebastian Laskawiec <[hidden email]> writes:

> On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Sebastian Laskawiec <[hidden email]> writes:
>
>     > Hey Galder,
>     >
>     > Comments inlined.
>     >
>     > Thanks,
>     > Seb
>     >
>     > On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño
>     <[hidden email]>
>     > wrote:
>     >
>     > Hi,
>     >
>     > Looking at [1] and I'm wondering why the templates have to
>     > maintain a
>     > different XML file for OpenShift?
>     >
>     > We already ship an XML in the server called `cloud.xml`, that
>     > should
>     > just work. Having a separate XML file in the templates means
>     we're
>     > duplicating the maintainance of XML files.
>     >
>     > Also, users can now create caches programmatically. This is by
>     far
>     > the
>     > most common tweak that had to be done to the config. So, I see
>     the
>     > urgency to change XML files less immediate.
>     >
>     > So just to give you guys a bit more context - the templates were
>     > created pretty long time ago when we didn't have admin
>     capabilities in
>     > Hot Rod and REST. The main argument for putting the whole
>     > configuration into a ConfigMap was to make configuration changes
>     > easier for the users. With ConfigMap approach they can log into
>     > OpenShift UI, go to Resources -> ConfigMaps and edit everything
>     using
>     > UI. That's super convenient for hacking in my opinion. Of
>     course, you
>     > don't need to do that at all if you don't want. You can just
>     spin up a
>     > new Infinispan cluster using `oc new-app`.
>
>     I agree with the usability of the ConfigMap. However, the
>     duplication is
>     very annoying. Would it be possible for the ConfigMap to be
>     created on
>     the fly out of the cloud.xml that's shipped by Infinispan Server?
>     That
>     way we'd still have a ConfigMap without having to duplicate XML.
>
> Probably not. This would require special permissions to call
> Kubernetes API from the Pod. In other words, I can't think about any
> other way that would work in OpenShift Online for the instance.
>
>     > There are at least two other ways for changing the configuration
>     that
>     > I can think of. The first one is S2I [1][2] (long story short,
>     you
>     > need to put your configuration into a git repository and tell
>     > OpenShift to build an image based on it). Even though it may
>     seem very
>     > convenient, it's OpenShift only solution (and there are no easy
>     (out
>     > of the box) options to get this running on raw Kubernetes). I'm
>     not
>     > judging whether it's good or bad here, just telling you how it
>     works.
>     > The other option would be to tell the users to do exactly the
>     same
>     > things we do in our templates themselves. In other words we
>     would
>     > remove configuration from the templates and provide a manual for
>     the
>     > users how to deal with configuration. I believe this is exactly
>     what
>     > Galder is suggesting, right?
>
>     What we do in the templates right now to show users how to tweak
>     their
>     config is in convoluted.
>
>     Ideally, adding their own custom configuration should be just a
>     matter
>     of:
>
>     1. Creating a ConfigMap yaml pointing to an XML.
>     2. Ask users to put their XML in a separate file pointed by the
>     ConfigMap.
>     3. Deploy ConfigMap and XML.
>     4. Trigger a new Infinispan redeployment.
>
> That would probably need to be a new deployment. Most of the
> StatefulSet spec is immutable.
>
>     Not sure how doable this is with the current template approach, or
>     we
>     could explain how to do this for an already up and running
>     application
>     that has Infinispan created out of the default template?
>
> I've been thinking about this for a while and this is what I think we
> should do:
>
> 1 Wait a couple of weeks and review the community image created by the
>   CE Team. See if this is a good fit for us. If it is, I would focus
>   on adopting this approach and adjust our templates to handle it.
> 2 Whether or not we adopt the CE community work, we could put all
>   necessary stuff into cloud.xml or services.xml configuration. We
>   could do one step forward and merge them together.
> 3 Make sure that dynamically created caches are persisted (this is
>   super important!!)
> 4 Once #3 is verified we should have a decision whether or not we are
>   adopting the CE way. At this point we could document how to use
>   custom configuration with a ConfigMap and drop it from the
>   templates.
>
> WDYT? Does this plan makes sense to you?

Sounds good

>
>     >
>     > Recently we implemented admin commands in the Hot Rod. Assuming
>     that
>     > caches created this way are not wiped out during restart (that
>     needs
>     > to be checked), we could remove the configuration from the
>     templates
>     > and tell the users to create their caches over Hot Rod and REST.
>     > However we still need to have a back door for modifying
>     configuration
>     > manually since there are some changes that can not be done via
>     admin
>     > API.
>     >
>     > [1] https://github.com/openshift/source-to-image
>     > [2]
>     >
>     https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>     
>     >
>     >
>     > Sure, there will always be people who modify/tweak things and
>     > that's
>     > fine. We should however show the people how to do that in a way
>     > that
>     > doesn't require us to duplicate our maintanence work.
>     >
>     > If we think about further maintenance, I believe we should take
>     more
>     > things into consideration. During the last planning meeting
>     Tristan
>     > mentioned about bringing the project and the product closer
>     together.
>     > On the Cloud Enablement side of things there are ongoing
>     experiments
>     > to get a community images out.
>     >
>     > If we decided to take this direction (the CE way), our templates
>     would
>     > need to be deprecated or will change drastically. The image will
>     react
>     > on different set of variables and configuration options.
>     >
>     > Also, if we want to show the users how to use a custom XML file,
>     I
>     > don't
>     > think we should show them how to embedd it in the template as
>     JSON
>     > [2]. It's quite a pain. Instead, the XML should be kept as a
>     > separate
>     > file and the JSON file reference it.
>     >
>     > I'm still struggling to understand why this is a pain. Could you
>     > please explain it a bit more? If you look into the maintenance
>     guide
>     > [3], there are only a few steps. For me it takes no longer than
>     15
>     > minutes to do the upgrade. You also mentioned on IRC that this
>     > approach is a pain for our users (I believe you mentioned
>     something
>     > about Ray). I also can not understand why, could you please
>     explain it
>     > a bit more?
>     >
>     > [3]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     >
>     > Cheers,
>     >
>     > [1]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>     
>     >
>     > [2]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Gustavo Fernandes-2
Given that the docs mention that "Cekit and Concreate are the very same tool, Concreate was rename to Cekit in 2.0 release.", does it change the outcome of the discussion in [1]?

[1] https://www.mail-archive.com/infinispan-dev@.../msg10847.html

Thanks,
Gustavo

On Mon, May 14, 2018 at 2:32 PM, Sebastian Laskawiec <[hidden email]> wrote:
Just to follow up on this subject - a new toolkit called Cekit has been released [1] (Cekit is a replacement for Concreate). It supports ODCS repositories so it should be possible to build a community image from it. 

IMO, we should start looking at it now or after GA is released. Even though the second approach (after GA) makes much more sense, the release cycle will be much longer since then.

Thanks,

On Wed, Mar 7, 2018 at 12:14 PM Galder Zamarreño <[hidden email]> wrote:
Sebastian Laskawiec <[hidden email]> writes:

> On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Sebastian Laskawiec <[hidden email]> writes:
>
>     > Hey Galder,
>     >
>     > Comments inlined.
>     >
>     > Thanks,
>     > Seb
>     >
>     > On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño
>     <[hidden email]>
>     > wrote:
>     >
>     > Hi,
>     >
>     > Looking at [1] and I'm wondering why the templates have to
>     > maintain a
>     > different XML file for OpenShift?
>     >
>     > We already ship an XML in the server called `cloud.xml`, that
>     > should
>     > just work. Having a separate XML file in the templates means
>     we're
>     > duplicating the maintainance of XML files.
>     >
>     > Also, users can now create caches programmatically. This is by
>     far
>     > the
>     > most common tweak that had to be done to the config. So, I see
>     the
>     > urgency to change XML files less immediate.
>     >
>     > So just to give you guys a bit more context - the templates were
>     > created pretty long time ago when we didn't have admin
>     capabilities in
>     > Hot Rod and REST. The main argument for putting the whole
>     > configuration into a ConfigMap was to make configuration changes
>     > easier for the users. With ConfigMap approach they can log into
>     > OpenShift UI, go to Resources -> ConfigMaps and edit everything
>     using
>     > UI. That's super convenient for hacking in my opinion. Of
>     course, you
>     > don't need to do that at all if you don't want. You can just
>     spin up a
>     > new Infinispan cluster using `oc new-app`.
>
>     I agree with the usability of the ConfigMap. However, the
>     duplication is
>     very annoying. Would it be possible for the ConfigMap to be
>     created on
>     the fly out of the cloud.xml that's shipped by Infinispan Server?
>     That
>     way we'd still have a ConfigMap without having to duplicate XML.
>
> Probably not. This would require special permissions to call
> Kubernetes API from the Pod. In other words, I can't think about any
> other way that would work in OpenShift Online for the instance.
>
>     > There are at least two other ways for changing the configuration
>     that
>     > I can think of. The first one is S2I [1][2] (long story short,
>     you
>     > need to put your configuration into a git repository and tell
>     > OpenShift to build an image based on it). Even though it may
>     seem very
>     > convenient, it's OpenShift only solution (and there are no easy
>     (out
>     > of the box) options to get this running on raw Kubernetes). I'm
>     not
>     > judging whether it's good or bad here, just telling you how it
>     works.
>     > The other option would be to tell the users to do exactly the
>     same
>     > things we do in our templates themselves. In other words we
>     would
>     > remove configuration from the templates and provide a manual for
>     the
>     > users how to deal with configuration. I believe this is exactly
>     what
>     > Galder is suggesting, right?
>
>     What we do in the templates right now to show users how to tweak
>     their
>     config is in convoluted.
>
>     Ideally, adding their own custom configuration should be just a
>     matter
>     of:
>
>     1. Creating a ConfigMap yaml pointing to an XML.
>     2. Ask users to put their XML in a separate file pointed by the
>     ConfigMap.
>     3. Deploy ConfigMap and XML.
>     4. Trigger a new Infinispan redeployment.
>
> That would probably need to be a new deployment. Most of the
> StatefulSet spec is immutable.
>
>     Not sure how doable this is with the current template approach, or
>     we
>     could explain how to do this for an already up and running
>     application
>     that has Infinispan created out of the default template?
>
> I've been thinking about this for a while and this is what I think we
> should do:
>
> 1 Wait a couple of weeks and review the community image created by the
>   CE Team. See if this is a good fit for us. If it is, I would focus
>   on adopting this approach and adjust our templates to handle it.
> 2 Whether or not we adopt the CE community work, we could put all
>   necessary stuff into cloud.xml or services.xml configuration. We
>   could do one step forward and merge them together.
> 3 Make sure that dynamically created caches are persisted (this is
>   super important!!)
> 4 Once #3 is verified we should have a decision whether or not we are
>   adopting the CE way. At this point we could document how to use
>   custom configuration with a ConfigMap and drop it from the
>   templates.
>
> WDYT? Does this plan makes sense to you?

Sounds good

>
>     >
>     > Recently we implemented admin commands in the Hot Rod. Assuming
>     that
>     > caches created this way are not wiped out during restart (that
>     needs
>     > to be checked), we could remove the configuration from the
>     templates
>     > and tell the users to create their caches over Hot Rod and REST.
>     > However we still need to have a back door for modifying
>     configuration
>     > manually since there are some changes that can not be done via
>     admin
>     > API.
>     >
>     > [1] https://github.com/openshift/source-to-image
>     > [2]
>     >
>     https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>     
>     >
>     >
>     > Sure, there will always be people who modify/tweak things and
>     > that's
>     > fine. We should however show the people how to do that in a way
>     > that
>     > doesn't require us to duplicate our maintanence work.
>     >
>     > If we think about further maintenance, I believe we should take
>     more
>     > things into consideration. During the last planning meeting
>     Tristan
>     > mentioned about bringing the project and the product closer
>     together.
>     > On the Cloud Enablement side of things there are ongoing
>     experiments
>     > to get a community images out.
>     >
>     > If we decided to take this direction (the CE way), our templates
>     would
>     > need to be deprecated or will change drastically. The image will
>     react
>     > on different set of variables and configuration options.
>     >
>     > Also, if we want to show the users how to use a custom XML file,
>     I
>     > don't
>     > think we should show them how to embedd it in the template as
>     JSON
>     > [2]. It's quite a pain. Instead, the XML should be kept as a
>     > separate
>     > file and the JSON file reference it.
>     >
>     > I'm still struggling to understand why this is a pain. Could you
>     > please explain it a bit more? If you look into the maintenance
>     guide
>     > [3], there are only a few steps. For me it takes no longer than
>     15
>     > minutes to do the upgrade. You also mentioned on IRC that this
>     > approach is a pain for our users (I believe you mentioned
>     something
>     > about Ray). I also can not understand why, could you please
>     explain it
>     > a bit more?
>     >
>     > [3]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     >
>     > Cheers,
>     >
>     > [1]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>     
>     >
>     > [2]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Maintenance of OpenShift templates

Sebastian Laskawiec
It's exactly opposite ;) But one, big thing changed after the conversation you linked had happened - we decided to bring both project and product closer together. There is no other way to do it, than using Concreate/Cekit unfortunately.

Unfortunately the alternative to this proposal means spending cycles on maintaining two, concurrent solutions - community Dockerfile approach, and product Concreate/Cekit approach.

On Mon, May 14, 2018 at 3:53 PM Gustavo Fernandes <[hidden email]> wrote:
Given that the docs mention that "Cekit and Concreate are the very same tool, Concreate was rename to Cekit in 2.0 release.", does it change the outcome of the discussion in [1]?

[1] https://www.mail-archive.com/infinispan-dev@.../msg10847.html

Thanks,
Gustavo


On Mon, May 14, 2018 at 2:32 PM, Sebastian Laskawiec <[hidden email]> wrote:
Just to follow up on this subject - a new toolkit called Cekit has been released [1] (Cekit is a replacement for Concreate). It supports ODCS repositories so it should be possible to build a community image from it. 

IMO, we should start looking at it now or after GA is released. Even though the second approach (after GA) makes much more sense, the release cycle will be much longer since then.

Thanks,

On Wed, Mar 7, 2018 at 12:14 PM Galder Zamarreño <[hidden email]> wrote:
Sebastian Laskawiec <[hidden email]> writes:

> On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <[hidden email]>
> wrote:
>
>     Sebastian Laskawiec <[hidden email]> writes:
>
>     > Hey Galder,
>     >
>     > Comments inlined.
>     >
>     > Thanks,
>     > Seb
>     >
>     > On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño
>     <[hidden email]>
>     > wrote:
>     >
>     > Hi,
>     >
>     > Looking at [1] and I'm wondering why the templates have to
>     > maintain a
>     > different XML file for OpenShift?
>     >
>     > We already ship an XML in the server called `cloud.xml`, that
>     > should
>     > just work. Having a separate XML file in the templates means
>     we're
>     > duplicating the maintainance of XML files.
>     >
>     > Also, users can now create caches programmatically. This is by
>     far
>     > the
>     > most common tweak that had to be done to the config. So, I see
>     the
>     > urgency to change XML files less immediate.
>     >
>     > So just to give you guys a bit more context - the templates were
>     > created pretty long time ago when we didn't have admin
>     capabilities in
>     > Hot Rod and REST. The main argument for putting the whole
>     > configuration into a ConfigMap was to make configuration changes
>     > easier for the users. With ConfigMap approach they can log into
>     > OpenShift UI, go to Resources -> ConfigMaps and edit everything
>     using
>     > UI. That's super convenient for hacking in my opinion. Of
>     course, you
>     > don't need to do that at all if you don't want. You can just
>     spin up a
>     > new Infinispan cluster using `oc new-app`.
>
>     I agree with the usability of the ConfigMap. However, the
>     duplication is
>     very annoying. Would it be possible for the ConfigMap to be
>     created on
>     the fly out of the cloud.xml that's shipped by Infinispan Server?
>     That
>     way we'd still have a ConfigMap without having to duplicate XML.
>
> Probably not. This would require special permissions to call
> Kubernetes API from the Pod. In other words, I can't think about any
> other way that would work in OpenShift Online for the instance.
>
>     > There are at least two other ways for changing the configuration
>     that
>     > I can think of. The first one is S2I [1][2] (long story short,
>     you
>     > need to put your configuration into a git repository and tell
>     > OpenShift to build an image based on it). Even though it may
>     seem very
>     > convenient, it's OpenShift only solution (and there are no easy
>     (out
>     > of the box) options to get this running on raw Kubernetes). I'm
>     not
>     > judging whether it's good or bad here, just telling you how it
>     works.
>     > The other option would be to tell the users to do exactly the
>     same
>     > things we do in our templates themselves. In other words we
>     would
>     > remove configuration from the templates and provide a manual for
>     the
>     > users how to deal with configuration. I believe this is exactly
>     what
>     > Galder is suggesting, right?
>
>     What we do in the templates right now to show users how to tweak
>     their
>     config is in convoluted.
>
>     Ideally, adding their own custom configuration should be just a
>     matter
>     of:
>
>     1. Creating a ConfigMap yaml pointing to an XML.
>     2. Ask users to put their XML in a separate file pointed by the
>     ConfigMap.
>     3. Deploy ConfigMap and XML.
>     4. Trigger a new Infinispan redeployment.
>
> That would probably need to be a new deployment. Most of the
> StatefulSet spec is immutable.
>
>     Not sure how doable this is with the current template approach, or
>     we
>     could explain how to do this for an already up and running
>     application
>     that has Infinispan created out of the default template?
>
> I've been thinking about this for a while and this is what I think we
> should do:
>
> 1 Wait a couple of weeks and review the community image created by the
>   CE Team. See if this is a good fit for us. If it is, I would focus
>   on adopting this approach and adjust our templates to handle it.
> 2 Whether or not we adopt the CE community work, we could put all
>   necessary stuff into cloud.xml or services.xml configuration. We
>   could do one step forward and merge them together.
> 3 Make sure that dynamically created caches are persisted (this is
>   super important!!)
> 4 Once #3 is verified we should have a decision whether or not we are
>   adopting the CE way. At this point we could document how to use
>   custom configuration with a ConfigMap and drop it from the
>   templates.
>
> WDYT? Does this plan makes sense to you?

Sounds good

>
>     >
>     > Recently we implemented admin commands in the Hot Rod. Assuming
>     that
>     > caches created this way are not wiped out during restart (that
>     needs
>     > to be checked), we could remove the configuration from the
>     templates
>     > and tell the users to create their caches over Hot Rod and REST.
>     > However we still need to have a back door for modifying
>     configuration
>     > manually since there are some changes that can not be done via
>     admin
>     > API.
>     >
>     > [1] https://github.com/openshift/source-to-image
>     > [2]
>     >
>     https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble
>     
>     >
>     >
>     > Sure, there will always be people who modify/tweak things and
>     > that's
>     > fine. We should however show the people how to do that in a way
>     > that
>     > doesn't require us to duplicate our maintanence work.
>     >
>     > If we think about further maintenance, I believe we should take
>     more
>     > things into consideration. During the last planning meeting
>     Tristan
>     > mentioned about bringing the project and the product closer
>     together.
>     > On the Cloud Enablement side of things there are ongoing
>     experiments
>     > to get a community images out.
>     >
>     > If we decided to take this direction (the CE way), our templates
>     would
>     > need to be deprecated or will change drastically. The image will
>     react
>     > on different set of variables and configuration options.
>     >
>     > Also, if we want to show the users how to use a custom XML file,
>     I
>     > don't
>     > think we should show them how to embedd it in the template as
>     JSON
>     > [2]. It's quite a pain. Instead, the XML should be kept as a
>     > separate
>     > file and the JSON file reference it.
>     >
>     > I'm still struggling to understand why this is a pain. Could you
>     > please explain it a bit more? If you look into the maintenance
>     guide
>     > [3], there are only a few steps. For me it takes no longer than
>     15
>     > minutes to do the upgrade. You also mentioned on IRC that this
>     > approach is a pain for our users (I believe you mentioned
>     something
>     > about Ray). I also can not understand why, could you please
>     explain it
>     > a bit more?
>     >
>     > [3]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     >
>     > Cheers,
>     >
>     > [1]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates/pull/16/files
>     
>     >
>     > [2]
>     >
>     https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide
>     
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > [hidden email]
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev