[infinispan-dev] Quick fix for ISPN-7710

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Quick fix for ISPN-7710

Galder Zamarreño
Hi all,

I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.

One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.

So, in essence, adding the following to org/infinispan/main/module.xml:

      <module name="org.infinispan.remote-query.server" optional="true"/>

Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.

Cheers,

[1] https://github.com/infinispan-demos/swiss-transport-datagrid
[2] https://issues.jboss.org/browse/ISPN-7710
--
Galder Zamarreño
Infinispan, Red Hat


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Quick fix for ISPN-7710

Adrian Nistor
Hi Galder,

this fix is acceptable for now as it quickly enables users to use
CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the
long run we would want users to be able to specify a custom marshaller
class that comes from a user supplied module or even a deployment - the
general case.

With the introduction of encoders and deprecation of compat mode we
still have the same class loading issue in the general case. So I
propose to refine a bit our approach and instead of specifying just a
class name we should use a naming schema like "moduleId:slot:className",
giving users the ability to specify a class that comes from a different
module or deployment. I'm currently experimenting with this. I'll come
back with results soon.

There are also other code bits that need to be deployed in the server
ASAP: protostream entity marshallers, lucene analyzers. I'm thinking
these could all benefit from the same solution.

Btw, what is the relation between ISPN-7814 and ISPN-7710 ?

Adrian

On 06/14/2017 06:35 PM, Galder Zamarreño wrote:

> Hi all,
>
> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>
> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>
> So, in essence, adding the following to org/infinispan/main/module.xml:
>
>        <module name="org.infinispan.remote-query.server" optional="true"/>
>
> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>
> Cheers,
>
> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
> [2] https://issues.jboss.org/browse/ISPN-7710
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Important feedback for transcoding work - Re: Quick fix for ISPN-7710

Galder Zamarreño
@Gustavo, some important info for your transcoding work below:

--
Galder Zamarreño
Infinispan, Red Hat

> On 15 Jun 2017, at 11:05, Adrian Nistor <[hidden email]> wrote:
>
> Hi Galder,
>
> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
>
> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
>
> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.

I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:

https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java

In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.

However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...

That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...

The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.

> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?

The relationship between them is explained here:

https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image

I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.

Cheers,

>
> Adrian
>
> On 06/14/2017 06:35 PM, Galder Zamarreño wrote:
>> Hi all,
>>
>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>
>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>
>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>
>>       <module name="org.infinispan.remote-query.server" optional="true"/>
>>
>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>
>> Cheers,
>>
>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>> [2] https://issues.jboss.org/browse/ISPN-7710
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Quick fix for ISPN-7710

Sanne Grinovero-3
In reply to this post by Adrian Nistor
On 15 June 2017 at 10:05, Adrian Nistor <[hidden email]> wrote:

> Hi Galder,
>
> this fix is acceptable for now as it quickly enables users to use
> CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the
> long run we would want users to be able to specify a custom marshaller
> class that comes from a user supplied module or even a deployment - the
> general case.
>
> With the introduction of encoders and deprecation of compat mode we
> still have the same class loading issue in the general case. So I
> propose to refine a bit our approach and instead of specifying just a
> class name we should use a naming schema like "moduleId:slot:className",
> giving users the ability to specify a class that comes from a different
> module or deployment. I'm currently experimenting with this. I'll come
> back with results soon.

Great idea!
Can you also figure out how to identify loading something from the
user deployment ?
Maybe we could apply a simple convention, as in when the module
identification is missing then we look into TCCL, while when it's
provided we look for the specific module (exclusively) ?

Thanks,
Sanne

>
> There are also other code bits that need to be deployed in the server
> ASAP: protostream entity marshallers, lucene analyzers. I'm thinking
> these could all benefit from the same solution.
>
> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>
> Adrian
>
> On 06/14/2017 06:35 PM, Galder Zamarreño wrote:
>> Hi all,
>>
>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>
>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>
>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>
>>        <module name="org.infinispan.remote-query.server" optional="true"/>
>>
>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>
>> Cheers,
>>
>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>> [2] https://issues.jboss.org/browse/ISPN-7710
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Quick fix for ISPN-7710

Adrian Nistor
Yup. That's exactly what I'm proposing. But today I stopped working on
it because of some PR reviewing for the release. I hope to restart it
and have a PoC by tomorrow.

On 06/15/2017 04:06 PM, Sanne Grinovero wrote:

> On 15 June 2017 at 10:05, Adrian Nistor <[hidden email]> wrote:
>> Hi Galder,
>>
>> this fix is acceptable for now as it quickly enables users to use
>> CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the
>> long run we would want users to be able to specify a custom marshaller
>> class that comes from a user supplied module or even a deployment - the
>> general case.
>>
>> With the introduction of encoders and deprecation of compat mode we
>> still have the same class loading issue in the general case. So I
>> propose to refine a bit our approach and instead of specifying just a
>> class name we should use a naming schema like "moduleId:slot:className",
>> giving users the ability to specify a class that comes from a different
>> module or deployment. I'm currently experimenting with this. I'll come
>> back with results soon.
> Great idea!
> Can you also figure out how to identify loading something from the
> user deployment ?
> Maybe we could apply a simple convention, as in when the module
> identification is missing then we look into TCCL, while when it's
> provided we look for the specific module (exclusively) ?
>
> Thanks,
> Sanne
>
>> There are also other code bits that need to be deployed in the server
>> ASAP: protostream entity marshallers, lucene analyzers. I'm thinking
>> these could all benefit from the same solution.
>>
>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>>
>> Adrian
>>
>> On 06/14/2017 06:35 PM, Galder Zamarreño wrote:
>>> Hi all,
>>>
>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>
>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>
>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>
>>>         <module name="org.infinispan.remote-query.server" optional="true"/>
>>>
>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>> --
>>> Galder Zamarreño
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Important feedback for transcoding work - Re: Quick fix for ISPN-7710

Adrian Nistor
In reply to this post by Galder Zamarreño
Galder, I've seen AddProtobufTask in March or April when you mentioned
this issue on the devlist; that approach can work for protostream
marshallers or any other code bits that the Cache does not depend on
during startup, and which can be deployed anytime later. In this
category we currently have : filters, converters. These are currently
deployed with the help of a DeploymentUnitProcessor, but we could have
done it using a ServerTask as well. Now that we took the route of DUP, I
think we should continue in a consistent manner and use it for other
'deployables' we identify from now on, ie. the protobuf entity marshallers.

As for encoders, lucene analyzers, compatibility marshaller, event
marshaller - these are all needed during cache startup. We need to come
up with something for these, so I propose to look them up using the
"moduleId:slot:className" convention.


On 06/15/2017 03:40 PM, Galder Zamarreño wrote:

> @Gustavo, some important info for your transcoding work below:
>
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 15 Jun 2017, at 11:05, Adrian Nistor <[hidden email]> wrote:
>>
>> Hi Galder,
>>
>> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
>>
>> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
>>
>> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.
> I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:
>
> https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java
>
> In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.
>
> However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...
>
> That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...
>
> The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.
>
>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
> The relationship between them is explained here:
>
> https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image
>
> I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.
>
> Cheers,
>
>> Adrian
>>
>> On 06/14/2017 06:35 PM, Galder Zamarreño wrote:
>>> Hi all,
>>>
>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>
>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>
>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>
>>>        <module name="org.infinispan.remote-query.server" optional="true"/>
>>>
>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>> --
>>> Galder Zamarreño
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Important feedback for transcoding work - Re: Quick fix for ISPN-7710

Galder Zamarreño

--
Galder Zamarreño
Infinispan, Red Hat

> On 15 Jun 2017, at 15:25, Adrian Nistor <[hidden email]> wrote:
>
> Galder, I've seen AddProtobufTask in March or April when you mentioned this issue on the devlist; that approach can work for protostream marshallers or any other code bits that the Cache does not depend on during startup, and which can be deployed anytime later. In this category we currently have : filters, converters. These are currently deployed with the help of a DeploymentUnitProcessor, but we could have done it using a ServerTask as well.

^ I'm not sure we had ServerTasks in place when we had filters and converters... But if we had server tasks then, we should have used that path. My bad if we didn't do it :\

> Now that we took the route of DUP, I think we should continue in a consistent manner and use it for other 'deployables' we identify from now on, ie. the protobuf entity marshallers.

^ Having written DUPs, I consider them to be a royal PITA. So, I'm against that.

> As for encoders, lucene analyzers, compatibility marshaller, event marshaller - these are all needed during cache startup. We need to come up with something for these, so I propose to look them up using the "moduleId:slot:className" convention.

As I see it, encoders/compatibility-marshaller/event-marshaller might not be needed on startup. If data is kept in binary and only deserialized lazily when needed, you only need them when you're going to do what you need...

To be more precise, yday Gustavo and I had a discussion and you'd need maybe pre-registering the encoders when a server tasks is deployed so that the server task, when executed in other nodes, can find the encoder. E.g. this is some pseudo code:

on ServerTask initialization:
Encoder encoder = new ProtoStreamEncoder(customMarshallers)
cache.registerEncoder("blah", encoder);

on ServerTask call:
cache.withEncoder("blah").values().stream()....

^ Btw, I know that's not how withEncoder() works... This would be much flexible to use than forcing encoders to be defined on startup. The server task deployment could ship the marshallers for POJOs since it already has to ship the POJOs to be able to use them.

I'd like to hear which use cases would really require those components you mention to be available on startup... assuming we don't by default transform data into their deserialized format and we keep data in binary format instead...

Cheers,

>
>
> On 06/15/2017 03:40 PM, Galder Zamarreño wrote:
>> @Gustavo, some important info for your transcoding work below:
>>
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>> On 15 Jun 2017, at 11:05, Adrian Nistor <[hidden email]> wrote:
>>>
>>> Hi Galder,
>>>
>>> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
>>>
>>> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
>>>
>>> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.
>> I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:
>>
>> https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java
>>
>> In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.
>>
>> However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...
>>
>> That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...
>>
>> The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.
>>
>>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>> The relationship between them is explained here:
>>
>> https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image
>>
>> I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.
>>
>> Cheers,
>>
>>> Adrian
>>>
>>> On 06/14/2017 06:35 PM, Galder Zamarreño wrote:
>>>> Hi all,
>>>>
>>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>>
>>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>>
>>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>>
>>>>       <module name="org.infinispan.remote-query.server" optional="true"/>
>>>>
>>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>>
>>>> Cheers,
>>>>
>>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>>> --
>>>> Galder Zamarreño
>>>> Infinispan, Red Hat
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Important feedback for transcoding work - Re: Quick fix for ISPN-7710

Dan Berindei
On Fri, Jun 16, 2017 at 1:07 PM, Galder Zamarreño <[hidden email]> wrote:

>
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 15 Jun 2017, at 15:25, Adrian Nistor <[hidden email]> wrote:
>>
>> Galder, I've seen AddProtobufTask in March or April when you mentioned this issue on the devlist; that approach can work for protostream marshallers or any other code bits that the Cache does not depend on during startup, and which can be deployed anytime later. In this category we currently have : filters, converters. These are currently deployed with the help of a DeploymentUnitProcessor, but we could have done it using a ServerTask as well.
>
> ^ I'm not sure we had ServerTasks in place when we had filters and converters... But if we had server tasks then, we should have used that path. My bad if we didn't do it :\
>
>> Now that we took the route of DUP, I think we should continue in a consistent manner and use it for other 'deployables' we identify from now on, ie. the protobuf entity marshallers.
>
> ^ Having written DUPs, I consider them to be a royal PITA. So, I'm against that.
>
>> As for encoders, lucene analyzers, compatibility marshaller, event marshaller - these are all needed during cache startup. We need to come up with something for these, so I propose to look them up using the "moduleId:slot:className" convention.
>
> As I see it, encoders/compatibility-marshaller/event-marshaller might not be needed on startup. If data is kept in binary and only deserialized lazily when needed, you only need them when you're going to do what you need...
>

What if you start a node and a client immediately tries to register an
even listener?

Not sure about the others, but for the lucene analyzers, I assume some
configurations will have to analyze/index entries that we receive via
state transfer during startup.

Dan

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Important feedback for transcoding work - Re: Quick fix for ISPN-7710

Galder Zamarreño

--
Galder Zamarreño
Infinispan, Red Hat

> On 19 Jun 2017, at 13:17, Dan Berindei <[hidden email]> wrote:
>
> On Fri, Jun 16, 2017 at 1:07 PM, Galder Zamarreño <[hidden email]> wrote:
>>
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>> On 15 Jun 2017, at 15:25, Adrian Nistor <[hidden email]> wrote:
>>>
>>> Galder, I've seen AddProtobufTask in March or April when you mentioned this issue on the devlist; that approach can work for protostream marshallers or any other code bits that the Cache does not depend on during startup, and which can be deployed anytime later. In this category we currently have : filters, converters. These are currently deployed with the help of a DeploymentUnitProcessor, but we could have done it using a ServerTask as well.
>>
>> ^ I'm not sure we had ServerTasks in place when we had filters and converters... But if we had server tasks then, we should have used that path. My bad if we didn't do it :\
>>
>>> Now that we took the route of DUP, I think we should continue in a consistent manner and use it for other 'deployables' we identify from now on, ie. the protobuf entity marshallers.
>>
>> ^ Having written DUPs, I consider them to be a royal PITA. So, I'm against that.
>>
>>> As for encoders, lucene analyzers, compatibility marshaller, event marshaller - these are all needed during cache startup. We need to come up with something for these, so I propose to look them up using the "moduleId:slot:className" convention.
>>
>> As I see it, encoders/compatibility-marshaller/event-marshaller might not be needed on startup. If data is kept in binary and only deserialized lazily when needed, you only need them when you're going to do what you need...
>>
>
> What if you start a node and a client immediately tries to register an
> even listener?

If the event listener server side requires any deserialization, I'd expect the node on startup to have a way to load the encoder to be used, either via config or a server tasks that's deployed by the user or pre-registered by the server.

>
> Not sure about the others, but for the lucene analyzers, I assume some
> configurations will have to analyze/index entries that we receive via
> state transfer during startup.

Good point. This is a use case where unmarshalling/deserialization/decoding would be required on startup, to be able to index data.

>
> Dan
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...