[infinispan-dev] Auto-detection of appropriate key2StringMapperClass

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Sanne Grinovero-3
It seems I keep forgetting that the "key2StringMapperClass" needs to
be explicitly set when using the String based JDBC CacheStore, which I
think is a quite popular store.

http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database

I think we could remove the need to configure this with an appropriate
service discovery?

We could keep the property so to be able to override the discovery process.

Would anyone volunteer to implement this?

Thanks,
Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Dan Berindei
Every store can have its own Key2StringMapper, so I really don't think
service discovery is appropriate here.

My favourite solution would be to have a default Key2StringMapper
implementation that uses Java serialization (not our internal
marshalling, because that can change between versions). It could even
do base64 encoding for databases that don't have a VARBINARY/BYTEA
equivalent.

Cheers
Dan


On Wed, Mar 8, 2017 at 6:34 PM, Sanne Grinovero <[hidden email]> wrote:

> It seems I keep forgetting that the "key2StringMapperClass" needs to
> be explicitly set when using the String based JDBC CacheStore, which I
> think is a quite popular store.
>
> http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database
>
> I think we could remove the need to configure this with an appropriate
> service discovery?
>
> We could keep the property so to be able to override the discovery process.
>
> Would anyone volunteer to implement this?
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Sanne Grinovero-3
Hi Dan!

On 9 March 2017 at 09:38, Dan Berindei <[hidden email]> wrote:
> Every store can have its own Key2StringMapper, so I really don't think
> service discovery is appropriate here.

I suggested a default, while maintaining the property as an override option.

It won't make any difference to people in the situation you describe,
but it will avoid trouble for many more users.

In my experience when one develops a custom type & a custom
Key2StringMapper, you really wouldn't have interest in creating
multiple implementations for the same type. The only case in which I
can see some use for allowing multiple conflicting registrations to
the same type is for upgrade / migration cases..

>
> My favourite solution would be to have a default Key2StringMapper
> implementation that uses Java serialization (not our internal
> marshalling, because that can change between versions). It could even
> do base64 encoding for databases that don't have a VARBINARY/BYTEA
> equivalent.
>
> Cheers
> Dan
>
>
> On Wed, Mar 8, 2017 at 6:34 PM, Sanne Grinovero <[hidden email]> wrote:
>> It seems I keep forgetting that the "key2StringMapperClass" needs to
>> be explicitly set when using the String based JDBC CacheStore, which I
>> think is a quite popular store.
>>
>> http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database
>>
>> I think we could remove the need to configure this with an appropriate
>> service discovery?
>>
>> We could keep the property so to be able to override the discovery process.
>>
>> Would anyone volunteer to implement this?
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Dan Berindei
On Thu, Mar 9, 2017 at 11:48 AM, Sanne Grinovero <[hidden email]> wrote:
> Hi Dan!
>

Hey Sanne :)

> On 9 March 2017 at 09:38, Dan Berindei <[hidden email]> wrote:
>> Every store can have its own Key2StringMapper, so I really don't think
>> service discovery is appropriate here.
>
> I suggested a default, while maintaining the property as an override option.
>
> It won't make any difference to people in the situation you describe,
> but it will avoid trouble for many more users.
>
> In my experience when one develops a custom type & a custom
> Key2StringMapper, you really wouldn't have interest in creating
> multiple implementations for the same type. The only case in which I
> can see some use for allowing multiple conflicting registrations to
> the same type is for upgrade / migration cases..
>

The problem I see is that each store can only have one
Key2StringMapper implementation, so you're going to have conflicts
even if your caches use completely different key types. The store
doesn't know which key types the application actually uses, so it
can't pick the appropriate implementation when there are multiple ones
on the classpath.

>>
>> My favourite solution would be to have a default Key2StringMapper
>> implementation that uses Java serialization (not our internal
>> marshalling, because that can change between versions). It could even
>> do base64 encoding for databases that don't have a VARBINARY/BYTEA
>> equivalent.
>>

Do you see any problem with using serialization everywhere?

Dan


>> Cheers
>> Dan
>>
>>
>> On Wed, Mar 8, 2017 at 6:34 PM, Sanne Grinovero <[hidden email]> wrote:
>>> It seems I keep forgetting that the "key2StringMapperClass" needs to
>>> be explicitly set when using the String based JDBC CacheStore, which I
>>> think is a quite popular store.
>>>
>>> http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database
>>>
>>> I think we could remove the need to configure this with an appropriate
>>> service discovery?
>>>
>>> We could keep the property so to be able to override the discovery process.
>>>
>>> Would anyone volunteer to implement this?
>>>
>>> Thanks,
>>> Sanne
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Sanne Grinovero-3
On 9 March 2017 at 10:22, Dan Berindei <[hidden email]> wrote:

> On Thu, Mar 9, 2017 at 11:48 AM, Sanne Grinovero <[hidden email]> wrote:
>> Hi Dan!
>>
>
> Hey Sanne :)
>
>> On 9 March 2017 at 09:38, Dan Berindei <[hidden email]> wrote:
>>> Every store can have its own Key2StringMapper, so I really don't think
>>> service discovery is appropriate here.
>>
>> I suggested a default, while maintaining the property as an override option.
>>
>> It won't make any difference to people in the situation you describe,
>> but it will avoid trouble for many more users.
>>
>> In my experience when one develops a custom type & a custom
>> Key2StringMapper, you really wouldn't have interest in creating
>> multiple implementations for the same type. The only case in which I
>> can see some use for allowing multiple conflicting registrations to
>> the same type is for upgrade / migration cases..
>>
>
> The problem I see is that each store can only have one
> Key2StringMapper implementation, so you're going to have conflicts
> even if your caches use completely different key types. The store
> doesn't know which key types the application actually uses, so it
> can't pick the appropriate implementation when there are multiple ones
> on the classpath.

Good point. I was hoping to allow different services to "contribute"
their different converters but I realise now that it would introduce a
risk of ambiguous representations.
They would need some standard scoping to know to which one to delegate
to on read operations. I wonder if we could enforce / validate that?

>>> My favourite solution would be to have a default Key2StringMapper
>>> implementation that uses Java serialization (not our internal
>>> marshalling, because that can change between versions). It could even
>>> do base64 encoding for databases that don't have a VARBINARY/BYTEA
>>> equivalent.
>>>
>
> Do you see any problem with using serialization everywhere?

I guess we could discuss alternative strategies, but it's an
orthogonal discussion right? You're proposing a different CacheStore
implementation, but people would still need a better way to associate
the adaptors to the types.

But to answer your question: yes I wouldn't like using Java
serialization: people need to be able to evolve their domain model.
When we had to "break" the string format in the Lucene mappers (as I
added a new field) we managed to do it in an evolutionary way: it
wasn't too hard to make sure we could read the old string format,
while with Java serialization this would have been a mess.

Also it would be nice to keep the representation as short as possible
for both efficient processing and storage space efficiency: Java
Serialization isn't particularly compact.

Thanks,
Sanne

>
> Dan
>
>
>>> Cheers
>>> Dan
>>>
>>>
>>> On Wed, Mar 8, 2017 at 6:34 PM, Sanne Grinovero <[hidden email]> wrote:
>>>> It seems I keep forgetting that the "key2StringMapperClass" needs to
>>>> be explicitly set when using the String based JDBC CacheStore, which I
>>>> think is a quite popular store.
>>>>
>>>> http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database
>>>>
>>>> I think we could remove the need to configure this with an appropriate
>>>> service discovery?
>>>>
>>>> We could keep the property so to be able to override the discovery process.
>>>>
>>>> Would anyone volunteer to implement this?
>>>>
>>>> Thanks,
>>>> Sanne
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Dan Berindei
On Thu, Mar 9, 2017 at 1:09 PM, Sanne Grinovero <[hidden email]> wrote:

> On 9 March 2017 at 10:22, Dan Berindei <[hidden email]> wrote:
>> On Thu, Mar 9, 2017 at 11:48 AM, Sanne Grinovero <[hidden email]> wrote:
>>> Hi Dan!
>>>
>>
>> Hey Sanne :)
>>
>>> On 9 March 2017 at 09:38, Dan Berindei <[hidden email]> wrote:
>>>> Every store can have its own Key2StringMapper, so I really don't think
>>>> service discovery is appropriate here.
>>>
>>> I suggested a default, while maintaining the property as an override option.
>>>
>>> It won't make any difference to people in the situation you describe,
>>> but it will avoid trouble for many more users.
>>>
>>> In my experience when one develops a custom type & a custom
>>> Key2StringMapper, you really wouldn't have interest in creating
>>> multiple implementations for the same type. The only case in which I
>>> can see some use for allowing multiple conflicting registrations to
>>> the same type is for upgrade / migration cases..
>>>
>>
>> The problem I see is that each store can only have one
>> Key2StringMapper implementation, so you're going to have conflicts
>> even if your caches use completely different key types. The store
>> doesn't know which key types the application actually uses, so it
>> can't pick the appropriate implementation when there are multiple ones
>> on the classpath.
>
> Good point. I was hoping to allow different services to "contribute"
> their different converters but I realise now that it would introduce a
> risk of ambiguous representations.
> They would need some standard scoping to know to which one to delegate
> to on read operations. I wonder if we could enforce / validate that?
>

We could prefix the string representation with the class and module
name of the mapper ;)

>>>> My favourite solution would be to have a default Key2StringMapper
>>>> implementation that uses Java serialization (not our internal
>>>> marshalling, because that can change between versions). It could even
>>>> do base64 encoding for databases that don't have a VARBINARY/BYTEA
>>>> equivalent.
>>>>
>>
>> Do you see any problem with using serialization everywhere?
>
> I guess we could discuss alternative strategies, but it's an
> orthogonal discussion right? You're proposing a different CacheStore
> implementation, but people would still need a better way to associate
> the adaptors to the types.
>

I don't think it's orthogonal, because having a single key mapper that
supports all key types would make this discussion unnecessary.

> But to answer your question: yes I wouldn't like using Java
> serialization: people need to be able to evolve their domain model.
> When we had to "break" the string format in the Lucene mappers (as I
> added a new field) we managed to do it in an evolutionary way: it
> wasn't too hard to make sure we could read the old string format,
> while with Java serialization this would have been a mess.
>

If your keys are internal it should be possible to use a new key type
and to keep supporting the old type for a while. But yeah, it's a
little more involved than adding support for a new prefix in your key
mapper.

> Also it would be nice to keep the representation as short as possible
> for both efficient processing and storage space efficiency: Java
> Serialization isn't particularly compact.
>

Yeah, I should have just said we should use the same marshalling for
the keys that we use for the values.

I didn't want to bring up our marshalling because we don't have a very
good record for preserving the format of JDK types that get included
in the user types, but Java serialization has its own warts. In the
end, the important point is that what's good for the values must be
good for the keys too.

Cheers
Dan
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Auto-detection of appropriate key2StringMapperClass

Paul Ferraro
In reply to this post by Sanne Grinovero-3
Funny that you mention this, because I need to implement this for
WildFly.next to support Infinispan 9, where the binary key store is no
longer an option.  We need to be able to map complex keys for storing
web sessions, Hibernate keys, and @Stateful EJBs to strings.
An auto-discovered mapper is simple enough to integrate into WF since
we allow a given cache-container to be configured with a specific
module, from which marshalling services are auto-discovered.

Paul

On Wed, Mar 8, 2017 at 11:34 AM, Sanne Grinovero <[hidden email]> wrote:

> It seems I keep forgetting that the "key2StringMapperClass" needs to
> be explicitly set when using the String based JDBC CacheStore, which I
> think is a quite popular store.
>
> http://infinispan.org/docs/dev/user_guide/user_guide.html#storing_the_index_in_a_database
>
> I think we could remove the need to configure this with an appropriate
> service discovery?
>
> We could keep the property so to be able to override the discovery process.
>
> Would anyone volunteer to implement this?
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev