[infinispan-dev] Missing Externalizers for the Lucene Query classes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Missing Externalizers for the Lucene Query classes

Sanne Grinovero-3
Following up on today's meeting minutes.

Galder asked if Hibernate Search was going to provide externalizers
for the Lucene Query classes; let me clarify that I don't think that
those belong in the Hibernate Search code base, not least we hope to
avoid ever needing to implement them.

A soft reason to not have them in Hibernate Search is that this
project never needs to serialise any Query; this is a requirement of
Infinispan Query only, needed to implement Infinispan specific
extensions to the query engine.
Although, these are very nice extensions so I'd not like to see them
dropped: I'd hope that Infinispan could work around the lack of proper
externalizers for the moment, as it has always been able to do so far.

A stronger reason is that this would introduce circular dependencies
between the two projects, and a big overhead of release coordination:
we had this in the past, very all very glad this is in the past!

When we'll have IQL, this will both define a good "on the wire"
representation which would solve the serialization problem, and IQL
will also limit the amount of Query types which we will need to
support, as at that point we will be able to limit the support for
Clustered Queries (which is the feature needing to serialize the
queries) to those which IQL can express, and thus serialize.

At that point we'll be able to deprecate the the Clustered Query API
which accepts a user instance of the Lucene Query, and only run
clustered queries for queries expressed over IQL. Not least we'll be
able to automatically determine if the query is best run as a
clustered query or as a local query, removing this complexity from the
user's responsibilities.

In conclusion, we'll still be using the "Clustered Query"
functionality, but not exposing it, and by doing so we won't need any
externalizer. But for now please keep the tests running with the
existing externalizer strategies: we just need to keep it functional,
but there's no need to optimise performance of these externalizers as
we'll get rid of them.

Thanks,
Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Missing Externalizers for the Lucene Query classes

Galder Zamarreño

--
Galder Zamarreño
Infinispan, Red Hat

> On 31 Oct 2016, at 19:52, Sanne Grinovero <[hidden email]> wrote:
>
> Following up on today's meeting minutes.
>
> Galder asked if Hibernate Search was going to provide externalizers
> for the Lucene Query classes; let me clarify that I don't think that
> those belong in the Hibernate Search code base, not least we hope to
> avoid ever needing to implement them.
>
> A soft reason to not have them in Hibernate Search is that this
> project never needs to serialise any Query; this is a requirement of
> Infinispan Query only, needed to implement Infinispan specific
> extensions to the query engine.
> Although, these are very nice extensions so I'd not like to see them
> dropped: I'd hope that Infinispan could work around the lack of proper
> externalizers for the moment, as it has always been able to do so far.

^ We can only workaround it because the external marshaller right now relies on JBoss Marshalling, which enables us to plug in at the object table level even for Serializable objects and we can lookup externalizers.

I know for a fact we can't plug like that if using standard serialization, and there's no guarantee that we'll be able to support this use case with another marshaller.

So, we should really be avoiding such edge cases. I don't mind where the externalizers are placed.

>
> A stronger reason is that this would introduce circular dependencies
> between the two projects, and a big overhead of release coordination:
> we had this in the past, very all very glad this is in the past!
>
> When we'll have IQL, this will both define a good "on the wire"
> representation which would solve the serialization problem, and IQL
> will also limit the amount of Query types which we will need to
> support, as at that point we will be able to limit the support for
> Clustered Queries (which is the feature needing to serialize the
> queries) to those which IQL can express, and thus serialize.
>
> At that point we'll be able to deprecate the the Clustered Query API
> which accepts a user instance of the Lucene Query, and only run
> clustered queries for queries expressed over IQL. Not least we'll be
> able to automatically determine if the query is best run as a
> clustered query or as a local query, removing this complexity from the
> user's responsibilities.
>
> In conclusion, we'll still be using the "Clustered Query"
> functionality, but not exposing it, and by doing so we won't need any
> externalizer. But for now please keep the tests running with the
> existing externalizer strategies: we just need to keep it functional,
> but there's no need to optimise performance of these externalizers as
> we'll get rid of them.
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Missing Externalizers for the Lucene Query classes

Sanne Grinovero-3
On 14 November 2016 at 16:05, Galder Zamarreño <[hidden email]> wrote:

>
>
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
> > On 31 Oct 2016, at 19:52, Sanne Grinovero <[hidden email]> wrote:
> >
> > Following up on today's meeting minutes.
> >
> > Galder asked if Hibernate Search was going to provide externalizers
> > for the Lucene Query classes; let me clarify that I don't think that
> > those belong in the Hibernate Search code base, not least we hope to
> > avoid ever needing to implement them.
> >
> > A soft reason to not have them in Hibernate Search is that this
> > project never needs to serialise any Query; this is a requirement of
> > Infinispan Query only, needed to implement Infinispan specific
> > extensions to the query engine.
> > Although, these are very nice extensions so I'd not like to see them
> > dropped: I'd hope that Infinispan could work around the lack of proper
> > externalizers for the moment, as it has always been able to do so far.
>
> ^ We can only workaround it because the external marshaller right now relies on JBoss Marshalling, which enables us to plug in at the object table level even for Serializable objects and we can lookup externalizers.
>
> I know for a fact we can't plug like that if using standard serialization, and there's no guarantee that we'll be able to support this use case with another marshaller.
>
> So, we should really be avoiding such edge cases. I don't mind where the externalizers are placed.


Hi Galder,

sorry but I don't understand if you're asking us to do something, and
what is the proposal?

I'd strongly prefer to keep them as-is for now, and refactor this as
needed in the near future when we'll change the Clustered Query API
and take advantage of ICKLE.

Thanks,
Sanne

>
>
> >
> > A stronger reason is that this would introduce circular dependencies
> > between the two projects, and a big overhead of release coordination:
> > we had this in the past, very all very glad this is in the past!
> >
> > When we'll have IQL, this will both define a good "on the wire"
> > representation which would solve the serialization problem, and IQL
> > will also limit the amount of Query types which we will need to
> > support, as at that point we will be able to limit the support for
> > Clustered Queries (which is the feature needing to serialize the
> > queries) to those which IQL can express, and thus serialize.
> >
> > At that point we'll be able to deprecate the the Clustered Query API
> > which accepts a user instance of the Lucene Query, and only run
> > clustered queries for queries expressed over IQL. Not least we'll be
> > able to automatically determine if the query is best run as a
> > clustered query or as a local query, removing this complexity from the
> > user's responsibilities.
> >
> > In conclusion, we'll still be using the "Clustered Query"
> > functionality, but not exposing it, and by doing so we won't need any
> > externalizer. But for now please keep the tests running with the
> > existing externalizer strategies: we just need to keep it functional,
> > but there's no need to optimise performance of these externalizers as
> > we'll get rid of them.
> >
> > Thanks,
> > Sanne
> > _______________________________________________
> > infinispan-dev mailing list
> > [hidden email]
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Missing Externalizers for the Lucene Query classes

Gustavo Fernandes-2
On Tue, Nov 15, 2016 at 11:48 AM, Sanne Grinovero <[hidden email]> wrote:
On 14 November 2016 at 16:05, Galder Zamarreño <[hidden email]> wrote:
>
>
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
> > On 31 Oct 2016, at 19:52, Sanne Grinovero <[hidden email]> wrote:
> >
> > Following up on today's meeting minutes.
> >
> > Galder asked if Hibernate Search was going to provide externalizers
> > for the Lucene Query classes; let me clarify that I don't think that
> > those belong in the Hibernate Search code base, not least we hope to
> > avoid ever needing to implement them.
> >
> > A soft reason to not have them in Hibernate Search is that this
> > project never needs to serialise any Query; this is a requirement of
> > Infinispan Query only, needed to implement Infinispan specific
> > extensions to the query engine.
> > Although, these are very nice extensions so I'd not like to see them
> > dropped: I'd hope that Infinispan could work around the lack of proper
> > externalizers for the moment, as it has always been able to do so far.
>
> ^ We can only workaround it because the external marshaller right now relies on JBoss Marshalling, which enables us to plug in at the object table level even for Serializable objects and we can lookup externalizers.
>
> I know for a fact we can't plug like that if using standard serialization, and there's no guarantee that we'll be able to support this use case with another marshaller.
>
> So, we should really be avoiding such edge cases. I don't mind where the externalizers are placed.


Hi Galder,

sorry but I don't understand if you're asking us to do something, and
what is the proposal?


As I pointed out before, the short term solution is [1]: we will add an Externalizer *on Infinispan* for the HSearch
classes that don't have one and are affected by Clustered Queries. This should remove the edge case altogether and no change is
required on Hibernate Search side
Thanks,
Gustavo
 

I'd strongly prefer to keep them as-is for now, and refactor this as
needed in the near future when we'll change the Clustered Query API
and take advantage of ICKLE.

Thanks,
Sanne

>
>
> >
> > A stronger reason is that this would introduce circular dependencies
> > between the two projects, and a big overhead of release coordination:
> > we had this in the past, very all very glad this is in the past!
> >
> > When we'll have IQL, this will both define a good "on the wire"
> > representation which would solve the serialization problem, and IQL
> > will also limit the amount of Query types which we will need to
> > support, as at that point we will be able to limit the support for
> > Clustered Queries (which is the feature needing to serialize the
> > queries) to those which IQL can express, and thus serialize.
> >
> > At that point we'll be able to deprecate the the Clustered Query API
> > which accepts a user instance of the Lucene Query, and only run
> > clustered queries for queries expressed over IQL. Not least we'll be
> > able to automatically determine if the query is best run as a
> > clustered query or as a local query, removing this complexity from the
> > user's responsibilities.
> >
> > In conclusion, we'll still be using the "Clustered Query"
> > functionality, but not exposing it, and by doing so we won't need any
> > externalizer. But for now please keep the tests running with the
> > existing externalizer strategies: we just need to keep it functional,
> > but there's no need to optimise performance of these externalizers as
> > we'll get rid of them.
> >
> > Thanks,
> > Sanne
> > _______________________________________________
> > infinispan-dev mailing list
> > [hidden email]
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev