Quantcast

[infinispan-dev] Infinispan Query API simplification

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Infinispan Query API simplification

Tristan Tarrant-2
Querying an Infinispan cache is currently a bit cumbersome. There are
two paths:

Ickle:
Search.getQueryFactory(cache).create("...").list();

DSL (one possible example):
Search.getQueryFactory(cache).from(class).[filters].build().list();

Ideally we should have something like:

Ickle:
cache.query("...").list();

DSL:
cache.query(class).[filters].list();

Additionally, the query module is separate from infinispan-core. While
this made sense when we didn't have non-indexed query capabilities (and
is somewhat mitigated by the uberjars), I feel that query should be a
1st class citizen API-wise.
For this reason I propose that we extract the query API to
infinispan-commons,  put the query SPI in infinispan-core together with
the non-indexed implementation and have the hibernate-search backend as
a pluggable implementation.

Thoughts ?

Tristan

--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Sanne Grinovero-3
Looks good to me, at high level.

This will have to focus on the capabilities of the non-indexed query
engine though, as we won't be able to expose the more advanced
capabilities over the DSL without introducing an hard dependency on
Hibernate Search.

Conversely, the method accepting an Ickle query will have to fail at
runtime if it's using full-text capabilities and the indexing engine
wasn't enabled.
I think that's fine, as it's not wildly different than attempting a
full-text query today when having the wrong fields indexed: we can
only validate such things when the query is parsed.
[N.B. I'd like to see a way to pre-register queries - like "prepared
statement", for many other good reasons so that might improve
eventually]

So the tricky part will be to cleanly separate the query API from the
full-text query API, and the downside will be that using full-text
queries will still require some form of API unwrapping.

I take it you'd like to see the same API which we'll make for Cache to
be exposed on Remote Cache ?

Thanks,
Sanne


On 20 April 2017 at 13:08, Tristan Tarrant <[hidden email]> wrote:

> Querying an Infinispan cache is currently a bit cumbersome. There are
> two paths:
>
> Ickle:
> Search.getQueryFactory(cache).create("...").list();
>
> DSL (one possible example):
> Search.getQueryFactory(cache).from(class).[filters].build().list();
>
> Ideally we should have something like:
>
> Ickle:
> cache.query("...").list();
>
> DSL:
> cache.query(class).[filters].list();
>
> Additionally, the query module is separate from infinispan-core. While
> this made sense when we didn't have non-indexed query capabilities (and
> is somewhat mitigated by the uberjars), I feel that query should be a
> 1st class citizen API-wise.
> For this reason I propose that we extract the query API to
> infinispan-commons,  put the query SPI in infinispan-core together with
> the non-indexed implementation and have the hibernate-search backend as
> a pluggable implementation.
>
> Thoughts ?
>
> Tristan
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Radim Vansa
In reply to this post by Tristan Tarrant-2
That's completely opposite approach from the one outlined for
distributed counters and other "on-top" functionality (the same approach
was later suggested for conflict resolution manager, multimap and maybe
others). Why is query 1st level citizen and those others are not?

I am not opposing the idea but let's define the line between patriarchs
and plebeians.

How big is the DSL API surface (which will be brought into commons)?

R.

On 04/20/2017 02:08 PM, Tristan Tarrant wrote:

> Querying an Infinispan cache is currently a bit cumbersome. There are
> two paths:
>
> Ickle:
> Search.getQueryFactory(cache).create("...").list();
>
> DSL (one possible example):
> Search.getQueryFactory(cache).from(class).[filters].build().list();
>
> Ideally we should have something like:
>
> Ickle:
> cache.query("...").list();
>
> DSL:
> cache.query(class).[filters].list();
>
> Additionally, the query module is separate from infinispan-core. While
> this made sense when we didn't have non-indexed query capabilities (and
> is somewhat mitigated by the uberjars), I feel that query should be a
> 1st class citizen API-wise.
> For this reason I propose that we extract the query API to
> infinispan-commons,  put the query SPI in infinispan-core together with
> the non-indexed implementation and have the hibernate-search backend as
> a pluggable implementation.
>
> Thoughts ?
>
> Tristan
>


--
Radim Vansa <[hidden email]>
JBoss Performance Team

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Dan Berindei
On Thu, Apr 20, 2017 at 3:56 PM, Radim Vansa <[hidden email]> wrote:
> That's completely opposite approach from the one outlined for
> distributed counters and other "on-top" functionality (the same approach
> was later suggested for conflict resolution manager, multimap and maybe
> others). Why is query 1st level citizen and those others are not?
>
> I am not opposing the idea but let's define the line between patriarchs
> and plebeians.

I think you meant patricians, not patriarchs :)

But your question makes sense, especially as Sanne pointed out that
some query functionality will not be available via the commons/core
API.

>
> How big is the DSL API surface (which will be brought into commons)?

-1 from me to add anything in commons, I don't think allowing the
users to query both embedded caches and remote caches with the same
code is that important. I'd rather go the opposite way and remove the
BasicCache interface completely.

Dan

>
> R.
>
> On 04/20/2017 02:08 PM, Tristan Tarrant wrote:
>> Querying an Infinispan cache is currently a bit cumbersome. There are
>> two paths:
>>
>> Ickle:
>> Search.getQueryFactory(cache).create("...").list();
>>
>> DSL (one possible example):
>> Search.getQueryFactory(cache).from(class).[filters].build().list();
>>
>> Ideally we should have something like:
>>
>> Ickle:
>> cache.query("...").list();
>>
>> DSL:
>> cache.query(class).[filters].list();
>>
>> Additionally, the query module is separate from infinispan-core. While
>> this made sense when we didn't have non-indexed query capabilities (and
>> is somewhat mitigated by the uberjars), I feel that query should be a
>> 1st class citizen API-wise.
>> For this reason I propose that we extract the query API to
>> infinispan-commons,  put the query SPI in infinispan-core together with
>> the non-indexed implementation and have the hibernate-search backend as
>> a pluggable implementation.
>>
>> Thoughts ?
>>
>> Tristan
>>
>
>
> --
> Radim Vansa <[hidden email]>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Tristan Tarrant-2
In reply to this post by Radim Vansa
Actually, the API for counters et alia is going into commons (so that it
can be shared between embedded and remote). Additionally, something like
a counter has no API relationship with a Cache, whereas query does.

Tristan

On 20/04/2017 14:56, Radim Vansa wrote:

> That's completely opposite approach from the one outlined for
> distributed counters and other "on-top" functionality (the same approach
> was later suggested for conflict resolution manager, multimap and maybe
> others). Why is query 1st level citizen and those others are not?
>
> I am not opposing the idea but let's define the line between patriarchs
> and plebeians.
>
> How big is the DSL API surface (which will be brought into commons)?
>
> R.
>
> On 04/20/2017 02:08 PM, Tristan Tarrant wrote:
>> Querying an Infinispan cache is currently a bit cumbersome. There are
>> two paths:
>>
>> Ickle:
>> Search.getQueryFactory(cache).create("...").list();
>>
>> DSL (one possible example):
>> Search.getQueryFactory(cache).from(class).[filters].build().list();
>>
>> Ideally we should have something like:
>>
>> Ickle:
>> cache.query("...").list();
>>
>> DSL:
>> cache.query(class).[filters].list();
>>
>> Additionally, the query module is separate from infinispan-core. While
>> this made sense when we didn't have non-indexed query capabilities (and
>> is somewhat mitigated by the uberjars), I feel that query should be a
>> 1st class citizen API-wise.
>> For this reason I propose that we extract the query API to
>> infinispan-commons,  put the query SPI in infinispan-core together with
>> the non-indexed implementation and have the hibernate-search backend as
>> a pluggable implementation.
>>
>> Thoughts ?
>>
>> Tristan
>>
>
>

--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Sanne Grinovero-3
In reply to this post by Radim Vansa
On 20 April 2017 at 13:56, Radim Vansa <[hidden email]> wrote:
> That's completely opposite approach from the one outlined for
> distributed counters and other "on-top" functionality (the same approach
> was later suggested for conflict resolution manager, multimap and maybe
> others). Why is query 1st level citizen and those others are not?

I agree this needs to be clarified. Be aware that this is the main
reason for which it didn't happen earlier ;-)
It seems what changed recently is the demand for nicer Query API is
quite popular, but I'm skeptical that our existing helper is so hard
to use; are we sure people won't just be fine with us improving that
API but still keeping the helper?

Maybe just a lack of straight forward examples?

Either way, it would still be good to draft some guidelines for future
API development.

Maybe even just having a quick reference sheet with all extension
using the unwrap would make Query users happier.

## Alternative direction:

what about we actually simplify the Cache API in Infinispan Core, and
move anything which isn't strictly key/value store access to its own
module - to be accessed via unwrap.

This would allow people who need just the Cache to run that directly,
so good for integrators - I'm thinking use cases like Hibernate's
simple caching and WildFly's clustering needs.

Then we introduce a new module which depends on all of these and
exposes a nicer API. People using Infinispan "directly" would use
this, and specific system integrators could use the composition they
prefer by picking only the fragments they want/need.

> I am not opposing the idea but let's define the line between patriarchs
> and plebeians.
>
> How big is the DSL API surface (which will be brought into commons)?

It's one method on Cache to get started ;)
I have no idea how large it is going to be: surely not trivial but I
don't see that as a problem as in term of "fluent API" the IDE will
guide you from the single starting point.

It needs to support several options, hints, flags and not just
"list()" but also streams, and reusing a Query definition as filter
for other purposes.
Finally the fact that one might want to query for projections, keys
and combinations makes even the return type non-trivial.

>
> R.
>
> On 04/20/2017 02:08 PM, Tristan Tarrant wrote:
>> Querying an Infinispan cache is currently a bit cumbersome. There are
>> two paths:
>>
>> Ickle:
>> Search.getQueryFactory(cache).create("...").list();
>>
>> DSL (one possible example):
>> Search.getQueryFactory(cache).from(class).[filters].build().list();
>>
>> Ideally we should have something like:
>>
>> Ickle:
>> cache.query("...").list();
>>
>> DSL:
>> cache.query(class).[filters].list();
>>
>> Additionally, the query module is separate from infinispan-core. While
>> this made sense when we didn't have non-indexed query capabilities (and
>> is somewhat mitigated by the uberjars), I feel that query should be a
>> 1st class citizen API-wise.
>> For this reason I propose that we extract the query API to
>> infinispan-commons,  put the query SPI in infinispan-core together with
>> the non-indexed implementation and have the hibernate-search backend as
>> a pluggable implementation.
>>
>> Thoughts ?
>>
>> Tristan
>>
>
>
> --
> Radim Vansa <[hidden email]>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Tristan Tarrant-2
In reply to this post by Dan Berindei
On 20/04/2017 15:34, Dan Berindei wrote:
>> How big is the DSL API surface (which will be brought into commons)?
>
> -1 from me to add anything in commons, I don't think allowing the
> users to query both embedded caches and remote caches with the same
> code is that important. I'd rather go the opposite way and remove the
> BasicCache interface completely.

Actually, we've had requests for interchangeable APIs...

So, according to your strategy we either have each feature implemented
with a divergent specific embedded or remote API, or each feature has
its own feature-api with two separate feature-embedded and
feature-remote implementations. Both plans sound terrible.

Alternatively, we could go with an infinispan-api package (which Paul
has been advocating for a long time) which would contain the various
interfaces.

Tristan

--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Adrian Nistor
+1 for interchangeable apis
+1 for infinispan-api module

On 04/20/2017 05:06 PM, Tristan Tarrant wrote:

> On 20/04/2017 15:34, Dan Berindei wrote:
>>> How big is the DSL API surface (which will be brought into commons)?
>> -1 from me to add anything in commons, I don't think allowing the
>> users to query both embedded caches and remote caches with the same
>> code is that important. I'd rather go the opposite way and remove the
>> BasicCache interface completely.
> Actually, we've had requests for interchangeable APIs...
>
> So, according to your strategy we either have each feature implemented
> with a divergent specific embedded or remote API, or each feature has
> its own feature-api with two separate feature-embedded and
> feature-remote implementations. Both plans sound terrible.
>
> Alternatively, we could go with an infinispan-api package (which Paul
> has been advocating for a long time) which would contain the various
> interfaces.
>
> Tristan
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Dan Berindei
In reply to this post by Tristan Tarrant-2
On Thu, Apr 20, 2017 at 5:06 PM, Tristan Tarrant <[hidden email]> wrote:

> On 20/04/2017 15:34, Dan Berindei wrote:
>>>
>>> How big is the DSL API surface (which will be brought into commons)?
>>
>>
>> -1 from me to add anything in commons, I don't think allowing the
>> users to query both embedded caches and remote caches with the same
>> code is that important. I'd rather go the opposite way and remove the
>> BasicCache interface completely.
>
>
> Actually, we've had requests for interchangeable APIs...
>
> So, according to your strategy we either have each feature implemented with
> a divergent specific embedded or remote API, or each feature has its own
> feature-api with two separate feature-embedded and feature-remote
> implementations. Both plans sound terrible.
>

Would a divergent embedded vs remote API be that bad? If the
functionality really is different, then I'd rather have different APIs
then force 2 different things use the same API.

E.g. with BasicCache, IMO it would have been better to focus on the
versioned conditional write operations, and remove all the
non-versioned conditional write operations from RemoteCache. I'm sure
we could have improved the versioned API a lot, but instead we worked
mainly on the non-versioned API that we got from BasicCache.

> Alternatively, we could go with an infinispan-api package (which Paul has
> been advocating for a long time) which would contain the various interfaces.
>
>
> Tristan
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Adrian Nistor
If we are discussing how an Ickle query is defined, I insist we need
identical APIs. I don't want to go over the long list of benefits of
that. Let's just say it's trivial to implement because It's done already
:). And now we also have a string dsl, which makes it even easier.

We do not have ATM a unified way of executing an Ickle query. As Tristan
showed in the mail starting this thread, the incantations are slightly
different. And I'd like to have that unified too.

The BasicCache/RemoteCache mishap is a textbook demonstration of a leaky
abstraction, and is unrelated.  Should not stop us from unifying query
execution. But I would probably not add a 'query' method to the cache
interfaces and would rather go with something similar to what Sanne
proposed in a previous email in this thread (the so called 'Alternative
direction').

On 04/20/2017 08:35 PM, Dan Berindei wrote:

> On Thu, Apr 20, 2017 at 5:06 PM, Tristan Tarrant <[hidden email]> wrote:
>> On 20/04/2017 15:34, Dan Berindei wrote:
>>>> How big is the DSL API surface (which will be brought into commons)?
>>>
>>> -1 from me to add anything in commons, I don't think allowing the
>>> users to query both embedded caches and remote caches with the same
>>> code is that important. I'd rather go the opposite way and remove the
>>> BasicCache interface completely.
>>
>> Actually, we've had requests for interchangeable APIs...
>>
>> So, according to your strategy we either have each feature implemented with
>> a divergent specific embedded or remote API, or each feature has its own
>> feature-api with two separate feature-embedded and feature-remote
>> implementations. Both plans sound terrible.
>>
> Would a divergent embedded vs remote API be that bad? If the
> functionality really is different, then I'd rather have different APIs
> then force 2 different things use the same API.
>
> E.g. with BasicCache, IMO it would have been better to focus on the
> versioned conditional write operations, and remove all the
> non-versioned conditional write operations from RemoteCache. I'm sure
> we could have improved the versioned API a lot, but instead we worked
> mainly on the non-versioned API that we got from BasicCache.
>
>> Alternatively, we could go with an infinispan-api package (which Paul has
>> been advocating for a long time) which would contain the various interfaces.
>>
>>
>> Tristan
>>
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Emmanuel Bernard
In reply to this post by Tristan Tarrant-2
We did discuss something like

cache.using(QueryableCache.class).createQuery("...").[build].list();

With some service locator system to have extensible and pluggable modules.

We did discuss a potential problem but I forgot what it was.

Emmanuel

> On 20 Apr 2017, at 14:08, Tristan Tarrant <[hidden email]> wrote:
>
> Querying an Infinispan cache is currently a bit cumbersome. There are
> two paths:
>
> Ickle:
> Search.getQueryFactory(cache).create("...").list();
>
> DSL (one possible example):
> Search.getQueryFactory(cache).from(class).[filters].build().list();
>
> Ideally we should have something like:
>
> Ickle:
> cache.query("...").list();
>
> DSL:
> cache.query(class).[filters].list();
>
> Additionally, the query module is separate from infinispan-core. While
> this made sense when we didn't have non-indexed query capabilities (and
> is somewhat mitigated by the uberjars), I feel that query should be a
> 1st class citizen API-wise.
> For this reason I propose that we extract the query API to
> infinispan-commons,  put the query SPI in infinispan-core together with
> the non-indexed implementation and have the hibernate-search backend as
> a pluggable implementation.
>
> Thoughts ?
>
> Tristan
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Infinispan Query API simplification

Sanne Grinovero-3
On 20 April 2017 at 21:31, Emmanuel Bernard <[hidden email]> wrote:
> We did discuss something like
>
> cache.using(QueryableCache.class).createQuery("...").[build].list();
>
> With some service locator system to have extensible and pluggable modules.
>
> We did discuss a potential problem but I forgot what it was.

The problem is that apparently people don't like having to know that.

It's very similar to today's Search helper? In that case we might as
well keep things as they are.

Thanks,
Sanne


>
> Emmanuel
>
>> On 20 Apr 2017, at 14:08, Tristan Tarrant <[hidden email]> wrote:
>>
>> Querying an Infinispan cache is currently a bit cumbersome. There are
>> two paths:
>>
>> Ickle:
>> Search.getQueryFactory(cache).create("...").list();
>>
>> DSL (one possible example):
>> Search.getQueryFactory(cache).from(class).[filters].build().list();
>>
>> Ideally we should have something like:
>>
>> Ickle:
>> cache.query("...").list();
>>
>> DSL:
>> cache.query(class).[filters].list();
>>
>> Additionally, the query module is separate from infinispan-core. While
>> this made sense when we didn't have non-indexed query capabilities (and
>> is somewhat mitigated by the uberjars), I feel that query should be a
>> 1st class citizen API-wise.
>> For this reason I propose that we extract the query API to
>> infinispan-commons,  put the query SPI in infinispan-core together with
>> the non-indexed implementation and have the hibernate-search backend as
>> a pluggable implementation.
>>
>> Thoughts ?
>>
>> Tristan
>>
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...