[infinispan-dev] Distributed execution framework API - final review

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[infinispan-dev] Distributed execution framework API - final review

Vladimir Blagojevic
Hi,

Manik and I agreed on final look of distributed execution framework
API[1]. We removed DistributedTask used for migrating
DistributedCallable to execution nodes and replaced it with
DistributedExecutorService which follows ideas of a familiar
ExecutorService from util.concurrent.

MapReduce is still a beast of its own that can not fit into
ExecutorService paradigm but I think we nailed it with a simple and easy
to use API. See in particular examples provided.


The last item Manik and I disagree on is use of DistributedTaskContext.
DistributedTaskContext is given to each DistributedCallable once it has
migrated to remote node for execution. DistributedTaskContext might
evolve and I'd rather keep it in the framework while Manik wants to have
a simple setter on DistributedCallable:

setEnvironment(Cache, K...)

I think of it as an insurance policy in case we need to bootstrap
DistributedCallable with more parameters rather than only Cache and
input keys K.

Lets hear your thoughts and comments.

Regards,
Vladimir


[1]
https://github.com/vblagoje/infinispan/tree/76bcf603946955bfac58ab4f22995bbb19ed8509/core/src/main/java/org/infinispan/distexec 

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Vladimir Blagojevic
Proper link is
https://github.com/vblagoje/infinispan/tree/t_ispn-39_master/core/src/main/java/org/infinispan/distexec

Regards,
Vladimir

On 11-01-14 12:47 PM, Vladimir Blagojevic wrote:

> Hi,
>
> Manik and I agreed on final look of distributed execution framework
> API[1]. We removed DistributedTask used for migrating
> DistributedCallable to execution nodes and replaced it with
> DistributedExecutorService which follows ideas of a familiar
> ExecutorService from util.concurrent.
>
> MapReduce is still a beast of its own that can not fit into
> ExecutorService paradigm but I think we nailed it with a simple and easy
> to use API. See in particular examples provided.
>
>
> The last item Manik and I disagree on is use of DistributedTaskContext.
> DistributedTaskContext is given to each DistributedCallable once it has
> migrated to remote node for execution. DistributedTaskContext might
> evolve and I'd rather keep it in the framework while Manik wants to have
> a simple setter on DistributedCallable:
>
> setEnvironment(Cache, K...)
>
> I think of it as an insurance policy in case we need to bootstrap
> DistributedCallable with more parameters rather than only Cache and
> input keys K.
>
> Lets hear your thoughts and comments.
>
> Regards,
> Vladimir
>
>
> [1]
> https://github.com/vblagoje/infinispan/tree/76bcf603946955bfac58ab4f22995bbb19ed8509/core/src/main/java/org/infinispan/distexec
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Pete Muir
In reply to this post by Vladimir Blagojevic

On 14 Jan 2011, at 15:47, Vladimir Blagojevic wrote:

> Hi,
>
> Manik and I agreed on final look of distributed execution framework
> API[1]. We removed DistributedTask used for migrating
> DistributedCallable to execution nodes and replaced it with
> DistributedExecutorService which follows ideas of a familiar
> ExecutorService from util.concurrent.
>
> MapReduce is still a beast of its own that can not fit into
> ExecutorService paradigm but I think we nailed it with a simple and easy
> to use API. See in particular examples provided.
>
>
> The last item Manik and I disagree on is use of DistributedTaskContext.
> DistributedTaskContext is given to each DistributedCallable once it has
> migrated to remote node for execution. DistributedTaskContext might
> evolve and I'd rather keep it in the framework while Manik wants to have
> a simple setter on DistributedCallable:
>
> setEnvironment(Cache, K...)

I prefer Manik's design, I generally try to design APIs for known use cases.
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Emmanuel Bernard

On 14 janv. 2011, at 17:47, Pete Muir wrote:

>
> On 14 Jan 2011, at 15:47, Vladimir Blagojevic wrote:
>
>> Hi,
>>
>> Manik and I agreed on final look of distributed execution framework
>> API[1]. We removed DistributedTask used for migrating
>> DistributedCallable to execution nodes and replaced it with
>> DistributedExecutorService which follows ideas of a familiar
>> ExecutorService from util.concurrent.
>>
>> MapReduce is still a beast of its own that can not fit into
>> ExecutorService paradigm but I think we nailed it with a simple and easy
>> to use API. See in particular examples provided.
>>
>>
>> The last item Manik and I disagree on is use of DistributedTaskContext.
>> DistributedTaskContext is given to each DistributedCallable once it has
>> migrated to remote node for execution. DistributedTaskContext might
>> evolve and I'd rather keep it in the framework while Manik wants to have
>> a simple setter on DistributedCallable:
>>
>> setEnvironment(Cache, K...)
>
> I prefer Manik's design, I generally try to design APIs for known use cases.

I've taken a middle ground in Bean Validation.
When a contract needed A and B for 90% of the forseable use cases (currently covered and potential alike), we went for the following

void contract(A a, B b, ContractContext context);

It has proven to be quite robust for a few reasons including:
 - people know what's really important
 - we kept doors open for the future
 - various implementations used these doors to provide provider specific extensions
 - we will use these for BV 1.1 or 2

Also, I am a bit worried about K...
in theory it looks good but how are keys going to be carried. Are they known a compilation time (where varargs shine) or are they likely going to be added to a Set and passed around (in which case varargs is a pain as you need to convert from a set to an array).


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Mircea Markus
In reply to this post by Pete Muir

On 14 Jan 2011, at 16:47, Pete Muir wrote:

>
> On 14 Jan 2011, at 15:47, Vladimir Blagojevic wrote:
>
>> Hi,
>>
>> Manik and I agreed on final look of distributed execution framework
>> API[1]. We removed DistributedTask used for migrating
>> DistributedCallable to execution nodes and replaced it with
>> DistributedExecutorService which follows ideas of a familiar
>> ExecutorService from util.concurrent.
>>
>> MapReduce is still a beast of its own that can not fit into
>> ExecutorService paradigm but I think we nailed it with a simple and easy
>> to use API. See in particular examples provided.
>>
>>
>> The last item Manik and I disagree on is use of DistributedTaskContext.
>> DistributedTaskContext is given to each DistributedCallable once it has
>> migrated to remote node for execution. DistributedTaskContext might
>> evolve and I'd rather keep it in the framework while Manik wants to have
>> a simple setter on DistributedCallable:
>>
>> setEnvironment(Cache, K...)
>
> I prefer Manik's design, I generally try to design APIs for known use cases.
+1. If needed this can be added in an future releases (6.x)
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Vladimir Blagojevic
In reply to this post by Emmanuel Bernard
Emmanuel,

Great points. The input keys K are known at compilation time. In this
contract implementers are most likely to save a reference to input keys
in a field variable rather than iterate through keys immediately. I
think that it might be a better idea to use Collection, no?


On 11-01-14 2:55 PM, Emmanuel Bernard wrote:

> I've taken a middle ground in Bean Validation.
> When a contract needed A and B for 90% of the forseable use cases (currently covered and potential alike), we went for the following
>
> void contract(A a, B b, ContractContext context);
>
> It has proven to be quite robust for a few reasons including:
>   - people know what's really important
>   - we kept doors open for the future
>   - various implementations used these doors to provide provider specific extensions
>   - we will use these for BV 1.1 or 2
>
> Also, I am a bit worried about K...
> in theory it looks good but how are keys going to be carried. Are they known a compilation time (where varargs shine) or are they likely going to be added to a Set and passed around (in which case varargs is a pain as you need to convert from a set to an array).
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [infinispan-dev] Distributed execution framework API - final review

Manik Surtani
Actually yes - Set<K> makes more sense for this API than K...

Varargs, IMO, are only useful in an API where users call the API.  But if users have to *implement* the API (as in this case) then collections make more sense.   I would use Set rather than Collection as it enforces (and informs the implementor) that there won't be duplicates of K.


On 17 Jan 2011, at 13:57, Vladimir Blagojevic wrote:

> Emmanuel,
>
> Great points. The input keys K are known at compilation time. In this
> contract implementers are most likely to save a reference to input keys
> in a field variable rather than iterate through keys immediately. I
> think that it might be a better idea to use Collection, no?
>
>
> On 11-01-14 2:55 PM, Emmanuel Bernard wrote:
>> I've taken a middle ground in Bean Validation.
>> When a contract needed A and B for 90% of the forseable use cases (currently covered and potential alike), we went for the following
>>
>> void contract(A a, B b, ContractContext context);
>>
>> It has proven to be quite robust for a few reasons including:
>>  - people know what's really important
>>  - we kept doors open for the future
>>  - various implementations used these doors to provide provider specific extensions
>>  - we will use these for BV 1.1 or 2
>>
>> Also, I am a bit worried about K...
>> in theory it looks good but how are keys going to be carried. Are they known a compilation time (where varargs shine) or are they likely going to be added to a Set and passed around (in which case varargs is a pain as you need to convert from a set to an array).
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Loading...