[infinispan-dev] distribution's impact on performance

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] distribution's impact on performance

Mircea Markus
Hi,

I've enhanced Radargun with an ideal distribution function: this is  a perfectly correct behaving function that is aware of the keys passed to it and  makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.  
In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.

Below is the distribution of keys/nodes[1] :

Configuration : dist-sync.xml
Cluster size: 3 -> ( 2930 2612 458)
Cluster size: 5 -> ( 1922 1930 660 2969 2519)
Cluster size: 7 -> ( 4617 1271 720 1028 4394 328 1642)
Cluster size: 9 -> ( 2586 4507 1153 556 1292 274 582 2560 4490)

Configuration : idealdistribution/dist-sync-ideal-distribution.xml
Cluster size: 3 -> ( 2000 2000 2000)
Cluster size: 5 -> ( 2000 2000 2000 2000 2000)
Cluster size: 7 -> ( 2000 2000 2000 2000 2000 2000 2000)
Cluster size: 9 -> ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)

[1] this can now be obtained by running (./bin/dist.sh - just added).

Cheers,

Mircea  



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

All_GET.png (59K) Download Attachment
All_PUT.png (49K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] distribution's impact on performance

Manik Surtani
Does your dummy hash function guarantee any number of puts or gets hitting the local node?  Or does it guarantee that all calls are remote?

On 1 Feb 2011, at 13:11, Mircea Markus wrote:

> Hi,
>
> I've enhanced Radargun with an ideal distribution function: this is  a perfectly correct behaving function that is aware of the keys passed to it and  makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
> I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
> The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.  
> In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.
>
> Below is the distribution of keys/nodes[1] :
>
> Configuration : dist-sync.xml
> Cluster size: 3 -> ( 2930 2612 458)
> Cluster size: 5 -> ( 1922 1930 660 2969 2519)
> Cluster size: 7 -> ( 4617 1271 720 1028 4394 328 1642)
> Cluster size: 9 -> ( 2586 4507 1153 556 1292 274 582 2560 4490)
>
> Configuration : idealdistribution/dist-sync-ideal-distribution.xml
> Cluster size: 3 -> ( 2000 2000 2000)
> Cluster size: 5 -> ( 2000 2000 2000 2000 2000)
> Cluster size: 7 -> ( 2000 2000 2000 2000 2000 2000 2000)
> Cluster size: 9 -> ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)
>
> [1] this can now be obtained by running (./bin/dist.sh - just added).
>
> Cheers,
>
> Mircea  
>
>
> <All_GET.png><All_PUT.png>_______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] distribution's impact on performance

Mircea Markus

On 1 Feb 2011, at 13:31, Manik Surtani wrote:

> Does your dummy hash function
it's not necessarily  dummy. It's a function that is aware of the key structure and based on that knows how to build even distribution - but is a correct fucntion.
> guarantee any number of puts or gets hitting the local node?  Or does it guarantee that all calls are remote?
It spreads the keys from each node to all nodes(including self) equally.

>
> On 1 Feb 2011, at 13:11, Mircea Markus wrote:
>
>> Hi,
>>
>> I've enhanced Radargun with an ideal distribution function: this is  a perfectly correct behaving function that is aware of the keys passed to it and  makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
>> I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
>> The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.  
>> In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.
>>
>> Below is the distribution of keys/nodes[1] :
>>
>> Configuration : dist-sync.xml
>> Cluster size: 3 -> ( 2930 2612 458)
>> Cluster size: 5 -> ( 1922 1930 660 2969 2519)
>> Cluster size: 7 -> ( 4617 1271 720 1028 4394 328 1642)
>> Cluster size: 9 -> ( 2586 4507 1153 556 1292 274 582 2560 4490)
>>
>> Configuration : idealdistribution/dist-sync-ideal-distribution.xml
>> Cluster size: 3 -> ( 2000 2000 2000)
>> Cluster size: 5 -> ( 2000 2000 2000 2000 2000)
>> Cluster size: 7 -> ( 2000 2000 2000 2000 2000 2000 2000)
>> Cluster size: 9 -> ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)
>>
>> [1] this can now be obtained by running (./bin/dist.sh - just added).
>>
>> Cheers,
>>
>> Mircea  
>>
>>
>> <All_GET.png><All_PUT.png>_______________________________________________
>> infinispan-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> [hidden email]
> twitter.com/maniksurtani
>
> Lead, Infinispan
> http://www.infinispan.org
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] distribution's impact on performance

Bela Ban
In reply to this post by Mircea Markus
Hmm, interesting !

For GETs, I think the only reason the line degrades a only slightly
after 5 nodes is that the chance of hitting youself (therefore no
communication needed) decreases with increasing cluster size. I wonder
how this would line would look like on cluster sizes of 30+ ...

For PUTs, the degradation is more pronounced. I assume this is with
numOwners=2 ? In that case, the chance of hitting yourself is twice of
that for GETs.

Can you re-run this test with numOwners=1 ? I guess we should then see a
similar line for PUTs than for GETs.

I assume you're sending instructions on how to run this ideal distribution ?


On 2/1/11 2:11 PM, Mircea Markus wrote:

> Hi,
>
> I've enhanced Radargun with an ideal distribution function: this is  a perfectly correct behaving function that is aware of the keys passed to it and  makes sure that all the nodes have the same share of keys(I'll send an email soon detailing this behavior),
> I've just run a 3-9 test on the cluster{01..10}.mw.lab.eng.bos.redhat.com, results attached.
> The ideal distribution causes the performance to be more linear. Also for puts we have an constant 10-35% performance increase.
> In the case of default CH 3 nodes, the get has a peek, this can be explained by the fact that two of the nodes moved really fast, almost as in replicated mode, as they hold most the data (90%) (see bellow num of keys on each node);.
>
> Below is the distribution of keys/nodes[1] :
>
> Configuration : dist-sync.xml
> Cluster size: 3 ->  ( 2930 2612 458)
> Cluster size: 5 ->  ( 1922 1930 660 2969 2519)
> Cluster size: 7 ->  ( 4617 1271 720 1028 4394 328 1642)
> Cluster size: 9 ->  ( 2586 4507 1153 556 1292 274 582 2560 4490)
>
> Configuration : idealdistribution/dist-sync-ideal-distribution.xml
> Cluster size: 3 ->  ( 2000 2000 2000)
> Cluster size: 5 ->  ( 2000 2000 2000 2000 2000)
> Cluster size: 7 ->  ( 2000 2000 2000 2000 2000 2000 2000)
> Cluster size: 9 ->  ( 2000 2000 2000 2000 2000 2000 2000 2000 2000)
>
> [1] this can now be obtained by running (./bin/dist.sh - just added).

--
Bela Ban
Lead JGroups / Clustering Team
JBoss
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] distribution's impact on performance

Mircea Markus

On 1 Feb 2011, at 15:12, Bela Ban wrote:
> Hmm, interesting !
>
> For GETs, I think the only reason the line degrades a only slightly
> after 5 nodes is that the chance of hitting youself (therefore no
> communication needed) decreases with increasing cluster size. I wonder
> how this would line would look like on cluster sizes of 30+ ...
+1
Did you have the chance to finish the jgroups plugin for Radargun? Very curious to see the results between (ISPN+evenDist) vs (jgroups with numOwners/cluster size local calls).
>
> For PUTs, the degradation is more pronounced. I assume this is with
> numOwners=2 ?
yes
> In that case, the chance of hitting yourself is twice of
> that for GETs.
>
indeed.
> Can you re-run this test with numOwners=1 ? I guess we should then see a
> similar line for PUTs than for GETs.
indeed, it's very similar! Results attached.
>
> I assume you're sending instructions on how to run this ideal distribution ?
https://sourceforge.net/apps/trac/radargun/wiki/WritingCustomPlugins

Cheers,
Mircea

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev

All_GET.png (49K) Download Attachment
All_PUT.png (51K) Download Attachment