[infinispan-dev] Using nio and/or aio for FileCacheStore

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Using nio and/or aio for FileCacheStore

Tristan Tarrant
Dear all,

has anybody looked into using either nio or aio for the FileCacheStore to improve its performance ?
I was thinking of working a little on this for fun.

Tristan

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Using nio and/or aio for FileCacheStore

Manik Surtani
There was some discussion on a better file cache store on this list not too long ago.  No one's taken it up though, feel free to put forward a proposal.  Using a btree structure (rather than hash buckets) may well be a good idea too - faster lookup?  :-)

On 8 Feb 2011, at 16:15, Tristan Tarrant wrote:

> Dear all,
>
> has anybody looked into using either nio or aio for the FileCacheStore to improve its performance ?
> I was thinking of working a little on this for fun.
>
> Tristan
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
[hidden email]
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org




_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Using nio and/or aio for FileCacheStore

Tristan Tarrant
When you mention btree you mean to overcome potential performance degradation caused by many files in one directory (at both the filesystem level and because the root.listFiles() is used) ?

Tristan

On Tue, Feb 8, 2011 at 17:48, Manik Surtani <[hidden email]> wrote:
There was some discussion on a better file cache store on this list not too long ago.  No one's taken it up though, feel free to put forward a proposal.  Using a btree structure (rather than hash buckets) may well be a good idea too - faster lookup?  :-)



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Using nio and/or aio for FileCacheStore

Sanne Grinovero
2011/2/9 Tristan Tarrant <[hidden email]>:
> When you mention btree you mean to overcome potential performance
> degradation caused by many files in one directory (at both the filesystem
> level and because the root.listFiles() is used) ?

No, that's the filesystem's business. I guess Manik was referring to
the internal structure of each file, they use "buckets" containing
multiple values.

>
> Tristan
>
> On Tue, Feb 8, 2011 at 17:48, Manik Surtani <[hidden email]> wrote:
>>
>> There was some discussion on a better file cache store on this list not
>> too long ago.  No one's taken it up though, feel free to put forward a
>> proposal.  Using a btree structure (rather than hash buckets) may well be a
>> good idea too - faster lookup?  :-)
>>
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Using nio and/or aio for FileCacheStore

Tristan Tarrant
Ah, yes, just noticed :)

On Wed, Feb 9, 2011 at 12:08, Sanne Grinovero <[hidden email]> wrote:
2011/2/9 Tristan Tarrant <[hidden email]>:
> When you mention btree you mean to overcome potential performance
> degradation caused by many files in one directory (at both the filesystem
> level and because the root.listFiles() is used) ?

No, that's the filesystem's business. I guess Manik was referring to
the internal structure of each file, they use "buckets" containing
multiple values.


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Using nio and/or aio for FileCacheStore

Sanne Grinovero
BTW, keep in mind that .listFiles() is not atomic on POSIX filesystems
(I don't know of others); if you have many files, and in sequence you
add A and *after that* you remove B, it's possible a single
.listFiles() will fail to list either file.

2011/2/9 Tristan Tarrant <[hidden email]>:

> Ah, yes, just noticed :)
>
> On Wed, Feb 9, 2011 at 12:08, Sanne Grinovero <[hidden email]>
> wrote:
>>
>> 2011/2/9 Tristan Tarrant <[hidden email]>:
>> > When you mention btree you mean to overcome potential performance
>> > degradation caused by many files in one directory (at both the
>> > filesystem
>> > level and because the root.listFiles() is used) ?
>>
>> No, that's the filesystem's business. I guess Manik was referring to
>> the internal structure of each file, they use "buckets" containing
>> multiple values.
>
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev