[ic] Performance hit from having many images in one directory?

[John1]

On Tuesday, August 23, 2005 6:30 PM, mike at perusion.com wrote:

> Quoting Davor Ocelic (docelic at mail.inet.hr):
>>
>>>>    1. Switch filesystems to XFS or ReiserFS for their tree-based
>>>>       scheme. They don't degrade like ext2.
>>>>
>>> I like this idea.  Are there any good reasons why I *shouldn't*
>>> switch to one of these filing systems?  If not, then I guess this
>>> has to be the way to go with any new partitions?
>>
>> I use ext3 because of all the established tools that are available
>> for it (the stuff from e2fsprogs and e2tools packages, for example).
>>
That's a fair point - important when you find yourself needing to perform 
some data recovery tricks.

However, but I have had a quick Google for info on Reiser and XFS and it 
seems that these are now quite mature and stable filing systems with a 
growing number tools and support.  Indeed it seems a number of leading Linux 
distributions now install Reiser or XFS as their default filing system, so I 
think I will make plans to migrate to one or other of these newer filing 
systems.

>> Also, I never had the need to try it myself, but I hear that ext3
>> filesystems mount with larger set of default options than xfs or
>> reiserfs do. Mounting ext3 filesystems with options equal to default
>> reiser or xfs supposedly achieves excellent performance.
>
> I am not sure that you are getting why we are talking about this -- we
> are talking specifically about directory lookup performance for many
> thousands of files in a single directory.
>
> I don't think anything has changed recently in the ext2/ext3 schema
> which addresses this.
>
> I personally have noticed slowdowns of systems when the number of
> files in a directory gets into the thousands. It is a very real
> problem that you need to think about if you have large file sets. I
> have personally always used the hashing scheme, partly because
> running directory listings on very large directory is difficult. But
> changing filesystems is a viable option.
>
Thanks Mike - so it seems to me there are no real downsides to migrating to 
one of these newer filing systems apart from the initial hassle so I think I 
will go down this route, even if I do eventually also decide to employ a 
hashing scheme.