NYCPHP Meetup

NYPHP.org

[nycphp-talk] [OT] number of files in a directory?

leam at reuel.net leam at reuel.net
Sat Dec 31 23:12:01 EST 2005


I don't know about performance, but it should at least work. Would hate to build an array off it though.

I've set up a test directory on a SuSE system. I also have a Red Hat 3 and Fedora Core 3 box at hand. If you have any tests you'd like to e-mail me feel free to send them over. I'll be diving tomorrow but might have time to run them Monday.

The ~/tmp/test directory is in export. It went from about 8% of inodes used to 22%. If you have a say in the building of the system and the files are small you can probably increase the inode count a bit. 

leam at leam:~/tmp/test> ls | wc -l
75001

leam at leam:~/tmp/test> df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/hda5             787968  134582  653386   18% /
tmpfs                 129537       5  129532    1% /dev/shm
/dev/hda1              51400      36   51364    1% /boot
/dev/hda3             393984   85709  308275   22% /export

leam at leam:~/tmp/test> df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda5              6192512   2896792   2981132  50% /
tmpfs                   518148         8    518140   1% /dev/shm
/dev/hda1               198313      5535    182527   3% /boot
/dev/hda3              3096240    251408   2687540   9% /export


ciao!

leam



> Hi there:
> 
> Thanks for the response.
> 
> The OS will be Red Hat Enterprise 3 or 4; That will be determined 
> when we get the server in February.
> 
> But, this will be a dedicated hosting box at 
> servermatrix...whatever's in the client's price range at the time we 
> buy (probably their low-end Pentium 4 server).  Anyway, it'll be our 
> machine.
> 
> So, inodes are the only particular worry?  The directory structure is 
> irrelevant?
> 
> I can see how things are working on my OS X development machine in 
> the meantime, which has a hell of a lot more going on than will be 
> going on on this server.
> 
> Cheers,
> 
> Marc
> 
> 
> 
> >Happy New Year to you as well!
> >
> >What OS?
> 
> 
> >Most unix filesystems have a set number of inodes and you need at 
> >least one inode per file. Files larger than the block size (4k-8k 
> >usually) take more than one inode. If you have shell access to the 
> >box try:
> >
> >	df -i
> >
> >It should tell you the number of inodes. The other issue you may 
> >have, even if this works, is if the box is a shared host for other 
> >clients. You'll take up a significant chunk of inodes.
> >
> >That's what springs off the top of my head.
> >
> >ciao!
> >
> >leam
> >
> >
> >On Sat, Dec 31, 2005 at 07:53:38PM -0500, Marc Antony Vose wrote:
> >>  Hey all:
> >>
> >>  First of all:  Happy New Year!
> >>
> >>  Secondly: I am rebuilding a site that was coded somewhat sloppily,
> >>  and they have product images all stored in one directory (a script
> >>  that I am not writing auto-uploads them to the web server from
> >>  elsewhere).  Presently, this directory contains about 33,000 files.
> >>  It will be more like 75,000 when the site launches, if things remain
> >>  the same.
> >>
> >>  The question is:  should I be worried about this, or was this only a
> >>  problem several years ago? (I remember people at one time attempting
> >>  to not put too many files in one place.)
> >>
> >>  If I should be worried, what could happen?  Will we ever reach a hard
> >>  limit of files per directory?
> >>
> >>  Is it better if each product instead has its own directory inside
> >>  there (i.e., 75,000 directories), each with as many files as we need
> >>  inside, or is that just the same problem?
> >>
> >>  Cheers,
> >>
> >>  --
> >>  Marc Antony Vose
> >>  http://www.suzerain.com/
> >>
> >>  Imagination is more important than knowledge.
> >>  -- Albert Einstein
> >>  _______________________________________________
> >>  New York PHP Talk Mailing List
> >>  AMP Technology
> >>  Supporting Apache, MySQL and PHP
> >>  http://lists.nyphp.org/mailman/listinfo/talk
> >>  http://www.nyphp.org
> >>
> >>
> >_______________________________________________
> >New York PHP Talk Mailing List
> >AMP Technology
> >Supporting Apache, MySQL and PHP
> >http://lists.nyphp.org/mailman/listinfo/talk
> >http://www.nyphp.org
> 
> _______________________________________________
> New York PHP Talk Mailing List
> AMP Technology
> Supporting Apache, MySQL and PHP
> http://lists.nyphp.org/mailman/listinfo/talk
> http://www.nyphp.org
> 
> 



More information about the talk mailing list