Super User Asked by Mark Ransom on December 11, 2020
I’m guessing there’s no hard limit, but I know that performance degrades when you put too many files into a single Windows folder. Does anybody have any good rules of thumb for when it becomes noticeably slow to open a file?
For a decent consumer grade hardware, 150K files per folder is the number I have come across as per Windows 10 build 18362.356 using its native Explorer on a WD Blue 4TB hard disk drive in NTFS (partitioned 2 TB + 2TB ). For all files at fixed size of 24KB and filetype .7z . 150K is the number of files the explorer can display, I can select and do some operation. Any larger in the same folder and the windows explorer starts crawling.
It's highly likely this number is dependent on the File Explorer, filesystem, OS, drive speed, drive type (SSD/HDD/Raid HDD etc.) and also the supporting hardware itself like the Storage controllers, CPU and health of the SATA cables (or PATA or m.2 socket). For exammple the SouthBridge controllers would probably have a slower performance than CPU/NB controllers and connecting the drive to the SB should be a slower performance overall. Also 7zip's file explorer is much faster than Windows File Explorer in selecting huge number of files, in the range on 100Ks. I am not certain about the file sizes and if they will or won't affect the read time but my other folders with 1000s of images per folder take a long time, is it due to them having filetype of .jpg/.png etc. or is it due to their huge sizes or is it due to the explorer trying to generate thumbnails for them I am not sure. I have seen windows skips generating thumbnails for images >20 MB so that can be a valid concern.
If you want to be on the performant side, having about 50K files per folder in my opinion would be better as you wouldn't need to worry about different explorer's or os' etc. causing the file explorer to crash or take minutes of selection/display time.
Answered by CryO ELITE on December 11, 2020
I use Windows 10, and 25,000 images in one folder (average size of the file is 500 KB took more than an hour to load completely in the folder. The suggested number of files in one folder is 5000.
Answered by Gia on December 11, 2020
I had trouble some years ago with a directory which had about 30000 files, and new files couldn't be written (it was the "temp" directory of eMule...), it was on a FAT32 partition, but it's possible that I was using Win98 at the time, and that it was a limit of the OS itself.
Answered by Gabriel on December 11, 2020
Assuming NTFS here, in which case the technical limit is around 4 billion files. And until you go over 10s of thousands per directory you really should not worry too much.
Note however that programs like Explorer suffer much sooner than 10s of thousands, because they try to access all files in a given directory to get meta-data, etc.
Answered by jerryjvl on December 11, 2020
Opening a file won't be very slow regardless of how many files you have in a folder. What certainly is going to kill you is enumerating files in that folder. So taking a look at the contents of that folder with Explorer, Far, dir, Get-ChildItem, whatever.
That being said, I have around 2.5k files and folders in my temp folder and display is instantaneous, so that's apparently still a small number.
ETA: Ok, just tried it, 10000 files in a folder take around one second to open that folder in Far, this and 20000 files don't even matter in Explorer.
Answered by Joey on December 11, 2020
Look here. It says for files per volume, not per folder.
Answered by FerranB on December 11, 2020
It depends on the file system. NTFS is way better than FAT32. FAT32 has a hard limit. However, the rule of thumb I go by is about 500 per directory.
Answered by Joel Lucsy on December 11, 2020
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP