'Directory/Prefix with 40,000 objects locks windows explorer and other applications
Here at work we use s3fs because our product uses Oracle and the legacy code load files uploaded thru web direct from Oracle pl/sql, so we need the Oracle Database to see the same file system the web server sees. And we cannot access the OS where Oracle is installed to mount Windows shares thru SAMBA.
The problem is, when a directory (prefix in s3) reaches rougly 40,000 files, the access to this directory thru samba become extremely slow, causing timeouts and even completely stop the application pool trying to access this directory.
Our web servers are EC2 instances with Windows 2019.
I wonder if someone know a solution to this.
amazon-s3Solution 1:[1]
S3 LIST API is slow when objects are lots. It should be better if you could re-organize files into multiple directory levels.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Rui Su |