Server Fault Asked on January 3, 2022
I am observing a very large delay in EFS detecting that a file has been re-created i.e. deleted and then created again.
In this simple test example, I have a single file that gets deleted and recreated around 5 seconds later. I have two EC2 instances mounted to the same EFS:
EC2-1: responsible for reading the file
EC2-2: responsible for deleting and creating the file.
The problem I am seeing is that when EC2-2 deletes the file, EC2-1 correctly updates to say it is no longer present. EC2-2 then recreates the file around 5 seconds later. EC2-1 does not detect the file has returned for another 25 to 30 seconds.
Now, if I run some sort of query on the file system on EC2-1 just after recreation (like and LS command), it DOES then immediately update to say the file is created.
To be clear, I visually see the file get created on EC2-1 FS immediately after creation, just by running an LS. Its reading it programmatically that fails. In my test case I have a Node.js script that literally just calls readFileSync() every second. I have also tested with the same in Python to conclude this is an EFS issue.
If I run the same script on EC2-2, is see expected results i.e. the file is missing for a second and then is available immediately once recreated. So, reading this on the instance that does the delete and create, works as expected.
Its as if EFS is not detecting the file delete/recreation at all.
OS is Ubuntu Server 18.04 on both EC2 VM’s. Tested on new EFS of type "General Purpose" and "High I/O".
Thanks to @kofemann who pointed me to the
noac option I was able to learn about the file system attribute cache and locate the post below that sums up the options nicely.
Answered by Ben on January 3, 2022
Get help from others!