CentOS Question posted in
The official documentation can be found here.

How Linux handles the case when multiple processes try to replace the same file at the same time? – CentOS

I know this is a bit of theoretical question but haven’t got any satisfactory answer yet. So thought to put this question here.
I have multiple C++ processes (would also like to know thread behaviour) which contend to replace the same file at the same time. How much is it safe to do in Linux (Using Ubuntu 14.04 and Centos 7)? Do I need to put locks?

Thanks in advance.

Tags:

2

Answers


  1. 0

    The filesystems of Unix-based OS’s like Linux are designed around the notion of inodes, which are internal records describing various metadata about the file. Normally these aren’t interacted with directly by users or programs, but their presence gives these filesystems a level of indirection that allows them to provide some useful semantics that other OS’s (read: Windows) cannot.

    filename --> inode --> data

    In particular, when a file gets deleted, what’s actually happening is the separation of the file’s inode from its filename; not (necessarily) the deletion of the file’s data itself. That is, the file and its contents can continue to exist (albeit invisibly, from the user’s point of view) until all processes have closed their file-handles that were open on that file; once the inode is no longer accessible to any process, only then will the filesystem actually mark the file’s data-blocks as free and available-for-reuse. In the meantime, the filename becomes available for another file’s inode (and data) to be associated with, even though the old file’s inode/data still technically exists.

    The upshot of that is that under Linux it’s perfectly valid to delete (or rename) a file at any time, even if other threads/processes are in the middle of using it; your delete will succeed, and any other programs that have that file open at that instant can simply continue reading/writing/using it, exactly as if it hadn’t been deleted. The only thing that is different is that the filename will no longer appear in its directory, and when they call fclose() (or close() or etc) on the file, the file’s data will go away.

    Since doing mv new.txt old.txt is essentially the same as doing a rm old.txt ; mv new.txt old.txt, there should be no problems with doing this from multiple threads without any synchronization. (note that the slightly different situation of having multiple threads or processes opening the same file simultaneously and writing into it at the same time is a bit more perilous; nothing will crash, but it would be easy for them to overwrite each other’s data and corrupt the file, if they aren’t careful)

    Login or Signup to reply.
  2. 0

    It depends a lot on exactly what you’re going to be doing and how you’re using the files. In general, in Unix/Posix systems like Linux, all file calls will succeed if multiple processes make them, and the general way the OS handles contention is “the last one to do something wins”. Essentially, all modifications to the filesystem are serialized, so the filesystem is always in a consistent state. But otherwise it’s a free-for-all.

    There are a lot of details here though. There’s flags used in opening a file like O_EXCL that can result in failure if another process did it first (a sort of lock). There’s advisory (aka, nobody is forced by the OS to pay attention to them) locking systems like flock (try typing man 2 flock to learn more) for file contents. There are more Linux specific mandatory locking system.

    And there are also details like “What happens if someone deleted a file I have open?” that the other answer explains correctly and well.

    And lastly, there’s a whole mess of detail surrounding whether it’s guaranteed that any particular change to the filesystem is recorded for all eternity, or whether it has a chance of disappearing if someone flicks the power switch. And that’s a mess-and-a-half once you really dive into it, between dodgy hardware that lies to the OS about things to the confusing morass of different Linux system calls covering different aspects of this problem, often entering Linux from different eras of Unix/Posix history and interacting with each other in strange and arcane ways.

    So, an answer to your very general and open-ended question is going to have to necessarily be vague, abstract, and hand-wavey.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search