[f2fs] Support file-level fsync

This change enables file-level logging for fsync(),
based on which roll-forward recovery recovers fsyncd files.
It handles fsync(x) in two ways:
1) it conducts checkpointing when the parent of x has not been checkpointed, or
2) it writes out x's dirty data blocks and all dirty node blocks in order.

Since roll-forward recovery highly depends on cache, we'll support it
after implementing any internal cache. When we enabled it in a sync manner,
there are so many workarounds.. look like ugly code.

For testing, it introduces a new gn variable 'f2fs_roll_fwd'
that enables fsync() to perform file-level logging.
By default, it is disabled, and every fsync triggers checkpoint.

Prerequisite: set f2fs_roll_fwd to true

Test: fx test fs-tests
Failed tests: RwFullDiskTest.PartialWriteSucceedsForFullDisk/F2fs

Test: new file, fsync
- linux host
 $ fx emu -N --headless -hda third_party/f2fs/test_files/blk1g.bin
- fuchsia vm
 $ mkdir /tmp/mnt;mount /dev/class/block/000 /tmp/mnt
 $ cp /tmp/mnt/10mb.bin /tmp/mnt/10mb.bin.2
 $ f2fs fsync /tmp/mnt/10mb.bin.2
 kill the fuchsia vm process..
- linux host
 $ mount -t f2fs -o noinline_dentry,noinline_data blk1g.bin ~/mnt/
 $ diff ~/mnt/10mb.bin ~/mnt/10mb.bin.2
 $ umount ../mnt

Test: new dir/new file, fsync
- linux host
 $ fx emu -N --headless -hda third_party/f2fs/test_files/blk1g.bin
- fuchsia vm
 $ mkdir /tmp/mnt;mount /dev/class/block/000 /tmp/mnt
 $ mkdir /tmp/mnt/d;cp /tmp/mnt/10mb.bin /tmp/mnt/d/10mb.bin
 $ f2fs fsync /tmp/mnt/d/10mb.bin
 kill the fuchsia vm process..
- linux host
 $ mount -t f2fs -o noinline_dentry,noinline_data blk1g.bin ~/mnt/
 $ diff ~/mnt/10mb.bin ~/mnt/d/10mb.bin
 $ umount ../mnt

Test: modified file, fsync
- linux host
 $ fx emu -N --headless -hda third_party/f2fs/test_files/blk1g.bin
- fuchsia vm
 $ mkdir /tmp/mnt;mount /dev/class/block/000 /tmp/mnt
 $ mkdir /tmp/mnt/d;cp /tmp/mnt/10mb.bin /tmp/mnt/d/10mb.bin
 $ f2fs fsync /tmp/mnt/d/10mb.bin
 $ cat /tmp/mnt/1mb.bin >> /tmp/mnt/d/10mb.bin
 $ f2fs fsync /tmp/mnt/d/10mb.bin
 kill the fuchsia vm process..
- linux host
 $ mount -t f2fs -o noinline_dentry,noinline_data blk1g.bin ~/mnt/
 $ cat ~/mnt/1mb.bin >> ~/mnt/10mb.bin
 $ diff ~/mnt/10mb.bin ~/mnt/d/10mb.bin
 $ umount ../mnt

Prerequisite: set f2fs_roll_fwd to true
Change-Id: I602a4ebc2669f94636f6014b51048fff8cc5d979
Reviewed-on: https://fuchsia-review.googlesource.com/c/third_party/f2fs/+/538201
Reviewed-by: Brett Wilson <brettw@google.com>
11 files changed
tree: eba35a7f7628d5a1ba37644f63ac9fcd096243a8
  1. patches/
  2. test_files/
  3. third_party/
  4. tools/
  5. AUTHORS
  6. bcache.cc
  7. bcache.h
  8. BUILD.gn
  9. checkpoint.cc
  10. CONTRIBUTING.md
  11. data.cc
  12. dir.cc
  13. dir.h
  14. dir_hash.cc
  15. f2fs.cc
  16. f2fs.h
  17. f2fs_internal.h
  18. f2fs_layout.h
  19. f2fs_lib.h
  20. f2fs_types.h
  21. file.cc
  22. file.h
  23. gc.cc
  24. LICENSE
  25. mkfs.cc
  26. mkfs.h
  27. namei.cc
  28. namestring.h
  29. node.cc
  30. node.h
  31. PATENTS
  32. README.md
  33. recovery.cc
  34. segment.cc
  35. segment.h
  36. super.cc
  37. vnode.cc
  38. vnode.h
README.md

What is F2FS?

F2FS is the most commonly used log-structured file system in Linux. It supports flash-friendly features such as adaptive logging, multi-head logging, fsync acceleration, and node address translation. For more information see: https://www.usenix.org/conference/fast15/technical-sessions/presentation/lee

Major Release

1st release (Apr-06-2021)

  • Features: mkfs, mount/umount, read, write, rmdir, mkdir, rename, fsync, lseek, link, unlink
  • Remarks
  • Fuchsia base code ; Thu Mar 11 08:53:24 2021 Prashanth Swaminathan,76a08ad1474 [speak] Migrate to new component templates
  • There is no cache. Every request is handled as a synchronous operation.
  • fsync() triggers checkpointing
  • lock granularity: file operation
  • LFS is used for block allocation by default, and IPU is forced when the number of free sections is below 50%.
  • no background/foreground gc
  • disable the roll-forward recovery
  • 6 active logs
  • disable the ext-identify feature
  • block allocation: LFS, IPU
  • fsck do nothing, but it returns true
  • no discard

2nd release (May-28-2021)

  • Features: truncate, ssr, fsync, recovery
  • Remarks
  • Fuchsia base code ; Thu May 20 07:25:45 2021 Yilong Li, 02c0dfff0fdb
  • support the roll-forward recovery and file level fsync()
  • block allocation: LFS, IPU, SSR
  • truncate

3rd release (June-25-2021)

  • Features: fsck, mount option, mkfs option
  • Remarts
  • fsck (base: bbf6e62)
  • support the ext-identify feature
  • support discard

4th release (July-30-2021 ~)

  • Testability
  • Features: buffered IO, gc (if writable PagedVfs is available)
  • Remarks
  • background/foreground gc
  • caching node/meta blocks
  • read ahead
  • Page level locking
  • buffered IO
  • mmap
  • IPU can be disabled.
  • support fvm
  • xattr

How to build

$ fx set core.x64 --with-base //bundles:tools --with-base //bundles:tests --with-base third_party/f2fs $ fx emu -N --headless (see third_party/f2fs/test_files/README.md for test)