Documentation/filesystems/ext4.txt

   1
   2 Ext4 Filesystem
   3 ===============
   4
   5 Ext4 is an an advanced level of the ext3 filesystem which incorporates
   6 scalability and reliability enhancements for supporting large filesystems
   7 (64 bit) in keeping with increasing disk capacities and state-of-the-art
   8 feature requirements.
   9
  10 Mailing list:   linux-ext4@vger.kernel.org
  11 Web site:       http://ext4.wiki.kernel.org
  12
  13
  14 1. Quick usage instructions:
  15 ===========================
  16
  17 Note: More extensive information for getting started with ext4 can be
  18       found at the ext4 wiki site at the URL:
  19       http://ext4.wiki.kernel.org/index.php/Ext4_Howto
  20
  21   - Compile and install the latest version of e2fsprogs (as of this
  22     writing version 1.41.3) from:
  23
  24     http://sourceforge.net/project/showfiles.php?group_id=2406
  25
  26         or
  27
  28     ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/
  29
  30         or grab the latest git repository from:
  31
  32     git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
  33
  34   - Note that it is highly important to install the mke2fs.conf file
  35     that comes with the e2fsprogs 1.41.x sources in /etc/mke2fs.conf. If
  36     you have edited the /etc/mke2fs.conf file installed on your system,
  37     you will need to merge your changes with the version from e2fsprogs
  38     1.41.x.
  39
  40   - Create a new filesystem using the ext4 filesystem type:
  41
  42         # mke2fs -t ext4 /dev/hda1
  43
  44     Or to configure an existing ext3 filesystem to support extents:
  45
  46         # tune2fs -O extents /dev/hda1
  47
  48     If the filesystem was created with 128 byte inodes, it can be
  49     converted to use 256 byte for greater efficiency via:
  50
  51         # tune2fs -I 256 /dev/hda1
  52
  53     (Note: we currently do not have tools to convert an ext4
  54     filesystem back to ext3; so please do not do try this on production
  55     filesystems.)
  56
  57   - Mounting:
  58
  59         # mount -t ext4 /dev/hda1 /wherever
  60
  61   - When comparing performance with other filesystems, it's always
  62     important to try multiple workloads; very often a subtle change in a
  63     workload parameter can completely change the ranking of which
  64     filesystems do well compared to others.  When comparing versus ext3,
  65     note that ext4 enables write barriers by default, while ext3 does
  66     not enable write barriers by default.  So it is useful to use
  67     explicitly specify whether barriers are enabled or not when via the
  68     '-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems
  69     for a fair comparison.  When tuning ext3 for best benchmark numbers,
  70     it is often worthwhile to try changing the data journaling mode; '-o
  71     data=writeback,nobh' can be faster for some workloads.  (Note
  72     however that running mounted with data=writeback can potentially
  73     leave stale data exposed in recently written files in case of an
  74     unclean shutdown, which could be a security exposure in some
  75     situations.)  Configuring the filesystem with a large journal can
  76     also be helpful for metadata-intensive workloads.
  77
  78 2. Features
  79 ===========
  80
  81 2.1 Currently available
  82
  83 * ability to use filesystems > 16TB (e2fsprogs support not available yet)
  84 * extent format reduces metadata overhead (RAM, IO for access, transactions)
  85 * extent format more robust in face of on-disk corruption due to magics,
  86 * internal redundancy in tree
  87 * improved file allocation (multi-block alloc)
  88 * lift 32000 subdirectory limit imposed by i_links_count[1]
  89 * nsec timestamps for mtime, atime, ctime, create time
  90 * inode version field on disk (NFSv4, Lustre)
  91 * reduced e2fsck time via uninit_bg feature
  92 * journal checksumming for robustness, performance
  93 * persistent file preallocation (e.g for streaming media, databases)
  94 * ability to pack bitmaps and inode tables into larger virtual groups via the
  95   flex_bg feature
  96 * large file support
  97 * Inode allocation using large virtual block groups via flex_bg
  98 * delayed allocation
  99 * large block (up to pagesize) support
 100 * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force
 101   the ordering)
 102
 103 [1] Filesystems with a block size of 1k may see a limit imposed by the
 104 directory hash tree having a maximum depth of two.
 105
 106 2.2 Candidate features for future inclusion
 107
 108 * Online defrag (patches available but not well tested)
 109 * reduced mke2fs time via lazy itable initialization in conjuction with
 110   the uninit_bg feature (capability to do this is available in e2fsprogs
 111   but a kernel thread to do lazy zeroing of unused inode table blocks
 112   after filesystem is first mounted is required for safety)
 113
 114 There are several others under discussion, whether they all make it in is
 115 partly a function of how much time everyone has to work on them. Features like
 116 metadata checksumming have been discussed and planned for a bit but no patches
 117 exist yet so I'm not sure they're in the near-term roadmap.
 118
 119 The big performance win will come with mballoc, delalloc and flex_bg
 120 grouping of bitmaps and inode tables.  Some test results available here:
 121
 122  - http://www.bullopensource.org/ext4/20080818-ffsb/ffsb-write-2.6.27-rc1.html
 123  - http://www.bullopensource.org/ext4/20080818-ffsb/ffsb-readwrite-2.6.27-rc1.html
 124
 125 3. Options
 126 ==========
 127
 128 When mounting an ext4 filesystem, the following option are accepted:
 129 (*) == default
 130
 131 ro                      Mount filesystem read only. Note that ext4 will
 132                         replay the journal (and thus write to the
 133                         partition) even when mounted "read only". The
 134                         mount options "ro,noload" can be used to prevent
 135                         writes to the filesystem.
 136
 137 journal_checksum        Enable checksumming of the journal transactions.
 138                         This will allow the recovery code in e2fsck and the
 139                         kernel to detect corruption in the kernel.  It is a
 140                         compatible change and will be ignored by older kernels.
 141
 142 journal_async_commit    Commit block can be written to disk without waiting
 143                         for descriptor blocks. If enabled older kernels cannot
 144                         mount the device. This will enable 'journal_checksum'
 145                         internally.
 146
 147 journal=update          Update the ext4 file system's journal to the current
 148                         format.
 149
 150 journal_dev=devnum      When the external journal device's major/minor numbers
 151                         have changed, this option allows the user to specify
 152                         the new journal location.  The journal device is
 153                         identified through its new major/minor numbers encoded
 154                         in devnum.
 155
 156 noload                  Don't load the journal on mounting.  Note that
 157                         if the filesystem was not unmounted cleanly,
 158                         skipping the journal replay will lead to the
 159                         filesystem containing inconsistencies that can
 160                         lead to any number of problems.
 161
 162 data=journal            All data are committed into the journal prior to being
 163                         written into the main file system.
 164
 165 data=ordered    (*)     All data are forced directly out to the main file
 166                         system prior to its metadata being committed to the
 167                         journal.
 168
 169 data=writeback          Data ordering is not preserved, data may be written
 170                         into the main file system after its metadata has been
 171                         committed to the journal.
 172
 173 commit=nrsec    (*)     Ext4 can be told to sync all its data and metadata
 174                         every 'nrsec' seconds. The default value is 5 seconds.
 175                         This means that if you lose your power, you will lose
 176                         as much as the latest 5 seconds of work (your
 177                         filesystem will not be damaged though, thanks to the
 178                         journaling).  This default value (or any low value)
 179                         will hurt performance, but it's good for data-safety.
 180                         Setting it to 0 will have the same effect as leaving
 181                         it at the default (5 seconds).
 182                         Setting it to very large values will improve
 183                         performance.
 184
 185 barrier=<0|1(*)>        This enables/disables the use of write barriers in
 186 barrier(*)              the jbd code.  barrier=0 disables, barrier=1 enables.
 187 nobarrier               This also requires an IO stack which can support
 188                         barriers, and if jbd gets an error on a barrier
 189                         write, it will disable again with a warning.
 190                         Write barriers enforce proper on-disk ordering
 191                         of journal commits, making volatile disk write caches
 192                         safe to use, at some performance penalty.  If
 193                         your disks are battery-backed in one way or another,
 194                         disabling barriers may safely improve performance.
 195                         The mount options "barrier" and "nobarrier" can
 196                         also be used to enable or disable barriers, for
 197                         consistency with other ext4 mount options.
 198
 199 inode_readahead=n       This tuning parameter controls the maximum
 200                         number of inode table blocks that ext4's inode
 201                         table readahead algorithm will pre-read into
 202                         the buffer cache.  The default value is 32 blocks.
 203
 204 orlov           (*)     This enables the new Orlov block allocator. It is
 205                         enabled by default.
 206
 207 oldalloc                This disables the Orlov block allocator and enables
 208                         the old block allocator.  Orlov should have better
 209                         performance - we'd like to get some feedback if it's
 210                         the contrary for you.
 211
 212 user_xattr              Enables Extended User Attributes.  Additionally, you
 213                         need to have extended attribute support enabled in the
 214                         kernel configuration (CONFIG_EXT4_FS_XATTR).  See the
 215                         attr(5) manual page and http://acl.bestbits.at/ to
 216                         learn more about extended attributes.
 217
 218 nouser_xattr            Disables Extended User Attributes.
 219
 220 acl                     Enables POSIX Access Control Lists support.
 221                         Additionally, you need to have ACL support enabled in
 222                         the kernel configuration (CONFIG_EXT4_FS_POSIX_ACL).
 223                         See the acl(5) manual page and http://acl.bestbits.at/
 224                         for more information.
 225
 226 noacl                   This option disables POSIX Access Control List
 227                         support.
 228
 229 reservation
 230
 231 noreservation
 232
 233 bsddf           (*)     Make 'df' act like BSD.
 234 minixdf                 Make 'df' act like Minix.
 235
 236 debug                   Extra debugging information is sent to syslog.
 237
 238 errors=remount-ro       Remount the filesystem read-only on an error.
 239 errors=continue         Keep going on a filesystem error.
 240 errors=panic            Panic and halt the machine if an error occurs.
 241                         (These mount options override the errors behavior
 242                         specified in the superblock, which can be configured
 243                         using tune2fs)
 244
 245 data_err=ignore(*)      Just print an error message if an error occurs
 246                         in a file data buffer in ordered mode.
 247 data_err=abort          Abort the journal if an error occurs in a file
 248                         data buffer in ordered mode.
 249
 250 grpid                   Give objects the same group ID as their creator.
 251 bsdgroups
 252
 253 nogrpid         (*)     New objects have the group ID of their creator.
 254 sysvgroups
 255
 256 resgid=n                The group ID which may use the reserved blocks.
 257
 258 resuid=n                The user ID which may use the reserved blocks.
 259
 260 sb=n                    Use alternate superblock at this location.
 261
 262 quota
 263 noquota
 264 grpquota
 265 usrquota
 266
 267 bh              (*)     ext4 associates buffer heads to data pages to
 268 nobh                    (a) cache disk block mapping information
 269                         (b) link pages into transaction to provide
 270                             ordering guarantees.
 271                         "bh" option forces use of buffer heads.
 272                         "nobh" option tries to avoid associating buffer
 273                         heads (supported only for "writeback" mode).
 274
 275 stripe=n                Number of filesystem blocks that mballoc will try
 276                         to use for allocation size and alignment. For RAID5/6
 277                         systems this should be the number of data
 278                         disks *  RAID chunk size in file system blocks.
 279 delalloc        (*)     Deferring block allocation until write-out time.
 280 nodelalloc              Disable delayed allocation. Blocks are allocation
 281                         when data is copied from user to page cache.
 282
 283 max_batch_time=usec     Maximum amount of time ext4 should wait for
 284                         additional filesystem operations to be batch
 285                         together with a synchronous write operation.
 286                         Since a synchronous write operation is going to
 287                         force a commit and then a wait for the I/O
 288                         complete, it doesn't cost much, and can be a
 289                         huge throughput win, we wait for a small amount
 290                         of time to see if any other transactions can
 291                         piggyback on the synchronous write.   The
 292                         algorithm used is designed to automatically tune
 293                         for the speed of the disk, by measuring the
 294                         amount of time (on average) that it takes to
 295                         finish committing a transaction.  Call this time
 296                         the "commit time".  If the time that the
 297                         transaction has been running is less than the
 298                         commit time, ext4 will try sleeping for the
 299                         commit time to see if other operations will join
 300                         the transaction.   The commit time is capped by
 301                         the max_batch_time, which defaults to 15000us
 302                         (15ms).   This optimization can be turned off
 303                         entirely by setting max_batch_time to 0.
 304
 305 min_batch_time=usec     This parameter sets the commit time (as
 306                         described above) to be at least min_batch_time.
 307                         It defaults to zero microseconds.  Increasing
 308                         this parameter may improve the throughput of
 309                         multi-threaded, synchronous workloads on very
 310                         fast disks, at the cost of increasing latency.
 311
 312 journal_ioprio=prio     The I/O priority (from 0 to 7, where 0 is the
 313                         highest priorty) which should be used for I/O
 314                         operations submitted by kjournald2 during a
 315                         commit operation.  This defaults to 3, which is
 316                         a slightly higher priority than the default I/O
 317                         priority.
 318
 319 auto_da_alloc(*)        Many broken applications don't use fsync() when
 320 noauto_da_alloc         replacing existing files via patterns such as
 321                         fd = open("foo.new")/write(fd,..)/close(fd)/
 322                         rename("foo.new", "foo"), or worse yet,
 323                         fd = open("foo", O_TRUNC)/write(fd,..)/close(fd).
 324                         If auto_da_alloc is enabled, ext4 will detect
 325                         the replace-via-rename and replace-via-truncate
 326                         patterns and force that any delayed allocation
 327                         blocks are allocated such that at the next
 328                         journal commit, in the default data=ordered
 329                         mode, the data blocks of the new file are forced
 330                         to disk before the rename() operation is
 331                         committed.  This provides roughly the same level
 332                         of guarantees as ext3, and avoids the
 333                         "zero-length" problem that can happen when a
 334                         system crashes before the delayed allocation
 335                         blocks are forced to disk.
 336
 337 Data Mode
 338 =========
 339 There are 3 different data modes:
 340
 341 * writeback mode
 342 In data=writeback mode, ext4 does not journal data at all.  This mode provides
 343 a similar level of journaling as that of XFS, JFS, and ReiserFS in its default
 344 mode - metadata journaling.  A crash+recovery can cause incorrect data to
 345 appear in files which were written shortly before the crash.  This mode will
 346 typically provide the best ext4 performance.
 347
 348 * ordered mode
 349 In data=ordered mode, ext4 only officially journals metadata, but it logically
 350 groups metadata information related to data changes with the data blocks into a
 351 single unit called a transaction.  When it's time to write the new metadata
 352 out to disk, the associated data blocks are written first.  In general,
 353 this mode performs slightly slower than writeback but significantly faster than journal mode.
 354
 355 * journal mode
 356 data=journal mode provides full data and metadata journaling.  All new data is
 357 written to the journal first, and then to its final location.
 358 In the event of a crash, the journal can be replayed, bringing both data and
 359 metadata into a consistent state.  This mode is the slowest except when data
 360 needs to be read from and written to disk at the same time where it
 361 outperforms all others modes.  Currently ext4 does not have delayed
 362 allocation support if this data journalling mode is selected.
 363
 364 References
 365 ==========
 366
 367 kernel source:  <file:fs/ext4/>
 368                 <file:fs/jbd2/>
 369
 370 programs:       http://e2fsprogs.sourceforge.net/
 371
 372 useful links:   http://fedoraproject.org/wiki/ext3-devel
 373                 http://www.bullopensource.org/ext4/
 374                 http://ext4.wiki.kernel.org/index.php/Main_Page
 375                 http://fedoraproject.org/wiki/Features/Ext4