A couple of minor ext3 performance tweaks
Posted by Steve on Mon 30 Apr 2007 at 06:42
The ext3 filesystem is probably the most common filesystem used upon GNU/Linux machines. It isn't necessarily the fastest, the best, or the most modern filesystem but it does perform adequately for the majority of users.
There are some people who wish to use the "fastest" filesystem around, regardless of any potential drawbacks ext3 probably isn't for them, but for the rest of us there are a couple of small tweaks you make to improve its performance.
In general there are two types of tweaks you can perform to your filesystem to improve performance:
- Change the options used by the filesystem itself.
- Change the way you mount your filesystem.
The ext3 filesystem is the successor to the ext2 filesystem, the version number changed primarily because of the addition of journalling.
What does this mean? Well if you go back a few years you'll discover that there are two ways to create an ext3 filesystem:
- Run mkfs.ext3 to create a new filesystem "from scratch".
- Create an ext2 filesystem and then add journalling to it - essentially converting it to ext3.
The process of adding the journal support to an existing ext2 filesystem is very simple (and safe!):
root@mine:~# tune2fs -j /dev/sda1
What a lot of people don't realise is that you can also use the tun2fs program to adjust other parameters of the filesystem - and this is what do.
The most useful tweak you can perform is the way that directory indexes are scanned when looking for files. This is controlled by the option "dir_index". By default this wouldn't be enabled but you can add it by running:
mine:~# tune2fs -O dir_index /dev/sda1
Once you've done this you'll be able to see the updated filesystem flags which are in use:
mine:~# tune2fs -l /dev/sda1 | grep features Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file
Once you've done this you should find that listing the contents of directories with large numbers of files becomes faster, and that finding files in directories is also better.
The final thing that we'll mention with regard to ext3 filesystem improvements is the mount option noatime.
When you mount a filesystem at bootup the options which are used are read from /etc/fstab. (I'm ignoring the case where you mount things manually!)
An entry in the fstab file usually looks something like this:
/dev/sda1 / ext3 defaults,errors=remount-ro 0 1
Here there are fields which specify what is mounted, and where it will be mounted. The fourth column contains the options which should be passed to the mount command. In this case we mount the filesystem with the two flags "defaults" (ie. nothing special) and "errors=remount-ro", which instructs the system to remount the filesystem read-only in the case of errors.
One speedup can be gained by adding noatime. This instructs the system not to update the access-time of any files which are accessed. There can be drawbacks here which are worth noting, for example if you're using find to find files which have recently been accessed, etc, this will fail. But otherwise the benefit is that reading files will not require the access time to be updated, not a huge speed-gain, but certainly noticeable for many systems.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Another tweaks:
1) The mount option data=journal|ordered|writeback.
Man mount says:
data=journal / data=ordered / data=writeback
Specifies the journalling mode for file data. Metadata is always journaled. To use modes other than ordered on the root file system, pass the mode to the kernel as boot parameter, e.g. rootflags=data=journal.
journal
All data is committed into the journal prior to being written into the main file system.
ordered
This is the default mode. All data is forced directly out to the main file system prior to its meta data being committed to the journal.
writeback
Data ordering is not preserved - data may be written into the main file system after its metadata has been committed to the journal. This is rumoured to be the highest-throughput option. It guarantees internal file system integrity, however it can allow old data to appear in files after a crash and journal recovery.
2) You can have an external journal on a faster disk or on some nvram.
Bence Romsics
[ Parent | Reply to this comment ]
how to deploy please? any directions or manual?
[ Parent | Reply to this comment ]
To create an external Ext3 log, run the mkfs utility on the journal device, making sure that the block size of the external journal is the same block size as the Ext3 file system. For example, the commandsââ¬Â¦
$ mkfs.ext3 ââ¬âb 4096 ââ¬âO journal_dev /dev/hda1
$ mkfs.ext3 ââ¬âb 4096 ââ¬âJ device=/dev/hda1 /dev/hdb1
ââ¬Â¦ /dev/hda1 is used as the external log for the Ext3 file system on /dev/hdb1.
Today I felt like sharing my toys with the other children ;)
[ Parent | Reply to this comment ]
You can use fsck -fD on the (unmounted!) FS to create them manually (add -C0 to get a neat progress bar).
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
No, but copying it and moving the copy back to the original position will create the b-tree index.
[ Parent | Reply to this comment ]
add nodiratime and reservation to your /etc/fstab
reservation is supported starting from 2.6.13
also you don't need reboot system on each update in fstab
just perform something like: mount -o remount /
and then check with 'mount' if settings been applied
use it on your own risk;)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
to files in the same directory, consider turning
preallocation on:
# mount -o reservation
[ Parent | Reply to this comment ]
Be aware that if you add features to an ext3 file system you may not be able to read the file system with an older kernel/distro as I found: http://www.debian-administration.org/tip/58
--
"It's Not Magic, It's Work"
Adam
[ Parent | Reply to this comment ]
echo 5 > /proc/sys/vm/dirty_background_ratio
http://lkml.org/lkml/2007/4/27/289
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
sudo bash -c 'echo text > file'
[ Parent | Reply to this comment ]
echo text | sudo tee file
"tee" redirects the output to the file, and it runs as superuser.
[ Parent | Reply to this comment ]
I know this is a different topic but I ask it anyway...
Did anybody try ext3cow? Any practical experience?
cb
[ Parent | Reply to this comment ]
I have a mail server that is using ext3 software raid1, and I want to enable dir_index.
Any one know if I should enable the index on the single drives that make up the raid1 mirror (/dev/sda,/dev/sdb), the software raid partition (/dev/md*), or both?
Thanks in advance
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
You need to do it on the raid volume - after all that is what you'd have run mkfs upon..
[ Parent | Reply to this comment ]
[1] http://packages.debian.org/unstable/misc/popularity-contest
[ Parent | Reply to this comment ]