Optimizing readwrite block size

The default read and write block size for NFS is 4096 bytes (4KB), which may not be optimal for all situations. You can perform a test to determine whether changing the block size will help you or not. Here's how you perform such a test.

This test assumes that you have an NFS server running on a Linux system and also have a Linux-based NFS client system. The test also assumes that the client mounts a filesystem called /mnt/nfs1 from the NFS server.

1. Su to root on the NFS client machine.

2. You need to know the total amount of memory your system has. You should know this by default because it's your system, but if you don't remember too well, you can run the cat /proc/meminfo command to view the memory information for your system. This will display similar to what is shown below:

total: used: free: shared: buffers: cached:

Mem: 263720960 260456448 3264512 30531584 228245504 6463488

Swap: 271392768 6209536 265183232

MemTotal:

257540

kB

MemFree:

3188

kB

MemShared:

29816

kB

Buffers:

222896

kB

Cached:

6312

kB

BigTotal:

0

kB

BigFree:

0

kB

SwapTotal:

265032

kB

SwapFree:

258968

kB

3. The total amount of system memory is shown under the column heading total:; divide this number by 1,048,576 (1024x1024) to get the total (approximate) memory size in megabytes. In the preceding example, this number is 251MB.

Interestingly, total memory is never reported accurately by most PC system BIOS, so you must round the number based on what you know about the total memory. In my example, I know that the system should have 256MB of RAM, so I use 256MB as the memory size in this test.

If you have RAM > 1GB, I recommend using 512MB as the RAM size for this experiment. Although you may have 1GB+ RAM, pretend that you have 512MB for this experiment.

4. Change directory to a currently mounted /mnt/nfsl NFS file directory. Run the du command to see whether you have at least 512MB (2 x total RAM) of free space available on the NFS directory. If you don't, you can't continue with this experiment. I assume that you do have such space available.

5. We want to measure the write performance of your current NFS setup. So we will write a 512MB (16KB/block x 32,768 blocks) file called 512MB.dat in the /mnt/nfsl directory using the following command:

of=/mnt/nfs1/512MB.dat \ bs=16k count=32768

This command runs the time command, which records execution time of the program named as the first argument. In this case, the dd command is timed. The dd command is given an input file (using if option) called /dev/zero. This file is a special device that returns a 0 (zero) character when read. If you open this file for reading, it keeps returning a 0 character until you close the file. This gives us an easy source to fill out an output file (specified using the of option) called /mnt/nfs1/512MB.dat; the dd command is told to use a block size (specified using bs option) of 16KB and write a total of 32,768 blocks (specified using the count option). Because 16KB/block times 32,768 blocks equal 512MB, we will create the file we intended. After this command is executed, it prints a few lines such as the following:

32768+0 records in 32768+0 records out

Here the dd command read 32,768 records from the /dev/zero device and also wrote back the same number of records to the /mnt/nfs1/512MB.dat file. The third line states that the copy operation took one minute and 58.91 seconds. Write this line in a text file as follows:

Here, you are noting that this was the first (1st) write experiment.

6. We need to measure the read performance of your current NFS setup. We can simply read the 512MB file we created earlier and see how long it takes to read it back. To read it back and time the read access, you can run the following command:

time dd if=/mnt/nfs1/512MB.dat \ of=/dev/null \ bs=16k count=32768

Here the dd command is timed again to read the /mnt/nfs1/512MB.dat file as input, then output the file contents to /dev/null, which is the official bottomless bit bucket for Linux. Like before, record the time used in the same file you wrote the read performance record. For example, the read test using the preceding command displayed the following output on my system.

Record the third line as follows:

Here, you are noting that this was the first (1st) read experiment.

7. Remove the 512MB.dat file from /mnt/nfsl and umount the partition using the umount /mnt/nfsl command. The unmounting of the NFS directory ensures that disk caching doesn't influence your next set of tests.

8. Repeat the write and read back test (Steps 5 - 7) at least five times. You should have a set of notes as follows:

Read,

1,

1

971u,

38

970s,

2

10

44,

31

3%

Read,

2,

1

973u,

38

970s,

2

10

49,

31

3%

Read,

3,

1

978u,

38

971s,

2

10

49,

31

3%

Read,

4,

1

978u,

38

971s,

2

10

49,

31

3%

Read,

5,

1

978u,

38

971s,

2

10

49,

31

3%

Write,

1,

1

610u,

71

800s,

1

58

91,

61

7%

Write,

2,

1

610u,

71

801s,

1

58

92,

61

7%

Write,

3,

1

610u,

71

801s,

1

58

92,

61

7%

Write,

4,

1

610u,

71

801s,

1

58

92,

61

7%

Write,

5,

1

611u,

71

809s,

1

58

92,

61

7%

9. Calculate the average read and write time from the fifth column (shown in bold).

You have completed the first phase of this test. You have discovered the average read and write access time for a 512MB file. Now you can start the second phase of the test as follows:

1. Unmount the /mnt/nfsl directory on the NFS client system using the umount /mnt/nfs1 command.

2. Modify the /etc/fstab file on the NFS client system such that the /mnt/nfsl filesystem is mounted with the rsize=8192, wsize=8192 options as shown below:

nfs-server-host:/nfs1 /mnt/nfsl nfs \ rsize=8192, wsize=8192 0 0

3. Mount the /mnt/nfsl directory back using the mount /mnt/nfsl command.

4. Perform Steps 4 to 9 of the previous experiment.

5. Compare the read and write access averages between phase 1 and phase 2 of the test. If the results in phase 2 (this part) of the test looks better, the changing of the read and write blocks have increased your NFS performance. If not, remove the rsize=8192, wsize=8192 options from the line in /etc/fstab. Most likely, the read and write block size change will increase NFS performance. You can also experiment with other block sizes. It's advisable that you use multiples of 1024 in block size because 1024 is the actual filesystem block size. Also, don't use larger numbers above 8192 bytes.

If the block size change works for you, keep the rsize=8192, wsize=8192 (or whatever you find optimal via further experiment) in the /etc/fstab line for the /mnt/nfs1 definition.

Was this article helpful?

0 0

Post a comment