Wacky uses for Raid, /dev/ram, and ramfs

ArticleCategory:

AuthorImage:

TranslationInfo:[Author and translation history]

original in en Mark Nielsen

AboutTheAuthor:

Abstract:

RamFs is a very simple file system with some interesting aspects. It is a new things in the 2.4 Kernel.

ArticleIllustration:[This is the title picture for your article]

ArticleBody:

Introduction to RamDisk

What is a RamDisk? A RamDisk is a portion of memory that you allocate to use as a partition. Or, in other words, you take some of your memory and pretend it is a hard drive that you can format, mount, save files to etc.

There are two types of ramdisks we will be talking about. The first is a ramdisk that you can format and mount. The second is a ramdisk you cannot format, but has some nice features. Here is a breakdown:

The old /dev/ram1 approach:

1. BAD: The size of the ramdisk is fixed upon boot-up.
2. GOOD: You can format the ramdisk anyway you want.
3. BAD: Once formatted (I don't believe you have to mount it), it takes up a portion of your memory and I am not sure how you can get it back.
4. GOOD: You can have more than one ramdisk at a time.

1. BAD: I couldn't format it other than what it is set to. It is on the VFS layer, whatever that is.
2. GOOD: You can have more than one.
3. GOOD: You can change its maximum size at the line command.
4. VERY GOOD: It only uses up the memory that it needs. In other words, you may have allocated 64 megs of ram, but if it is only has 2k of files in it, it will only use up 2k of memory. As soon as a file is deleted you immediately have more memory for your computer.
5. BAD: Well, it isn't really its own fault. Don't run into the trap of filling of two ramfs ramdisks or you will blow away all your memory. More on this.

How to use RamDisk the old way

To use your kernel 2.0 and 2.2 ramdisks, just type three commands:

mkfs -t ext2 /dev/ram1
mkdir -p /RAM1
mount /dev/ram1 /RAM1

mkfs -t ext2 /dev/ram2
mkdir -p /RAM2
mount /dev/ram2 /RAM2

How to use RamDisk with ramfs for kernel 2.4

Very simple. Make a directory and mount ramfs there.

mkdir -p /RAM1
mount -t ramfs none /RAM1

mkdir -p /RAM2
mount -t ramfs none /RAM2

Changing the size of the ramdisks and other options.

Here is an example of how to make a 10 meg ramdisk.

mkdir -p /RAM1
mount -t ramfs none /RAM1 -o maxsize=10000

+       ramfs - An automatically resizing memory based filesystem
+
+
+  Ramfs is a file system which keeps all files in RAM. It allows read
+  and write access. In contrast to RAM disks, which get allocated a
+  fixed amount of RAM, ramfs grows and shrinks to accommodate the
+  files it contains.
+
+  You can mount the ramfs with:
+      mount -t ramfs none /mnt/wherever
+
+  Then just create and use files. When the filesystem is unmounted, all
+  its contents are lost.
+
+  NOTE! This filesystem is probably most useful not as a real
+  filesystem, but as an example of how virtual filesystems can be
+  written.
+
+Resource limits:
+
+By default a ramfs will be limited to using half of (physical) memory
+for storing file contents, a bit over that when the metadata is
+included. The resource usage limits of ramfs can be controlled with
+the following mount options:
+
+       maxsize=NNN
+               Sets the maximum allowed memory usage of the
+filesystem to NNN kilobytes. This will be rounded down to a multiple
+of the page size. The default is half of physical memory. NB.  unlike
+most of the other limits, setting this to zero does *not* mean no
+limit, but will actually limit the size of the filesystem data to zero
+pages. There might be a use for this in some perverse situation.
+       
+       maxfilesize=NNN
+               Sets the maximum size of a single file on the
+filesystem to NNN kilobytes. This will be rounded down to a multiple
+of the page size. If NNN=0 there is no limit. The default is no limit.
+
+       maxdentries=NNN
+               Sets the maximum number of directory entries (hard
+links) on the filesystem to NNN. If NNN=0 there is no limit. By
+default this is set to maxsize/4.
+
+       maxinodes=NNN
+               Sets the maximum number of inodes (i.e. distinct
+files) on the filesystem to NNN. If NNN=0 there is no limit. The
+default is no limit (but there can never be more inodes than dentries).

Running services in RamDisk

Ram is cheap these days. I got 128 megs of ram for $120 recently, and I imagine the price will continue to drop. For 128 megs of ram, I can easily fit any of my webservers into it. If my webserver has huge files, I can configure httpd.conf to look a another directory not in the ram disk. My DNS server is very small. My largest database server is only 28 megs. Considering the fact you can get 1 gig memory computers these days real cheap, I think it would be cool to run everything I have in ram. The only tricky part with the database server is that you should first save all writes to a separate database server before writing to your database server in ram. This is slow, but considering most of the time you are reading and not writing, it should increase performance. Perhaps if we could mirror a ram disk to a hard drive

I already explained this in my other article, so please read it here: http://www.gnujobs.com/mark/articles/Ramdisk.html#Example. Just replace these commands

        ### Make the ramdisk partitions
/sbin/mkfs -t ext2 /dev/ram0
/sbin/mkfs -t ext2 /dev/ram1
/sbin/mkfs -t ext2 /dev/ram2

        ### Mount the ramdisks to their appropriate places

mount /dev/ram0 /home/httpd/cgi-bin
mount /dev/ram1 /home/httpd/icons
mount /dev/ram2 /home/httpd/html

mount -t ramfs none /home/httpd/cgi-bin -o maxsize=10000
mount -t ramfs none /home/httpd/icons -o maxsize=10000
mount -t ramfs none /home/httpd/html -o maxsize=10000

Ramdisk and Raid

There are three problems combining a ramdisk with a hard drive partition.

1. I don't know how how to do it with ramfs, but I can do it with /dev/ram. Perhaps if you could format a hard drive partition with the filesystem ramfs uses, you could do it. I have no idea what the VFS layer is, so I will probably study more about it later.
2. If the mirrored raid is truly running in parallel, then the extra speed of the memory won't help. I don't know if it is possible to make a mirrored raid to only do reads from the ramdisk, in which case, if it is possible, there would be benefits.
3. Upon reboot, you will have to reconstruct the ramdisk.

  raiddev /dev/md3
          raid-level      1
          nr-raid-disks   2
          nr-spare-disks  0
          chunk-size     4
          persistent-superblock 1
          device          /dev/hdb6
          raid-disk       1
          device          /dev/ram3
          raid-disk       0

mkraid /dev/md3
mkfs -t ext2 /dev/md3
mkdir -p /RAID1
mount /dev/md3 /RAID1

raidstart /dev/md3
raidhotadd /dev/md3 /dev/ram3
mount /dev/md3 /RAID1

It would be really cool to have a mirrored raid with 3 partitions. 1 being the ramdisk and 2 being hard drive paritions and to also make it so only the ramdisk is read for read cycles. I don't know if it is possible to isolate reads to only one of the partitions in a raid, but it would be cool if you could.

Comments

1. Ramfs defaults to half your physical memory (no swap space). Thus if you create two of these things and fill them up, your computer will run out of memory and stall, just like mine did. PLEASE specify the max amount of memory the ramdisk should have as described above.
2. I believe ramfs is its own filesystem and not ext2, so there, if you copy files back and forth using rsync or tar, don't expect the files to be exactly identical or the same size. You most likely won't see any major difference in using them, but since it is not ext2, the properties might be different on some levels.
3. I was reading the docs for ramfs in the kernel and it doesn't seem like ramfs was really intended for wide use, but I don't see why. It works the way I want it.

1. I want to see if I can raid ramfs with a hard drive.
2. If I can make it so reads to a raid are isolated to one of the partitions.
3. Putting a service, like a webserver, into a ramdisk with raid. The trick here is to use raid so that if any changes occur, you don't loose them. That is why raid is so freaking important to me ---- I will be able to put services into ram SAFELY and gain all the performance speed of memory over hard drive assuming:
   • That I can get a raid to do reads only from the ramdisk partition and not the others.
   • Read files from a hard drive are cached in memory. Only if they change, or cached out, would you notice a difference in speed. Also, initial startup is faster the first time the file is read.
   • You could use the built-in webserver that come in the new kernels. They should go really really fast as well. Maybe combining the built-in webserver with a ramdisk?
4. Setting up a computer to boot from a cdrom, start a service, and download changes off the network. No hard drive required. This would be cool. They you could have a webserver (or something else) running off a computer with no hard drive and just a cdrom with memory. If you bump your memory to 1 gig, I don't see why you couldn't have your system run entirely off of a cdrom. This would be nice for security reasons. Just reboot the sucker, have it load any changes (security fixes) from a secured resource, then start the services, and your computer is nice and clean and hack-free. The webserver should never save anything to the local computer but save logs and information to a separate computer on a secured connection or internal network. I made bootable networked cdroms with ramdisks in the past. It would be nice to combine my earlier project to achieve this.

Resources

1. kernel/v2.4/patch/patch-2.4.3-ac2/linux.ac_Documentation_filesystems_ramfs.txt.html
2. http://genericbooks.com/LDP/HOWTO/Software-RAID-HOWTO.html
3. Speeding up small databases on Linux
4. How to use a Ramdisk for Linux
5. If this article changes, it will be available here: http://www.gnujobs.com/Articles/21/Ramdisk2.html

Thanks to Katja for some help!

Copyright © 4/2001 Mark Nielsen
Article Version 1.3 : Date Last Changed Sat May 19 06:04:06 2001