mdadm is a program that can be used to create, manage, and monitor MD devices.
The name is derived from the md (multiple device) device nodes it administers or manages, and it replaced a previous utility mdctl. The original name was “Mirror Disk”, but was changed as the functionality increased.
As such it provides a similar set of functionality to the raidtools packages. The key differences between mdadm and raidtools are:
- mdadm is a single program and not a collection of programs.
- mdadm can perform (almost) all of its functions without having a configuration file and does not use one by default. Also mdadm helps with management of the configuration file.
- mdadm can provide information about your arrays (through Query, Detail, and Examine) that raidtools cannot. mdadm does not use /etc/raidtab, the raidtoolsconfiguration file, at all. It has a different configuration file with a different format and an different purpose.
RAID devices are virtual devices created from two or more real block devices. This allows multiple devices (typically disk drives or partitions there-of) to be combined into a single device to hold (for example) a single filesystem. Some RAID levels include redundancy and so can survive some degree of device failure.
Linux Software RAID devices are implemented through the md (Multiple Devices) device driver.
Currently, Linux supports LINEAR md devices, RAID0 (striping), RAID1 (mirroring), RAID4,RAID5, RAID6, and MULTIPATH.
MULTIPATH is not a Software RAID mechanism, but does involve multiple devices.
For MULTIPATH each device is a path to one common physical storage device.
1. Howto mount existing raid
An raid configuration is usually saved into mdadm.conf file (
/etc/mdadm.conf on redhat or
/etc/mdadm/mdadm.conf on debian).
In order to scan all the raid devices and generate a new configuration file we can use something like this
mdadm --detail --scan>>
2. How to create a new raid array
In this example we created a mirror array between /dev/sda1 and /dev/sdb1
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]1
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
You can also create an degraded array with a ‘missing’ drive that can be added latter
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 missing
and you can add the missing drive with (–manage can be ommitted)
mdadm –manage /dev/md0 –add /dev/sdb1
Note that ideally the drives should:
– not contain any raid information in the superblock (like if they were used in another array), you can clean the old data using :
mdadm –zero-superblock /dev/sdb
– be identical (in size), but if you have drives with different sizes or you want to grow your array size by adding bigger drives you still can create an array by using the –force option.
– be identical in the way they are partitioned, you can “dump” the partion from one drive (sda) to a new drive (sdb) with the following command.
sfdisk -d /dev/sda | sfdisk /dev/sdb
in this case I like to also use partprobe to inform the OS of partition table change
3. Howto remove a drive from an array
In order to remove a drive from the array, the drive needs to me marked as “failed”.
mdadm –fail /dev/md0 /dev/sda1
then we can remove it
mdadm –remove /dev/md0 /dev/sda1
We can do this also in a single step:
mdadm /dev/md0 –fail /dev/sda1 –remove /dev/sda1
4. Howto stop/start and array
An array can be stopped only if it is not mounted.
To stop an array :
mdadm –stop /dev/md0
To start an array (even if fewer drives were given than are needed for a full array):
mdadm –run /dev/md0
5. To remove (delete) an array
mdadm –remove /dev/md0
6. To see to status of an array durring build/rebuild
The array information can be displayed by :
or to have a constant update every 10 seconds
watch -n 10 ‘cat /proc/mdstat’
for more information about a particular md device you can see all the information in detail:
mdadm –detail /dev/md0
this may be usefull to see details about drives in degraded arrays.
7. To limit/increase the resync speed
– setup a minimum speed (10000k)
echo 10000 > /proc/sys/dev/raid/speed_limit_min
-to limit the maximum speed to 15000k (usually used to reduce the system stress during rebuilds)
echo 15000 > /proc/sys/dev/raid/speed_limit_max
the can be setup using sysctl also
sysctl -w dev.raid.speed_limit_max=15000
to have a permanent limit you can edit /etc/sysctl.conf and add “dev.raid.speed_limit_min=10000” or “dev.raid.speed_limit_max=15000”, folowed by a
sysctl -a -f /etc/sysctl.conf
8. Force the scsi bus to rescan
On some servers, after you plug the new drive it will not show up. You can force your scsi but to rescan.
– find your host id from dmesg.
dmesg | grep sdd | grep Attached
it will return something like:
sd 3:0:0:0: Attached scsi disk sdd
– now try to force a rescan rescan
echo ‘0 0 0’ > /sys/class/scsi-host/host3/scan
– you can also force a device to me removed
echo x > /sys/bus/scsi/devices/3:0:0:0/delete
9. Finding a drive
A nice trick to find a drive is to create ‘fake’ traffic on it and then look on the drive lights. I do it like this :
dd if=/dev/sdg of=/dev/null