The CCD driver allows the user to "concatenate" several physical disks into one pseudo volume. CCD also lets you overcome a feature limitation in CMU RAIDFrame that does not allow you to RAID0 (file system spanning across disks) across disks of different geometry. CCD also allows for an "interleave" to improve disk performance with a gained space loss. This example will not cover that feature.
The steps required to setup a CCD are as follows:
Install physical media
Configure kernel support
Disklabel each volume member of the CCD
Configure the CCD conf file
Initialize the CCD device
Create a 4.4BSD/UFS filesystem on the new CCD device
Mount the CDD filesystem
This example features a CCD setup on NetBSD/sparc 1.5. The CCD will reside on 4 SCSI disks in a generic external Sun disk pack chassis connected to the external 50 pin SCSI port.
This step is at your own discretion, depending on your platform and the hardware at your disposal.
From my DMESG:
Disk #1: probe(esp0:0:0): max sync rate 10.00MB/s sd0 at scsibus0 target 0 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed sd0: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors Disk #2 probe(esp0:1:0): max sync rate 10.00MB/s sd1 at scsibus0 target 1 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors Disk #3 probe(esp0:2:0): max sync rate 10.00MB/s sd2 at scsibus0 target 2 lun 0: <SEAGATE, ST11200N SUN1.05, 9500> SCSI2 0/direct fixed sd2: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors Disk #4 probe(esp0:3:0): max sync rate 10.00MB/s sd3 at scsibus0 target 3 lun 0: <SEAGATE, ST11200N SUN1.05, 8808 > SCSI2 0 sd3: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
The following kernel configuration directive is needed to provide CCD device support. It is enabled in the GENERIC kernel:
pseudo-device ccd 4 # concatenated disk devices
In my kernel config, I also hard code SCSI ID associations to /dev device entries to prevent bad things from happening:
sd0 at scsibus0 target 0 lun ? # SCSI disk drives sd1 at scsibus0 target 1 lun ? # SCSI disk drives sd2 at scsibus0 target 2 lun ? # SCSI disk drives sd3 at scsibus0 target 3 lun ? # SCSI disk drives sd4 at scsibus0 target 4 lun ? # SCSI disk drives sd5 at scsibus0 target 5 lun ? # SCSI disk drives sd6 at scsibus0 target 6 lun ? # SCSI disk drives
Each member disk of the CCD will need a special file system established. In this example, I will need to disklabel:
/dev/rsd0c
/dev/rsd1c
/dev/rsd2c
/dev/rsd3c
Note: always remember to disklabel the character device, not the block device, in /dev/r{s,w}d*
Note: on all platforms, the cslice is symbolic of the entire NetBSD partition and is reserved.
You will probably want to remove any pre-existing disklabels on the disks in the CCD. This can be accomplished one of two ways the dd command:
# dd if=/dev/zero of=/dev/rsd0c bs=8k count=1 # dd if=/dev/zero of=/dev/rsd1c bs=8k count=1 # dd if=/dev/zero of=/dev/rsd2c bs=8k count=1 # dd if=/dev/zero of=/dev/rsd3c bs=8k count=1
If your port uses a MBR (Master Boot Record) to partition the disks so that the NetBSD partitions are only part of the overall disk, and other OSs like Windows or Linux use other parts, you can void the MBR and all partitions on disk by using the command:
# dd if=/dev/zero of=/dev/rsd0d count=1
This will make all data on the entire disk unaccessible.
The default disklabel for the disk will look similar to this:
# disklabel -r /dev/rsd0c [...snip...] bytes/sector: 512 sectors/track: 116 tracks/cylinder: 9 sectors/cylinder: 1044 cylinders: 3992 total sectors: 4197405 [..snip...] 3 partitions: # size offset fstype [fsize bsize cpg] c: 4197405 0 unused 1024 8192 # (Cyl. 0 - 4020*)
You will need to create one "slice" on the NetBSD partition of the disk that consumes the entire partition. The slice must begin at least one cylinder offset from the beginning of the disk/partition to provide space for the special CCD disklabel. The offset should be 1x sectors/cylinder (see following note). Therefore, the "size" value should be "total sectors" minus 1x "sectors/cylinder".
Note: the offset of a slice of type "ccd" must be a multiple of the "sectors/cylinder" value.
Edit your disklabels accordingly. Be sure to specify the path to the character device, not the block device.
Note: be sure to export EDITOR=[path to your favorite editor] before editing the disklabels.
# disklabel -e /dev/rsd0c
Note: the slice must be fstype ccd.
Because there will only be one slice on this partition, you can recycle the c slice (normally reserved for symbolic uses). Change your disklabel to the following:
3 partitions: # size offset fstype [fsize bsize cpg] c: 4196361 1044 ccd # (Cyl. 1 - 4020*)
Optionally you can setup a slice other than c to use, simply adjust accordingly below:
3 partitions: # size offset fstype [fsize bsize cpg] a: 4196361 1044 ccd # (Cyl. 1 - 4020*) c: 4197405 0 unused 1024 8192 # (Cyl. 0 - 4020*)
Be sure to write the label when you have completed. Disklabel will object to your disklabel and prompt you to re-edit if it does not pass it's sanity checks.
Once all disk are properly labeled, you will need to generate a configuration file. The configuration file resides in /etc by default. You may need to create a new one. Format:
#ccd ileave flags component devices
Note: for the "ileave", if a value of zero is used then the disks are concatenated, but if you use a value equal to the "sectors/track" number the disks are interleaved.
Example in this case:
# more /etc/ccd.conf ccd0 0 none /dev/sd0c /dev/sd1c /dev/sd2c /dev/sd3c
Note: the CCD configuration file references the device file for the newly created CCD filesystems. Be sure not to reference the block device at this point, instead use the character device.
Once you are confident that your CCD configuration is sane, you can initialize the device using the ccdconfig command: Configure:
# ccdconfig -c -f /etc/ccd.conf
Unconfigure:
# ccdconfig -u -f /etc/ccd.conf
Initializing the CCD device will activate /dev entries: /dev/{,r}ccd#:
# ls -la /dev/{,r}ccd0*
brw-r----- 1 root operator 9, 0 Apr 28 21:35 /dev/ccd0a
brw-r----- 1 root operator 9, 1 Apr 28 21:35 /dev/ccd0b
brw-r----- 1 root operator 9, 2 May 12 00:10 /dev/ccd0c
brw-r----- 1 root operator 9, 3 Apr 28 21:35 /dev/ccd0d
brw-r----- 1 root operator 9, 4 Apr 28 21:35 /dev/ccd0e
brw-r----- 1 root operator 9, 5 Apr 28 21:35 /dev/ccd0f
brw-r----- 1 root operator 9, 6 Apr 28 21:35 /dev/ccd0g
brw-r----- 1 root operator 9, 7 Apr 28 21:35 /dev/ccd0h
crw-r----- 1 root operator 23, 0 Jun 12 20:40 /dev/rccd0a
crw-r----- 1 root operator 23, 1 Apr 28 21:35 /dev/rccd0b
crw-r----- 1 root operator 23, 2 Jun 12 20:58 /dev/rccd0c
crw-r----- 1 root operator 23, 3 Apr 28 21:35 /dev/rccd0d
crw-r----- 1 root operator 23, 4 Apr 28 21:35 /dev/rccd0e
crw-r----- 1 root operator 23, 5 Apr 28 21:35 /dev/rccd0f
crw-r----- 1 root operator 23, 6 Apr 28 21:35 /dev/rccd0g
crw-r----- 1 root operator 23, 7 Apr 28 21:35 /dev/rccd0h
You may now disklabel the new virtual disk device associated with your CCD. Be sure to use the character device:
# disklabel -e /dev/rccd0c
Once again, there will be only one slice, so you may either recycle the c slice or create a separate slice for use.
# disklabel -r /dev/rccd0c # /dev/rccd0c: type: ccd disk: ccd label: default label flags: bytes/sector: 512 sectors/track: 2048 tracks/cylinder: 1 sectors/cylinder: 2048 cylinders: 6107 total sectors: 12508812 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # microseconds track-to-track seek: 0 # microseconds drivedata: 0 # size offset fstype [fsize bsize cpg] c: 12508812 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 6107*)
The filesystem will then need formatted:
# newfs /dev/rccd0c
Warning: 372 sector(s) in last cylinder unallocated
/dev/rccd0c: 12508812 sectors in 6108 cylinders of 1 tracks, 2048 sectors
6107.8MB in 382 cyl groups (16 c/g, 16.00MB/g, 3968 i/g)
super-block backups (for fsck -b #) at:
[...]
Once you have a created a file system on the CCD device, you can can mount the file system against an mount point on your system. Be sure to mount the slice labeled type ffs or 4.4BSD:
# mount /dev/ccd0c /mnt
Then:
# export BLOCKSIZE=1024; df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/sd6a 376155 320290 37057 89% / /dev/ccd0c 6058800 1 5755859 0% /mnt
Congratulations, you now have a working CCD. Please consult the rest of the manual for instructions on how to initialize the device at boot-time via RC. For more detail on any of the commands here, please see their respective man pages.