Home > HA > OCFS2 + ISCSI Centralized storage in Ubuntu 10.10

OCFS2 + ISCSI Centralized storage in Ubuntu 10.10

In this article I will mention the steps to mount an ISCSI target on two ubuntu machines and then cluster it using oracle clustered file system. The newly mounted partition can be used as a centralized storage location in High availability, Failover or load-balancing setup.

The step by step howto is provided below,

1) Setup an ISCSI Server using Openfiler and create SAN LUN, and assign the IP to it.

For setting up openfiler based ISCI target, you can refer steps 1 to 8 mentioned in the url pasted below.

2) Setup two servers with Ubuntu 10.10 in it.

server1 ==> name: heuristics –> IP
server2 ==> name: heuristics2 –> IP

3) Install open-iscsi tool in both servers

apt-get install open-iscsi open-iscsi-utils
/etc/init.d/open-iscsi start

4) List out the ISCSI targets available in both servers.

iscsiadm -m discovery -t sendtargets -p

In my case the above command produced the following output,

root@heuristics2:~# iscsiadm -m discovery -t sendtargets -p,1 iqn.2006-01.com.openfiler:tsn.0d0c0c810c57

5) Mount the ISCSI target “iqn.2006-01.com.openfiler:tsn.0d0c0c810c57″(Lets call it TG57) to the local machine

iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.0d0c0c810c57 -p –login

6) Step5 will make the iscsi target TG57 as a device to the system. Which can be viewed as a block device.

root@heuristics# fdisk -l

Disk /dev/sdb: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003f05c

Device Boot Start End Blocks Id System
/dev/sdb1 1 1011 1047383 83 Linux

7) Install OCFS2 – Oracle Cluster File System for Linux

apt-get install ocfs2 ocfs2-tools

8) Configure OCFS2

Create a configuration file with proper indendation and copy it to both the servers. In my case “ocfs2” is the cluster name.

root@heuristics:~# cat /etc/ocfs2/cluster.conf



If proper indentation is not provided the following error will be shown,

Starting cluster oracle: Failed
o2cb_ctl: Unable to load cluster configuration file “/etc/ocfs2/cluster.conf”
Stopping cluster oracle: Failed
o2cb_ctl: Unable to load cluster configuration file “/etc/ocfs2/cluster.conf”

9) Start the cluster service in both the machines

/etc/init.d/ocfs2 start
/etc/init.d/o2cb start

10) Create one partition named /dev/sdb1 in the ISCSI target

fdisk /dev/sdb

11) Make ocfs cluster file system using the following command(need to execute only on one machine)

mkfs.ocfs2 -b 4k -C 32k -N3 -L cluster-storage /dev/sdb1

This creates a file system with 4096 block size and 32768 (32k) cluster size.
NOTE: N= 3 , for a cluster with 2 machines, N=3 and for a cluster with ‘n’ machines N=(n+1)

12) Update partition table on all servers in the cluster. In this case all the servers have /dev/sdb as the iSCSI target.

We will run the following to re-read the partition:

blockdev –rereadpt /dev/sdb

Next, we will want to create a mount point on the servers for this cluster.

mkdir /mycluster

Mount the partition,

mount -L cluster-storage /cluster-storage

13) Show results and test

root@heuristics2:~# hostname
root@heuristics2:~# df -TH /cluster-storage/
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 ocfs2 1.1G 330M 744M 31% /cluster-storage
root@heuristics:~# hostname
root@heuristics:~# df -TH /cluster-storage/
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 ocfs2 1.1G 330M 744M 31% /cluster-storage



Categories: HA
  1. Shaheer
    September 27, 2011 at 5:00 pm

    your heading is wrong OCSF2 + ISCSI Centralized storage in Ubuntu 10.10

    i think it should be OCFS2 + ISCSI Centralized storage in Ubuntu 10.10

    • September 28, 2011 at 2:47 am

      Thanks Shaheer, I have changed it:)

  2. Shaheer
    September 27, 2011 at 5:01 pm

    nice tutorial
    i am going to try it

  3. April 3, 2013 at 11:16 pm

    nice, i never knew how to do that!!! sweet, its just like multifunctional printers, well at least i can fix those, but you are good at obuntu!!!

  4. Rafael Campoverde
    February 25, 2015 at 5:59 am

    wow nice tutorial!!, thank you very mutch.

    I have a question, if I have tow luns (lun0 and lun1) shared by ISCSI, how can I do the same procedure for the two at same time, how to configure /etc/ocfs/cluster.conf to manage /dev/sda1 <- (for lun0) and /dev/sdb1 <- (for lun1) you know if that is possible?

    Thanks again!!

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: