0. Brief description
We will install 2-node RAC for testing purposes. With OS Oracle Enterprise Linux 64bit. Database version 11g Release 2(11.2.0.1.0) 64bit. Grid version also will be 11hR2.
1. Pre-requisites
– Disk space:
— 30GB for root directory
— 2.3GB for database
— 2.95GB for Oracle Grid Infrastructure
— approx. 4GB for swap
– Memory
–4GB
– Installation Software
Enterprise Linux:https://edelivery.oracle.com/linux
Database: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/112010-linx8664soft-100572.html (Choose database)
Grid:http://www.oracle.com/technetwork/products/clusterware/downloads/index.html (Choose Linux x86-64)
2. Installation
2.1 Configuring Servers













After creation edit virtual machine settings. Specify Enterprise Linux

Click OK. Open the console. And power on the machine.









Note swap size should be chosen by the following way:
1. If you have RAM between 1 and 2G, swap should be 1.5 x RAM (1.5 times RAM) .
2. For RAM 2 to 8G , swap space has to be equal to RAM.
3. For RAM more than 8G, swap needs to be 3/4 RAM.
Because of my RAM is 4GB , I choose 4GB of swap space.
I left logical volumes because , increasing disk space with it is easier than with physical volumes.







Desktop Environments ->GNOME Desktop Environment

Applications-> Editors, Graphical Internet, Text-based Internet

Developement->check all except KDE and Ruby

Click Development Tools and Optional packages button: check gcc44-4.4, gcc44-c++.
Click X Software Development and Optional packages button: check openmotif-devel.
Servers-> Server Configuration Tools

Base System-> Administration Tools, Base, Java, Legacy Software Support, System Tools, X Window System

Click Legacy Software Support and Optional packages button and choose: compat-db, openmotif22.
Click System Tools and Optional packages button and choose: 1:mc, oracleasm*, sysstat.















2.2 Installing Guest additions

From command line:
mkdir /0
cp /media/VMware\ Tools/VMwareTools-8.3.2-257589.tar.gz /0
cd /0
tar -zxf VMwareTools-8.3.2-257589.tar.gz
cd vmware-tools-distrib/
./vmware-install.pl
After installation completes , open new terminal and run:
vmware-toolbox
2.2.1 Enabling copy/paste on vSphere
For the following step server should be shutdowned:
“Starting with vSphere 4.1, the Copy and Paste options are, by default, disabled for security reasons.
To be able to copy and paste between the guest operating system and the remote console, you must enable the Copy and Paste options using the vSphere Client. Alternatively, you can use RDP (Remote Desktop Protocol) to connect to the Windows virtual machines.
To enable this option for a specific virtual machine:
1. Log into a vCenter Server system using the vSphere Client and power off the virtual machine.
2. Select the virtual machine and click the Summary tab.
3. Click Edit Settings.
4. Navigate to Options > Advanced > General and click Configuration Parameters.
5. Click Add Row.
6. Type these values in the Name and Value columns:
Name Value
isolation.tools.copy.disable false
isolation.tools.paste.disable false
Note: These options override any settings made in the VMware Tools control panel of the guest operating system.
1. Click OK to close the Configuration Parameters dialog, and click OK again to close the Virtual Machine Properties dialog.
2. Power on the virtual machine.“ source is from http://kb.vmware.com
2.3 Preparing Environment for Oracle Grid infrastructure
2.3.1 Install necessary RPM packages
For this we will need Oracle Enterprise Linux disk.

[root@node1 ~]# cp -R /media/Enterprise\ Linux\ dvd\ 20100405/Server /0
Note: Server folder contains necessary rpms. Most of rpms may already been installed.
cd /0/Server
rpm -Uvh binutils-2.17.50.0.6-14.el5.x86_64.rpm
rpm -Uvh compat-libstdc++-33-3.2.3-61.x86_64.rpm
rpm -Uvh elfutils-libelf-devel-0.137-3.el5.x86_64.rpm
rpm -Uvh elfutils-libelf-devel-static-0.137-3.el5.x86_64.rpm
rpm -Uvh gcc-4.1.2-48.el5.x86_64.rpm
rpm -Uvh gcc-c++-4.1.2-48.el5.x86_64.rpm
rpm -Uvh glibc-devel-2.5-49.x86_64.rpm
rpm -Uvh glibc-headers-2.5-49.x86_64.rpm
rpm -Uvh kernel-headers-2.6.18-194.el5.x86_64.rpm
rpm -Uvh libacl-devel-2.2.39-6.el5.x86_64.rpm
rpm -Uvh libaio-devel-0.3.106-5.i386.rpm
rpm -Uvh libaio-0.3.106-5.x86_64.rpm
rpm -Uvh libaio-0.3.106-5.i386.rpm
rpm -Uvh libaio-devel-0.3.106-5.x86_64.rpm
rpm -Uvh libgomp-4.4.0-6.el5.x86_64.rpm
rpm -Uvh libstdc++-devel-4.1.2-48.el5.x86_64.rpm
rpm -Uvh libstdc++-devel-4.1.2-48.el5.x86_64.rpm
rpm -Uvh sysstat-7.0.2-3.el5.x86_64.rpm
rpm -Uvh unixODBC-2.2.11-7.1.x86_64.rpm
rpm -Uvh unixODBC-2.2.11-7.1.i386.rpm
rpm -Uvh unixODBC-devel-2.2.11-7.1.x86_64.rpm
rpm -Uvh unixODBC-devel-2.2.11-7.1.i386.rpm
rpm -Uvh ksh-20100202-1.el5.x86_64.rpm
2.3.3 Modify /etc/hosts file
vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost
#Public
192.168.34.150 node1
192.168.34.151 node2
#Private
10.10.2.50 node1-priv
10.10.2.51 node2-priv
#Virtual
192.168.34.154 node1-vip
192.168.34.155 node2-vip
#SCAN
192.168.34.160 rac-scan
SCAN should be defined on DNS. But for now we will use hosts file.
2.3.4 Modify kernel parameters in /etc/sysctl.conf
vi /etc/sysctl.conf
# Controls the maximum shared segment size, in bytes
#kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
#kernel.shmall = 4294967296
#Added by DBA
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 1048576
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 10448576
To make changes take effect :
/sbin/sysctl -p
2.3.5 Modify /etc/security/limits.conf
vi /etc/security/limits.conf
#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student – maxlogins 4
#Added By DBA
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
# End of file
2.3.6 Modify /etc/pam.d/login
vi /etc/pam.d/login
#%PAM-1.0
auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so
auth include system-auth
account required pam_nologin.so
account include system-auth
password include system-auth
# pam_selinux.so close should be the first session rule
session required pam_selinux.so close
session include system-auth
session required pam_loginuid.so
session optional pam_console.so
# pam_selinux.so open should only be followed by sessions to be executed in the user context
session required pam_selinux.so open
session optional pam_keyinit.so force revoke
session required pam_limits.so
2.3.7 Deconfigure NTP
[root@node1 Server]# service ntpd stop
Shutting down ntpd: [FAILED]
[root@node1 Server]# chkconfig ntpd off
[root@node1 Server]# mv /etc/ntp.conf /etc/ntp.conf.backup
[root@node1 Server]# rm /var/run/ntpd.pid
rm: cannot lstat `/var/run/ntpd.pid’: No such file or directory
2.3.8 Add necessary Groups and User, and create directories.
#Creating groups for Grid Infrastructure
groupadd asmadmin
groupadd asmdba
groupadd asmoper
#Creating groups for Oracle Software
groupadd oinstall
groupadd dba
groupadd oper
#Creating user for Grid Infrastructure
useradd -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid grid
#Creatnig user for Oracle Software
useradd -g oinstall -G dba,oper,asmdba -d /home/oracle oracle
#Setting password for users
passwd grid
passwd oracle
#Creating necessary directories
mkdir -p /u01/app/grid
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01
2.3.9 Edit .bash_profile
#For Oracle user
su – oracle
vi .bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
ORACLE_SID=orcl1; export ORACLE_SID
ORACLE_UNQNAME=orcl1; export ORACLE_UNQNAME
JAVA_HOME=/usr/local/java; export JAVA_HOME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”
export NLS_DATE_FORMAT
TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN
ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH
THREADS_FLAG=native; export THREADS_FLAG
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
#For Grid user
su – grid
vi .bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
ORACLE_SID=+ASM1; export ORACLE_SID
JAVA_HOME=/usr/local/java; export JAVA_HOME
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”; export NLS_DATE_FORMAT
TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN
ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH
THREADS_FLAG=native; export THREADS_FLAG
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
2.3.10 Shutdown NODE1. Using clone, make NODE2 and configure it.
su –
[root@rac1 ~]# init 0





Open node2 console and run:
system-config-network

Delete .bak entries:

In DNS tab, change node1 to node2

In Devices tab, edit eth0 by the following way:

In the same window, in Hardware Device tab click Probe.

Click OK. Edit eth1.

In the same window, in Hardware Device tab click Probe.

Click OK. Activate eth0 and eth1 devices, Save(Ctrl+S) and Quite(Ctrl+Q)

Open terminal and run:
[root@node2 ~]# service network restart
Shutting down interface eth0: [ OK ]
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: [ OK ]
Bringing up interface eth1: [ OK ]
Edit bash profiles:
#For oracle bash_profile change the following lines:
ORACLE_SID=orcl2; export ORACLE_SID
ORACLE_UNQNAME=orcl2; export ORACLE_UNQNAME
#For grid bash_profile change the following line:
ORACLE_SID=+ASM2; export ORACLE_SID
2.3.11 Add necessary shared disks.
Shutdown both nodes.
init 0
Edit virtual machine settings and add hardware:



Choose SCSI (1:0), different from SCSI specified for 30GB hard disk.


Choose Virtual for New SCSI Controller:

Repeat the steps 2 more times. So create 2 more disks with 10GB space and 1 with 3GB for CRS.
The second hard disk should look like this. Note , choose SCSI(1:1).



Now configure node2: Edit virtual machine settings and add hard disk






Do the same steps for other disks, just choose SCSI(1:1) for the first ,SCSI(1:2) for the second and SCSI(1:3) for the third :



Power on both machines.
Open terminal in node1 server and partition newly added disks.(You don’t need to do the same from node2 because, disks are shared and node2 will see partitions created by node1, but it needs to reboot)
[root@node1 ~]# cd /dev
[root@node1 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde
[root@node1 dev]# fdisk sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won’t be recoverable.
The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Do the same steps for 3 other disks.
Reboot node2.
2.3.12 Install and configure ASMLIB
——————————-Download Link
http://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.html
To know your kernel version:
[root@node1 dev]# uname -r
2.6.18-194.el5xen
I’ve downloaded just the following package, because others were already in Enterprise Linux installation disk.
oracleasmlib-2.0.4-1.el5.x86_64.rpm
——————————-Installation
To make visible oracleasmlib-2.0.4-1.el5.x86_64.rpm package for our guest OS , we should make .iso file from it and bound to the system as disk. For this I use ISO Recorder 3.1
[root@node1 Server]# rpm -Uvh oracleasm-support-2.1.3-1.el5.x86_64.rpm
warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
package oracleasm-support-2.1.3-1.el5.x86_64 is already installed
[root@node1 Server]# rpm -Uvh oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.x86_64.rpm
warning: oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
package oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.x86_64 is already installed
file /lib/modules/2.6.18-194.el5xen/kernel/drivers/addon/oracleasm/oracleasm.ko from install of oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.x86_64 conflicts with file from package oracleasm-2.6.18-194.el5xen-2.0.5-1.el5.x86_64
[root@node1 Server]# rpm -Uvh oracleasm-2.6.18-194.el5debug-2.0.5-1.el5.x86_64.rpm
warning: oracleasm-2.6.18-194.el5debug-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
package oracleasm-2.6.18-194.el5debug-2.0.5-1.el5.x86_64 is already installed
[root@node1 Server]# rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm
warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:oracleasmlib ########################################### [100%]
Do the same on node2.
—————————–Configuration
/usr/sbin/oracleasm configure –i
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Do the same on node2.
# Load ASM kernel module
/usr/sbin/oracleasm init
Do the same on node2.
—————————Disk Creation
# Creating ASM disks
[root@node1 ~]# /usr/sbin/oracleasm createdisk CRSVOL /dev/sde1
Writing disk header: done
Instantiating disk: done
[root@node1 ~]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@node1 ~]# /usr/sbin/oracleasm createdisk DISK2 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@node1 ~]# /usr/sbin/oracleasm createdisk DISK3 /dev/sdd1
Writing disk header: done
Instantiating disk: done
You don’t need to do the above steps on node2.(Node2 may require reboot)
# List disks
[root@node1 ~]# /usr/sbin/oracleasm listdisks
CRSVOL
DISK1
DISK2
DISK3
# Testing disk discovery, that is used by DBCA
[root@node1 ~]# oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library – Generic Linux, version 2.0.4 (KABI_V2)]
Discovered disk: ORCL:CRSVOL [6281352 blocks (3216052224 bytes), maxio 512]
Discovered disk: ORCL:DISK1 [20964762 blocks (10733958144 bytes), maxio 512]
Discovered disk: ORCL:DISK2 [20964762 blocks (10733958144 bytes), maxio 512]
Discovered disk: ORCL:DISK3 [20964762 blocks (10733958144 bytes), maxio 512]
2.3.13 Check ping between nodes.
From node1: ping node2 and node2-priv
[root@node1 ~]# ping -c 1 node2
PING node2 (192.168.34.151) 56(84) bytes of data.
64 bytes from node2 (192.168.34.151): icmp_seq=1 ttl=64 time=1.51 ms
— node2 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.511/1.511/1.511/0.000 ms
[root@node1 ~]# ping -c 1 node2-priv
PING node2-priv (10.10.2.51) 56(84) bytes of data.
64 bytes from node2-priv (10.10.2.51): icmp_seq=1 ttl=64 time=2.76 ms
— node2-priv ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.763/2.763/2.763/0.000 ms
From node2: ping node1 and node1-priv
[root@node2 ~]# ping -c1 node1
PING node1 (192.168.34.150) 56(84) bytes of data.
64 bytes from node1 (192.168.34.150): icmp_seq=1 ttl=64 time=0.580 ms
— node1 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.580/0.580/0.580/0.000 ms
[root@node2 ~]# ping -c1 node1-priv
PING node1-priv (10.10.2.50) 56(84) bytes of data.
64 bytes from node1-priv (10.10.2.50): icmp_seq=1 ttl=64 time=0.560 ms
— node1-priv ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.560/0.560/0.560/0.000 ms
2.3.13 Create SSH connectivity manually
From both nodes, install ssh and telnet clients:
[root@node1 Server]# cd /0/Server/
[root@node1 Server]# rpm -Uvh openssh-clients-4.3p2-41.el5.x86_64.rpm
warning: openssh-clients-4.3p2-41.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:openssh-clients ########################################### [100%]
[root@node1 Server]# rpm -Uvh telnet-0.17-39.el5.x86_64.rpm
warning: telnet-0.17-39.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
1:telnet ########################################### [100%]
[root@node1 Server]# rpm -Uvh openssh-server-4.3p2-41.el5.x86_64.rpm
warning: openssh-server-4.3p2-41.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing… ########################################### [100%]
package openssh-server-4.3p2-41.el5.x86_64 is already installed
[root@node1 Server]# rpm -qa | grep ssh
openssh-server-4.3p2-41.el5
openssh-askpass-4.3p2-41.el5
openssh-clients-4.3p2-41.el5
openssh-4.3p2-41.el5
From node1:
[root@node1 ~]# su – oracle
[oracle@node1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Created directory ‘/home/oracle/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
91:c3:7b:dc:20:c0:74:83:8a:f2:73:de:04:42:c8:fe oracle@node1
[oracle@node1 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
f5:79:ab:dd:05:a7:42:48:25:00:27:2a:9d:a5:e2:e2 oracle@node1
From node2:
[root@node2 ~]# su – oracle
[oracle@node2 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Created directory ‘/home/oracle/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
0d:8c:c2:24:1c:32:0e:bf:db:49:e1:e3:3c:71:ef:64 oracle@node2
[oracle@node2 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
1d:52:a7:20:b4:54:4b:c9:12:e1:65:ca:08:a8:e9:de oracle@node2
From node1:
[oracle@node1 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[oracle@node1 ~]$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
[oracle@node1 ~]$ ssh node2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host ‘node2 (192.168.34.151)’ can’t be established.
RSA key fingerprint is 19:1e:e6:3a:cc:14:69:87:22:35:11:6d:aa:e6:dc:15.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node2,192.168.34.151’ (RSA) to the list of known hosts.
oracle@node2’s password:
[oracle@node1 ~]$ ssh node2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
oracle@node2’s password:
[oracle@node1 ~]$ scp ~/.ssh/authorized_keys node2:~/.ssh/authorized_keys
oracle@node2’s password:
authorized_keys 100% 2594 2.5KB/s 00:00
———————————————Check that you are not prompted for the password
From node1 and node2:
[oracle@node1 ~]$ ssh node1 date
The authenticity of host ‘node1 (192.168.34.150)’ can’t be established.
RSA key fingerprint is 24:a1:fb:71:ce:8a:9c:2d:6c:42:89:6f:e5:74:b2:f2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node1,192.168.34.150’ (RSA) to the list of known hosts.
Tue Sep 4 12:09:53 GET 2012
[oracle@node1 ~]$ ssh node2 date
Tue Sep 4 12:10:01 GET 2012
[oracle@node1 ~]$ ssh node1-priv date
The authenticity of host ‘node1-priv (10.10.2.50)’ can’t be established.
RSA key fingerprint is 24:a1:fb:71:ce:8a:9c:2d:6c:42:89:6f:e5:74:b2:f2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node1-priv,10.10.2.50’ (RSA) to the list of known hosts.
Tue Sep 4 12:10:11 GET 2012
[oracle@node1 ~]$ ssh node2-priv date
The authenticity of host ‘node2-priv (10.10.2.51)’ can’t be established.
RSA key fingerprint is 19:1e:e6:3a:cc:14:69:87:22:35:11:6d:aa:e6:dc:15.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node2-priv,10.10.2.51’ (RSA) to the list of known hosts.
Tue Sep 4 12:10:23 GET 2012
Check one more time to ensure that you are not prompted for the password. From node1 and node2.
[oracle@node1 ~]$ ssh node1 date
Tue Sep 4 12:14:57 GET 2012
[oracle@node1 ~]$ ssh node2 date
Tue Sep 4 12:15:01 GET 2012
[oracle@node1 ~]$ ssh node1-priv date
Tue Sep 4 12:15:06 GET 2012
[oracle@node1 ~]$ ssh node2-priv date
Tue Sep 4 12:15:12 GET 2012
Do the same steps with grid user:
From node1:
[root@node1 ~]# su – grid
[grid@node1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_rsa):
Created directory ‘/home/grid/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_rsa.
Your public key has been saved in /home/grid/.ssh/id_rsa.pub.
The key fingerprint is:
e1:c2:55:3e:fc:01:75:64:f7:69:72:f6:9d:1e:08:0c grid@node1
[grid@node1 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_dsa.
Your public key has been saved in /home/grid/.ssh/id_dsa.pub.
The key fingerprint is:
26:36:b8:e4:ac:d5:62:91:f3:46:8a:ac:65:73:ec:6a grid@node1
From node2:
[root@node2 ~]# su – grid
[grid@node2 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_rsa):
Created directory ‘/home/grid/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_rsa.
Your public key has been saved in /home/grid/.ssh/id_rsa.pub.
The key fingerprint is:
f7:54:a6:21:e4:36:a8:9a:e7:93:fb:35:87:70:cc:05 grid@node2
[grid@node2 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_dsa.
Your public key has been saved in /home/grid/.ssh/id_dsa.pub.
The key fingerprint is:
33:36:04:e3:95:0e:b6:d8:92:67:a1:4e:e4:da:8f:26 grid@node2
From node1:
[grid@node1 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[grid@node1 ~]$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
[grid@node1 ~]$ ssh node2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host ‘node2 (192.168.34.151)’ can’t be established.
RSA key fingerprint is 19:1e:e6:3a:cc:14:69:87:22:35:11:6d:aa:e6:dc:15.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node2,192.168.34.151’ (RSA) to the list of known hosts.
grid@node2’s password:
[grid@node1 ~]$ ssh node2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
grid@node2’s password:
[grid@node1 ~]$ scp ~/.ssh/authorized_keys node2:~/.ssh/authorized_keys
grid@node2’s password:
authorized_keys
———————————————Check that you are not prompted for the password
From node1 and node2:
[grid@node1 ~]$ ssh node1 date
The authenticity of host ‘node1 (192.168.34.150)’ can’t be established.
RSA key fingerprint is 24:a1:fb:71:ce:8a:9c:2d:6c:42:89:6f:e5:74:b2:f2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node1,192.168.34.150’ (RSA) to the list of known hosts.
Tue Sep 4 12:44:31 GET 2012
[grid@node1 ~]$ ssh node2 date
Tue Sep 4 12:44:37 GET 2012
[grid@node1 ~]$ ssh node1-priv date
The authenticity of host ‘node1-priv (10.10.2.50)’ can’t be established.
RSA key fingerprint is 24:a1:fb:71:ce:8a:9c:2d:6c:42:89:6f:e5:74:b2:f2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node1-priv,10.10.2.50’ (RSA) to the list of known hosts.
Tue Sep 4 12:44:47 GET 2012
[grid@node1 ~]$ ssh node2-priv date
The authenticity of host ‘node2-priv (10.10.2.51)’ can’t be established.
RSA key fingerprint is 19:1e:e6:3a:cc:14:69:87:22:35:11:6d:aa:e6:dc:15.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘node2-priv,10.10.2.51’ (RSA) to the list of known hosts.
Tue Sep 4 12:44:59 GET 2012
Check one more time to ensure that you are not prompted for the password. From node1 and node2.
[grid@node1 ~]$ ssh node1 date
Tue Sep 4 12:46:57 GET 2012
[grid@node1 ~]$ ssh node2 date
Tue Sep 4 12:47:02 GET 2012
[grid@node1 ~]$ ssh node1-priv date
Tue Sep 4 12:47:08 GET 2012
[grid@node1 ~]$ ssh node2-priv date
Tue Sep 4 12:47:12 GET 2012
2.4 Oracle Grid Installation
Download Grid software installation. And locate it on the first node.
Download link: http://download.oracle.com/otn/linux/oracle11g/R2/linux.x64_11gR2_grid.zip
Login as a grid user.
# Unzip and install
mkdir /home/grid/myinstall
unzip /tmp/linux.x64_11gR2_grid.zip -d /home/grid/myinstall
cd /home/grid/myinstall
chmod -R 777 *
cd 20120613/
chown -R grid:oinstall *
First of all, run runcluvfy.sh script to ensure that pre-requisites are satisfied.
grid@node1 20120613]$ ./runcluvfy.sh stage -pre crsinst -n node1,node2 -verbose
No errors were returned from the script above:
Pre-check for cluster services setup was successful
But if you have some failures like some packages are missing, correct them by installing before continue.
Now run runInstaller.sh . Don’t forget to connect as grid user from the first node.
[grid@node1 20120613]$ ./runInstaller


Add the second node:







From node1:
[grid@node1 ~]$ su –
Password:
[root@node1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
From node2:
[root@node2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
From node1(wait until completes):
[root@node1 ~]# /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script…
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin …
Copying oraenv to /usr/local/bin …
Copying coraenv to /usr/local/bin …
Creating /etc/oratab file…
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2012-09-04 14:59:27: Parsing the host name
2012-09-04 14:59:27: Checking for super user privileges
2012-09-04 14:59:27: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start ‘ora.gipcd’ on ‘node1’
CRS-2672: Attempting to start ‘ora.mdnsd’ on ‘node1’
CRS-2676: Start of ‘ora.mdnsd’ on ‘node1’ succeeded
CRS-2676: Start of ‘ora.gipcd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.gpnpd’ on ‘node1’
CRS-2676: Start of ‘ora.gpnpd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘node1’
CRS-2676: Start of ‘ora.cssdmonitor’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.cssd’ on ‘node1’
CRS-2672: Attempting to start ‘ora.diskmon’ on ‘node1’
CRS-2676: Start of ‘ora.diskmon’ on ‘node1’ succeeded
CRS-2676: Start of ‘ora.cssd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.ctssd’ on ‘node1’
CRS-2676: Start of ‘ora.ctssd’ on ‘node1’ succeeded
ASM created and started successfully.
DiskGroup CRS created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
CRS-2672: Attempting to start ‘ora.crsd’ on ‘node1’
CRS-2676: Start of ‘ora.crsd’ on ‘node1’ succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 37e0beba09a14f4cbf7e8ba1524e1314.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
— —– —————– ——— ———
1. ONLINE 37e0beba09a14f4cbf7e8ba1524e1314 (ORCL:CRSVOL) [CRS]
Located 1 voting disk(s).
CRS-2673: Attempting to stop ‘ora.crsd’ on ‘node1’
CRS-2677: Stop of ‘ora.crsd’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.asm’ on ‘node1’
CRS-2677: Stop of ‘ora.asm’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘node1’
CRS-2677: Stop of ‘ora.ctssd’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.cssdmonitor’ on ‘node1’
CRS-2677: Stop of ‘ora.cssdmonitor’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.cssd’ on ‘node1’
CRS-2677: Stop of ‘ora.cssd’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘node1’
CRS-2677: Stop of ‘ora.gpnpd’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘node1’
CRS-2677: Stop of ‘ora.gipcd’ on ‘node1’ succeeded
CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘node1’
CRS-2677: Stop of ‘ora.mdnsd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.mdnsd’ on ‘node1’
CRS-2676: Start of ‘ora.mdnsd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.gipcd’ on ‘node1’
CRS-2676: Start of ‘ora.gipcd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.gpnpd’ on ‘node1’
CRS-2676: Start of ‘ora.gpnpd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘node1’
CRS-2676: Start of ‘ora.cssdmonitor’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.cssd’ on ‘node1’
CRS-2672: Attempting to start ‘ora.diskmon’ on ‘node1’
CRS-2676: Start of ‘ora.diskmon’ on ‘node1’ succeeded
CRS-2676: Start of ‘ora.cssd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.ctssd’ on ‘node1’
CRS-2676: Start of ‘ora.ctssd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.asm’ on ‘node1’
CRS-2676: Start of ‘ora.asm’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.crsd’ on ‘node1’
CRS-2676: Start of ‘ora.crsd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.evmd’ on ‘node1’
CRS-2676: Start of ‘ora.evmd’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.asm’ on ‘node1’
CRS-2676: Start of ‘ora.asm’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.CRS.dg’ on ‘node1’
CRS-2676: Start of ‘ora.CRS.dg’ on ‘node1’ succeeded
CRS-2672: Attempting to start ‘ora.registry.acfs’ on ‘node1’
CRS-2676: Start of ‘ora.registry.acfs’ on ‘node1’ succeeded
node1 2012/09/04 15:08:38 /u01/app/11.2.0/grid/cdata/node1/backup_20120904_150838.olr
Preparing packages for installation…
ls: /usr/sbin/smartctl: No such file or directory
/usr/sbin/smartctl not found.
error: %pre(cvuqdisk-1.0.7-1.x86_64) scriptlet failed, exit status 1
error: install: %pre scriptlet failed (2), skipping cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster … succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer…
Checking swap space: must be greater than 500 MB. Actual 4095 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
‘UpdateNodeList’ was successful.
If script execution was unsuccessful, then you are able to de-configure it by the following way(if script was successfully run then skip this step ):
[root@node1 ~] cd /u01/app/11.2.0/grid/crs/install
[root@node1 install]# perl rootcrs.pl -deconfig -force
From node2:
[root@node2 ~]# /u01/app/11.2.0/grid/root.sh
….

Click OK.
SCAN verification failed because we are not using DNS.


Check cluster status for both nodes:
[grid@node1 ~]$ crsctl check cluster -all
**************************************************************
node1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[grid@node1 ~]$ olsnodes
node1
node2
[grid@node1 ~]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora.CRS.dg ora….up.type 0/5 0/ ONLINE ONLINE node1
ora….ER.lsnr ora….er.type 0/5 0/ ONLINE ONLINE node1
ora….N1.lsnr ora….er.type 0/5 0/0 ONLINE ONLINE node1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE node1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE node1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora….network ora….rk.type 0/5 0/ ONLINE ONLINE node1
ora….SM1.asm application 0/5 0/0 ONLINE ONLINE node1
ora….E1.lsnr application 0/5 0/0 ONLINE ONLINE node1
ora.node1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.node1.ons application 0/3 0/0 ONLINE ONLINE node1
ora.node1.vip ora….t1.type 0/0 0/0 ONLINE ONLINE node1
ora….SM2.asm application 0/5 0/0 ONLINE ONLINE node2
ora….E2.lsnr application 0/5 0/0 ONLINE ONLINE node2
ora.node2.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.node2.ons application 0/3 0/0 ONLINE ONLINE node2
ora.node2.vip ora….t1.type 0/0 0/0 ONLINE ONLINE node2
ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE node1
ora….ry.acfs ora….fs.type 0/5 0/ ONLINE ONLINE node1
ora.scan1.vip ora….ip.type 0/0 0/0 ONLINE ONLINE node1
Run asmca to create DATA diskgroup:
[grid@node1 ~]$ asmca

Click Create.


Click Exit.
2.4 Oracle Database Installation
Connect as an oracle user.
Download Link: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html?ssSourceSiteId=ocomen
Unzip downloaded file to /home/oracle/myinstall.
[oracle@node1 linux.x64_11gR2_database]$ ./runInstaller


Click Yes.






After DBCA completes its job, there will appear the following window:

From node1:
[oracle@node1 orcl]$ su –
Password:
[root@node1 ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh
Running Oracle 11g root.sh script…
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file “dbhome” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin …
The file “oraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin …
The file “coraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin …
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
From node2:
[root@node2 ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh
Running Oracle 11g root.sh script…
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file “dbhome” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin …
The file “oraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin …
The file “coraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin …
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.

To check:
From node1:
[oracle@node1 ~]$ echo $ORACLE_SID
orcl1
[oracle@node1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.1.0 Production on Tue Sep 4 18:26:10 2012
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
orcl1
SQL> create table test(a number);
Table created.
SQL> insert into test values(1);
1 row created.
SQL> commit;
Commit complete.
From node2:
[oracle@node2 ~]$ echo $ORACLE_SID
orcl2
[oracle@node2 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.1.0 Production on Tue Sep 4 18:29:49 2012
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
orcl2
SQL> select * from test;
A
———-
1
Good Luck!