Resize ASM disks in AWS (FG enabled cluster)

  1. Connect to AWS console https://console.aws.amazon.com
  2. On the left side -> under the section ELASTIC BLOCK STORE -> choose Volumes
  3. Choose necessary disk -> click Actions button -> choose Modify Volume -> change Size
    Please note that all data disks (not quorum disk) must be increased under the same diskgroup, otherwise ASM will not let you to have different sized disks.

Choose another data disks and repeat the same steps.

4. Run the following on database nodes via root user:

# for i in /sys/block/*/device/rescan; do echo 1 > $i; done

5. Check that disks have correct sizes:

# flashgrid-node

6. Connect to the ASM instance from any database node and run:

[grid@rac1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 19.0.0.0.0 - Production on Fri Aug 23 10:17:50 2019
Version 19.4.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.4.0.0.0

SQL> alter diskgroup GRID resize all; 
Diskgroup altered.

UDEV rules for configuring ASM disks

Problem:

During my previous installations I used the following udev rule on multipath devices:

KERNEL=="dm-[0-9]*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="360050768028200a9a40000000000001c", NAME="oracleasm/asm-disk1", OWNER="oracle", GROUP="asmadmin", MODE="0660"

So to identify the exact disk I used PROGRAM option. The above script looks through `/dev/dm-*` devices and if any of them satisfy the condition, for example:

# scsci_id -gud /dev/dm-3
360050768028200a9a40000000000001c 

then device name will be changed to /dev/oracleasm/asm-disk1, owner:group to grid:asmadmin and permission to 0660

But on my new servers same udev rule was not working anymore. (Of course, it needs more investigation, but our time is really valuable and never enough and if we know another solution that works and is acceptable- let’s just use it)

Solution:

I used udevadm command to identify other properties of these devices and wrote new udev rule (to see all properties, just remove grep):

# udevadm info --query=property --name /dev/mapper/asm1 | grep DM_UUID
DM_UUID=mpath-360050768028200a9a40000000000001c

New udev rule looks like this:

# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
ENV{DM_UUID}=="mpath-360050768028200a9a40000000000001c",  SUBSYSTEM=="block", NAME="oracleasm/asm-disk1", OWNER="grid", GROUP="asmadmin", MODE="0660"

Trigger udev rules:

# udevadm trigger

Verify that name, owner, group and permissions are changed:

# ll /dev/oracleasm/
total 0
brw-rw---- 1 grid asmadmin 253, 3 Jul 17 17:33 asm-disk1

ASM Diskstring default values

In case you have trouble getting the newly added disks to your ASM instance check for the ASM_DISKSTRING init paramter. If ASM_DISKSTRING is NOT SET then the following default is used

Default ASM_DISKSTRING per Operating System

Operating System Default            Search String
=======================================
Solaris (32/64 bit)                        /dev/rdsk/*
Windows NT/XP                          .orcldisk*
Linux (32/64 bit)                          /dev/raw/*

LINUX (ASMLIB)                         ORCL:*
LINUX (ASMLIB)                        /dev/oracleasm/disks/*

HPUX                                             /dev/rdsk/*
HP-UX(Tru 64)                            /dev/rdisk/*
AIX                                                 /dev/*

IF ASM_DISKSTRING is SET  then we should verify that the setting includes the disks that are needed to be seen by ASM.

 

Source from :  https://abchatur.wordpress.com/2011/09/01/asm-diskstring-default-values/

 

Add/Drop ASM disks to DISKGROUP on RAC(or Standalone)

Note: The steps are described for RAC, but you can easily guess what are the steps for the standalone database.

1. First of all find the disk or partition name, that should be added to the ASM.

fdisk -l

My disk partition name is /dev/sdi1.

2. Assign the disk to ORACLEASM.

–On node1

/etc/init.d/oracleasm createdisk DISK7 /dev/sdi1

3. Scan disks in ALL NODES and list them to check if is presented.

–On node1

/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

–On node2

/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

4. Change the environment to the grid infrastructure, by setting ORACLE_SID to +ASM and so on :

$ . oraenv
ORACLE_SID = [media1] ? +ASM1
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle

# Connect as an SYSASM

sqlplus / as sysasm

Note: If you don’t remember the password for the sysasm user see How to reset SYSASM password.

# Find the diskgroup name

SQL> select name from v$asm_diskgroup;

NAME
——————————
DATA01

# Increase power limit, if you want, to complete rebalance operation in a short time.

SQL>  alter system set asm_power_limit=10

# Indicate disks location by the parameter asm_diskstring

SQL> alter system set asm_diskstring=’ORCL:DISK*’

SQL> alter diskgroup DATA01 add disk ‘ORCL:DISK7’;

It will do the rebalance automatically.

# To drop the disk , do the following:

SQL >  alter diskgroup DATA01 drop disk DISK7;

It will rebalance first and then drops the disk automatically.

You can see the current operation in v$asm_operation view.

Note: Until the view v$asm_operation contains a record you are able to undrop the disks by the following way:

SQL> alter diskgroup DATA01 undrop disks;

If the operation is already completed , you are not able to undrop the disk . But you can re-add the disk , if you want.

That is all.

Install Oracle 11.2.0.3 with ASM on Centos 6.3

For RHEL 6, Oracle will provide ASMLib software and updates only when configured with a kernel distributed by Oracle. Oracle will not provide ASMLib packages for kernels distributed by Red Hat as part of RHEL 6. ASMLib updates will be delivered via Unbreakable Linux Network (ULN), which is available to customers with Oracle Linux support. ULN works with both Oracle Linux or Red Hat Linux installations, but ASMLib usage will require replacing any Red Hat kernel with a kernel provided by Oracle.

Because of the above announcement we use UDEV rules to prepare disks for ASM installation.

So let’s start.

1. Install required RPMs.

RPM names:

compat-libcap1-1.10-1.i686.rpm
compat-libcap1-1.10-1.x86_64.rpm
compat-libstdc++-33-3.2.3-69.el6.x86_64.rpm
elfutils-devel-0.152-1.el6.x86_64.rpm
elfutils-libelf-devel-0.152-1.el6.x86_64.rpm
gcc-c++-4.4.6-4.el6.x86_64.rpm
glibc-2.12-1.80.el6.i686.rpm
glibc-devel-2.12-1.80.el6.i686.rpm
libaio-0.3.107-10.el6.i686.rpm
libaio-devel-0.3.107-10.el6.x86_64.rpm
libattr-2.4.44-7.el6.i686.rpm
libcap-2.16-5.5.el6.i686.rpm
libgcc-4.4.6-4.el6.i686.rpm
libstdc++-devel-4.4.6-4.el6.x86_64.rpm
libtool-ltdl-2.2.6-15.5.el6.i686.rpm
ncurses-devel-5.7-3.20090208.el6.i686.rpm
ncurses-libs-5.7-3.20090208.el6.i686.rpm
nss-softokn-freebl-3.12.9-11.el6.i686.rpm
pdksh-5.2.14-30.x86_64.rpm
readline-6.0-4.el6.i686.rpm

If you have Centos installation disk, these RPMs should be locate there. Or you can download them from  http://rpm.pbone.net/

Note: if during installing pdksh-5.2.14-30.x86_64.rpm it says that package conflicts with ksh then you should erase ksh package and install pdksh, like this:

# rpm -qa | grep ksh
# rpm -e ksh-…
# rpm –ivh pdksh-5.2.14-30.x86_64.rpm

2. Configure Kernel:

# vi /etc/sysctl.conf

kernel.shmall = 2097152
kernel.shmmax = 982431744
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

To make changes take effect:

# sysctl –p

Edit /etc/pam.d/login :

# vi /etc/pam.d/login

session required pam_limits.so

Edit /etc/security/limits.conf:

# vi /etc/security/limits.conf

oracle soft  nproc   2047
oracle hard  nproc   16384
oracle soft  nofile  1024
oracle hard  nofile  65536

grid   soft  nproc   2047
grid   hard  nproc   16384
grid   soft  nofile  1024
grid   hard  nofile  65536

Run the following to add lines in /etc/pam.d/login:

[root@orcl ~]# cat >> /etc/pam.d/login <<EOF
session required pam_limits.so
EOF

Run the following to add lines in /etc/profile:

[root@orcl ~]# cat >> /etc/profile <<EOF
if [ \$USER = "oracle" ] || [ \$USER = "grid" ]; then
if [ \$SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
EOF

Run the following to add lines in /etc/csh.login

[root@orcl ~]# cat >> /etc/csh.login <<EOF
if ( \$USER == "oracle" || \$USER == "grid" )
then
limit maxproc 16384
limit descriptors 65536
endif
EOF

Disable SELinux:

# /usr/sbin/getenforce
Enforcing

# /usr/sbin/setenforce 0

To make changes permanent, change /etc/sysconfig/selinux file by the following way:

cat /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing – SELinux security policy is enforced.
#     permissive – SELinux prints warnings instead of enforcing.
#     disabled – No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted – Targeted processes are protected,
#     mls – Multi Level Security protection.
SELINUXTYPE=targeted

To check the status again:

# /usr/sbin/getenforce
Disabled

3.  Creating OS groups and users.

#Creating groups for Grid Infrastructure

groupadd asmadmin
groupadd asmdba
groupadd asmoper

#Creating groups for Oracle Software

groupadd oinstall
groupadd dba
groupadd oper

#Creating user for Grid Infrastructure

useradd -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid grid

#Creating user for Oracle Software

useradd -g oinstall -G dba,oper,asmdba -d /home/oracle oracle

#Setting password for users

passwd grid
passwd oracle

4. Creating necessary directories

mkdir -p /u01/app/grid
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01

5. Creating .bash_profile-s

#For Oracle user

su – oracle
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=orcl; export ORACLE_SID

ORACLE_UNQNAME=orcl; export ORACLE_UNQNAME

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"
export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

#For Grid user

su – grid
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=+ASM; export ORACLE_SID

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

 

6. Add disks for ASM

If your disk is not partitioned yet, partition it by fdisk utility.

At this point you will need SCSI identifier.

So if your disk is physical just run the following command to identify it:

# scsi_id -g -u -d /dev/sdb
36000c292dfddac7b8934d3293313098e

Or if you have virtual disk , you will need to set disk.EnableUUID parameter to TRUE to see this identifier:

Shutdown VM, go to the directory where VM files are stored and edit VMX file. Add the following line:

disk.EnableUUID = "TRUE"

Restart your VM and run the above command (scsi_id -g -u -d /dev/sdb)

We will use this identifier for the UDEV rules to set permissions and alias for the new device in /etc/udev/rules.d/50-udev.rules file.

# vi /etc/udev/rules.d/50-udev.rules

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent",
RESULT=="36000c292dfddac7b8934d3293313098e", NAME="oracleasm/asm-disk1",
OWNER="oracle", GROUP="dba", MODE="0660"

Restart udev, by the following way:

# /sbin/start_udev
Starting udev:                                             [  OK  ]

Check that alias exists:

# ls -la /dev/oracleasm/*
brw-rw—- 1 oracle dba 8, 17 Feb 18 12:49 /dev/oracleasm/asm-disk1

7. Install grid

Go to the grid installation folder and run:

./runInstaller

Step 1:

Skip Software Updates.

Step 2:

Configure Oracle Grid Infrastructure for a Standalone Server.

Step 3:

Click Next.

Step 4:

Select the Change Discovery Path button and enter /dev/oracleasm.

There should appear /dev/oracleasm/asm-disk-1 and check it.

Type disk group name as DATA01.

Step 5:

Set the passwords for the SYS and ASMSNMP accounts:

Step 6:

ASM Database Administrator(OSDBA) Group : asmdba
ASM Instance Administration Operator(OSOPER) Group: asmoper
ASM Instance Administrator(OSASM) Group: asmadmin

Step 7:

Click Next.

Step 8:

Click Next.

Step 9:

Click Install.

After pop-upping the window , asking to run

      • /u01/app/oraInventory/orainstRoot.sh
  • /u01/app/oracle/product/11.2.0/db_1/root.sh

run these scripts one by one and click OK on pop-up window.

Note: if root.sh script shows the error like this:

…error while loading shared libraries: libcap.so.1: …

Then in your system libcap-1 and libcap-2 RPMs are missing, first install them.

8. Install database

Go to the database installation folder and run:

./runInstaller

Step 1:

I don’t want any more spam thanks.

Step 2:

Skip the updates.

Step 3:

Create and configure a database.

Step 4:

Server Class.

Step 5:

Single instance database installation.

Step 6:

Advanced install.

Step 7:

Choose the languages you want.

Step 8:

Enterprise Edition.

Step 9:

Choose the defaults.
Note: Grid and Database must be in the different folders.

Step 10:

General Purpose / Transaction Processing.

Step 11:

Write a database name.

Step 12:

Use Oracle Enterprise Manager Database Control for database management.

Step 13:

Use Automatic Storage Management.

Step 15:

Do not enable automated backups

Step 16:

Select the DATA01 diskgroup.

Step 17:

Set the passwords for the database.

Step 18:

Accept the defaults.

Database Administrator(OSDBA) Group: dba
Database Operator(OSOPER) Group: oper

Step 19:

Click Install to start the installer.

After the installation requires to run root.sh script, run it.

That is all.

ASM disk mapping to physical device

To find which ASM disk maps to which physical device, run  the following:

[root@orcl ~]# /etc/init.d/oracleasm querydisk -p VOL1
Disk “VOL1″ is a valid ASM disk
/dev/sdd1: LABEL=”VOL1″ TYPE=”oracleasm”

Install Oracle Database 11g on Linux with ASM

Let’s assume that you have already installed Linux on your server. And disks are already bounded to the server.

1. Creating OS groups and users.

#Creating groups for Grid Infrastructure

groupadd asmadmin
groupadd asmdba
groupadd asmoper

#Creating groups for Oracle Software

groupadd oinstall
groupadd dba
groupadd oper

#Creating user for Grid Infrastructure

useradd -g oinstall -G dba,asmadmin,asmdba,asmoper -d /home/grid grid

#Creating user for Oracle Software

useradd -g oinstall -G dba,oper,asmdba -d /home/oracle oracle

#Setting password for users

passwd grid
passwd oracle

2. Creating necessary directories

mkdir -p /u01/app/grid
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle
chown -R grid:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01

3. Creating .bash_profile-s

#For Oracle user

su – oracle
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=orcl; export ORACLE_SID

ORACLE_UNQNAME=orcl; export ORACLE_UNQNAME

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”
export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

#For Grid user

su – grid
vi .bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

ORACLE_SID=+ASM; export ORACLE_SID

JAVA_HOME=/usr/local/java; export JAVA_HOME

ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp
export TMPDIR=/tmp

umask 022

4. Setting resource limits

Edit the following files:

# /etc/security/limits.conf

[root@orcl ~]# cat >> /etc/security/limits.conf <<EOF
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536

oracle soft nproc 2047
oracle  hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
EOF

# /etc/pam.d/login

[root@orcl ~]# cat >> /etc/pam.d/login <<EOF
session required pam_limits.so
EOF

# /etc/profile

[root@orcl ~]# cat >> /etc/profile <<EOF
if [ \$USER = “oracle” ] || [ \$USER = “grid” ]; then
if [ \$SHELL = “/bin/ksh” ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
EOF

# /etc/csh.login

[root@orcl ~]# cat >> /etc/csh.login <<EOF
if ( \$USER == “oracle” || \$USER == “grid” )
then
limit maxproc 16384
limit descriptors 65536
endif
EOF

# /etc/hosts

[root@orcl ~]#  vi /etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.34.150 orcl

5. Installing ASMlib and creating disk groups

Download link: http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

# To know your kernel version

uname -r
2.6.18-194.el5xen

# Download the following files

oracleasm-support-2.1.7-1.el5.x86_64.rpm
oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm

oracleasmlib-2.0.4-1.el5.x86_64.rpm

# Installing

rpm -Uvh oracleasm-support-2.1.7-1.el5.x86_64.rpm
rpm -Uvh oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm
rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm

# Configure ASM

/usr/sbin/oracleasm configure -i

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

# Load ASM kernel module

/usr/sbin/oracleasm init

##Partition available disks for ASM

#list available disks and partitions

fdisk -l

#partitioning

fdisk /dev/sdb

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-52216, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-52216, default 52216):
Using default value 52216

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

fdisk /dev/sdc

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-52216, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-52216, default 52216): +100M

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

# Restart the server

init 6

# Creating ASM disks

/usr/sbin/oracleasm createdisk VOL1 /dev/sdb1
/usr/sbin/oracleasm createdisk CRSVOL /dev/sdc1

# List disks

/usr/sbin/oracleasm listdisks
VOL1
CRSVOL

# Testing disk discovery, that is used by DBCA

oracleasm-discover

Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library – Generic Linux, version 2.0.4 (KABI_V2)]
Discovered disk: ORCL:CRSVOL [208782 blocks (106896384 bytes), maxio 512]
Discovered disk: ORCL:VOL1 [838849977 blocks (429491188224 bytes), maxio 512]

6. Installing Oracle Grid Infrastructure

Download link: http://download.oracle.com/otn/linux/oracle11g/R2/linux.x64_11gR2_grid.zip

Login as a grid user.

# Unzip and install

mkdir  /home/grid/myinstall
unzip /tmp/linux.x64_11gR2_grid.zip  -d  /home/grid/myinstall

cd /home/grid/myinstall
chmod -R 777 *
./runInstaller

Choose the following options:

6.1. Install and Configure Grid Infrastructure for a Standalone Server

6.2.  English

6.3. English

6.4.
Disk Group Name: CRS
Redundancy: External
Add Disks->Candidate Disks, choose just  ORCL:CRSVOL

6.5. Speciify password(s).

6.6.
ASM Database Administrator(OSDBA) Group : asmdba
ASM Instance Administration Operator(OSOPER) Group: asmoper
ASM Instance Administrator(OSASM) Group: asmadmin

6.7.
Oracle Base: /u01/app/grid
Software Location: /u01/app/11.2.0/grid

6.8. On the pop-up windows press Yes.

6.9. Inventory Directory: /u01/app/oraInventory

6.10. On the prerequisite checks page, there will be failed checks. To solve:

6.10.1 Press Fix & Check Again
6.10.2 Login as root user and run the following:

[root@orcl ~]# /tmp/CVU_11.2.0.1.0_grid/runfixup.sh

6.10.3 Press OK on the Execute Fixup window

Left just libaio* and unixODBC* packages that should be installed. To solve:

6.10.4 Mount Oracle Enterprise Linux installation disk and go to the Server folder, where rpms are located.

[root@orcl Server]# rpm -Uvh libaio-devel-0.3.106-5.i386.rpm
warning: libaio-devel-0.3.106-5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:libaio-devel           ########################################### [100%]
[root@orcl Server]# rpm -Uvh libaio-devel-0.3.106-5.x86_64.rpm
warning: libaio-devel-0.3.106-5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:libaio-devel           ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-2.2.11-7.1.i386.rpm
warning: unixODBC-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC               ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-2.2.11-7.1.x86_64.rpm
warning: unixODBC-2.2.11-7.1.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC               ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-devel-2.2.11-7.1.i386.rpm
warning: unixODBC-devel-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC-devel         ########################################### [100%]
[root@orcl Server]# rpm -Uvh unixODBC-devel-2.2.11-7.1.x86_64.rpm
warning: unixODBC-devel-2.2.11-7.1.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing…                ########################################### [100%]
1:unixODBC-devel         ########################################### [100%]

6.10. 5   Press Check Again.(There should not be any error)

6.11.

[root@orcl ~]# /u01/app/oraInventory/orainstRoot.sh
[root@orcl ~]# /u01/app/11.2.0/grid/root.sh

Press enter…

7. Creating ASM disk groups for database

Connect as a grid user.

[grid@orcl ~]$ asmca

7.1 Press Create button.

7.2
Disk Group Name: DATA
Redundancy: External(None)
Select Member Disks->Show Eligible, choose ORCL:VOL1

8. Installing Database, create instance on ASM

Connect as an oracle user.

Download Link: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html?ssSourceSiteId=ocomen

Unzip downloaded file to /home/oracle/myinstall.

[oracle@orcl linux.x64_11gR2_database]$ ./runInstaller

8.1 Uncheck “I with to receive security updates via My Oracle Support”

8.2 On the pop-up window choose yes.

8.3 Install database software only

8.4 Single instance database installation

8.5 English

8.6 Enterprise Edition (4.29GB)

8.7
Oracle Base: /u01/app/oracle
Software Location:  /u01/app/oracle/product/11.2.0/db_1

8.8
Database Administrator(OSDBA) Group: dba
Database Operator(OSOPER) Group: oper

8.9 I have some failed checks that are fixable so I press Fix & Check Again button, login as a root user and run:

[root@orcl ~]# /tmp/CVU_11.2.0.1.0_oracle/runfixup.sh

Click OK on the Execute Fixup window.

8.10

[root@orcl ~]# /u01/app/oracle/product/11.2.0/db_1/root.sh
Press Enter…
y
y
y

9. Run DBCA as an oracle user.

[oracle@orcl ~]$ dbca

and follow the steps (choose ASM instead of File System option)

That’s all.

Could not validate ASMSNMP password due to…During DB creation

If you are creating database using DBCA and came across the following error:

Screenshot-Database Configuration Assistant

Do the following:

bash-3.2$ . oraenv
ORACLE_SID = [RDBMS] ? +ASM
The Oracle base for ORACLE_HOME=/u02/app/11.2.0/grid is /u01/app/oracle
bash-3.2$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.1.0 Production on Mon May 21 18:59:20 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Automatic Storage Management option

SQL> create user ASMSNMP identified by testpas1;
create user ASMSNMP identified by testpas1
*
ERROR at line 1:
ORA-01990: error opening password file ‘/u02/app/11.2.0/grid/dbs/orapw’

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Automatic Storage Management option
bash-3.2$ echo $ORACLE_HOME
/u02/app/11.2.0/grid
bash-3.2$ cd /u02/app/11.2.0/grid/dbs/
bash-3.2$ orapwd file=orapw+ASM password=testpas1 entries=2
bash-3.2$ ls
ab_+ASM.dat  hc_+ASM.dat  init+ASM.ora  init.ora  orapw+ASM  spfile+ASM.ora
bash-3.2$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.1.0 Production on Mon May 21 19:01:20 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – 64bit Production
With the Automatic Storage Management option

SQL> create user ASMSNMP identified by testpas1;

User created.

SQL> grant sysdba to ASMSNMP;

Grant succeeded.

That’s it. Re-run DBCA.

Candidate ASM disks not showing

If you are in the middle of grid installation, and trying to create ASM disk group but Eligible(Candidate) disks location is empty, for example like this:Screenshot-Create Disk Group

moreover you have configured candidate disks using /etc/init.d/oracleasm and listdisks command showing the disk.

bash-3.2$ /etc/init.d/oracleasm listdisks

VOL1

Then follow my steps that may help:

1. Click “Change Disk Discovery Path” button

2. Enter the path /dev/oracleasm/disks and click OK

Screenshot-Change Disk Discovery Path

3. The picture should look like this:

Screenshot-Create Disk Group-1

That’s it. GOOD LUCK!

Oracle 10g RAC Single Node Installation on Solaris 10

1. Pre-Installation Tasks

Create oracle user and appropriate groups

First of all, you should have already mounted /u0 on some slice of disk. If you don’t know how to mount directory on disk Smile click here.

Go to the folder /u0 and create the following directories.

cd /u0 mkdir –p app/oracle

Create oinstall and dba groups.

groupadd oinstall groupadd dba

Create oracle user and assign it oinstall and dba groups.

useradd –s /bin/ksh –d /u0/app/oracle –g oinstall –G dba oracle

Where,
-s option  specifies the user default  shell (i.e. Korn shell) ,
-d option specifies the home directory for oracle user (i.e.  /opt/app/oracle ) ,
-g option specifies the primary group (i.e. oinstall)
-G option specifies the secondary  group (i.e. dba)

Make oracle user as the owner of app folder.

chown –R oracle:oinstall app

Change the password for oracle user.

passwd oracle

Configuring SSH

Create RSA and DSA keys on each node.

Log in as the oracle user and go to the ORACLE_BASE directory(For me oracle_base directory is /u0/app/oracle).

mkdir .ssh

Generating RSA and DSA keys.

/usr/bin/ssh-keygen -t rsa /usr/bin/ssh-keygen -t dsa

Add keys to an authorized key file, this file should exist in .ssh directory. If it doesn’t exist, create it. Change your direction to .ssh directory and run the following.

ssh charly1 cat /u0/app/oracle/.ssh/id_rsa.pub >> authorized_keys ssh charly1 cat /u0/app/oracle/.ssh/id_dsa.pub >> authorized_keys

Enabling ssh user equivalency.

/usr/bin/ssh-agent $SHELL /usr/bin/ssh-add

To prevent Oracle clusterware installation errors caused by stty commands, add the following text in oracle user’s profile (you should change your directory to oracle home directory and open “.profile” for editing).

if [ -t 0 ]; then stty intr ^C fi

Network Requirements

We should create second IP interface.
Open or create the file /etc/hostname.e100g1 and enter the following text:

Host1-priv

Add the following entries into /etc/host file:

20.0.0.100           Host1 loghost 192.168.2.117   Host1-priv 20.0.0.105          Host1-vip

Where 192.168.2.117 is a private IP address and 20.0.0.105 is a virtual IP address of the server.

Connect as a root user and create /etc /hosts.equiv file. Then open this file for editing and enter the following text:

Host1 oracle

Configuring Kernel Parameters

To add the system parameters open /etc/system file and add the following entries:

set shmsys:shminfo_shmmax = 4294967295 set shmsys:shminfo_shmmin = 1 set shmsys:shminfo_shmmni = 100 set shmsys:shminfo_shmseg = 10 set semsys:seminfo_semmni = 100 set semsys:seminfo_semmns = 1024 set semsys:seminfo_semmsl = 256 set md:mirrored_root_flag=1 set noexec_user_stack=1

Identifying Required Software Directories

Creating an Oracle Base Directory

mkdir -p /u0/app/oracle chown -R oracle:oinstall /u0/app/oracle chmod -R 775 /u0/app/oracle

Creating the Oracle Clusterware Home Directory

mkdir -p /u0/app/oracle/product/10.2.0/crs chown -R root:oinstall /u0/app/oracle/product/10.2.0/crs chmod -R 775 /u0/app/oracle/product/10.2.0/crs

2. Installing Oracle Clusterware

First of all, you should download Oracle Clusterware from the oracle site.

Verifying Oracle Clusterware Requirements with CVU

Extract downloaded file and run runcluvfy.sh file, which should be located in clusterware/cluvfy folder.

./runcluvfy.sh comp nodecon -n Host1 –verbose

Installing Oracle Clusterware with OUI

Initialize the devices that you want to use for the Oracle Cluster Registry (OCR) and Voting Disk(VD)

1.Voting file– uses by the cluster synchronization service deamons for monitoring information across the cluster. It’s size around 20MB(use format utility and reserve 200MB on 5th slice)

2. Oracle cluster registry(OCR file) it maintain information about the high-availability components such as cluster node list, CRS application profiles(Virtual interconnect protocol address, services). It’s size around 200MB(use format utility and reserve on 7th slice)

For voting disk

dd if=/dev/zero of=/dev/rdsk/c5t200g35A100TK170Ta00ds5 bs=125829120 count=1 chown oracle:dba /dev/rdsk/c5t200g35A100TK170Ta00ds5
chmod 660 /dev/rdsk/c5t200g35A100TK170Ta00ds5

For OCR

dd if=/dev/zero of=/dev/rdsk/c5t200g35A100TK170Ta00ds7 bs=125829120 count=1 chown root:dba /dev/rdsk/c5t200g35A100TK170Ta00ds7
chmod 660 /dev/rdsk/c5t200g35A100TK170Ta00ds7

Connect as the oracle user and run runInstaller.sh, located in CRS installation.

./runInstaller

Specify Inventory Directory and Credentials

Specify Home Details

Product-Specific Prerequisite Checks

Specify Cluster Configuration

Specify Network Interface Usage

Specify OCR Location

Specify Voting Disk Location

OrainstRoot.sh and root.sh

Run these scripts and then click OK.

OUI-25031

Now we should run ./vipca and configure Host1-vip virtual ip address(20.0.0.105).

Go to the installation window, click ok and retry install failed component.


Installing Oracle database files


Welcome_Oracle_OUI

Select Installation Type

Specify Home Details

Specify Hardware Cluster Installation Node

Product-Specific Prerequisite Ckecks

Select Configuration Option

OUI_Install

Run root.sh script and click OK, then click Exit button.

Configure oracle listener by netca.

Configuring ASM

DBCA Oracle RAC

Configure ASM

ASM diskgroups

ASM Instance

{ORACLE_HOME}/dbs/spfile+ASM error

DBCA will create and start ASM

No diskgroups

Run the following command, for to make oracle user owner for the following disk slice

chown oracle:oinstall /dev/rdsk/c5156D778A145a11d0s6

DG_DATA Disk group Name

Choose Disk group

Perform Another Operation Alert

Creating database using DBCA


DBCA RAC Database

DBCA Create a Database

Choose Node

DBCA general purpose

Global Database Name

Click Next>> and check/uncheck Configure the Database with Enterprise Manager.

Specific Passwords for accounts

ASM

SYS password for ASM Alert

Choose Disk Group

Use OMF

Enable Archiving

Sample Schemas

DBCA Memory tab

DBCA General tab

DBCA Create Database