E492: Not an editor command: ASM2_ora_26576.trc

Problem:

I want to open ASM trace file using vim.

[root@rac2 trace]# ll +ASM2_ora_26576.trc
-rw-r----- 1 grid oinstall 1388 Mar  7 18:06 +ASM2_ora_26576.trc

But because of filename contains + sign, my command fails:

[root@rac2 trace]# vim +ASM2_ora_26576.trc

Error detected while processing command line:
E492: Not an editor command: ASM2_ora_26576.trc
Press ENTER or type command to continue

Solution:

Indicate — (two dashes)

[root@rac2 trace]# vim -- +ASM2_ora_26576.trc

From man page of vim — means the following:

 --          Denotes  the  end  of  the options.  Arguments after this will be handled as a file name.  This can be used to edit a filename that starts with a '-'.
Advertisement

Move Oracle controlfile from filesystem to ASM

1. Find out controlfile locations:

Please note another copy of controlfile is already on ASM, +FRA diskgroup. We are moving the first one.

SQL> show parameter control_files

NAME				     TYPE	 VALUE
------------------------------------ ----------- ------------------------------
control_files			     string	 /u01/app/oracle/oradata/PRSH01
						 /control01.ctl, +FRA/PRSH01/co
						 ntrol02.ctl

2. Place database into nomount mode using RMAN and copy controlfile to new location:

$ rman target /

RMAN> startup nomount;
RMAN> restore controlfile to '+DATA' from '/u01/app/oracle/oradata/PRSH01/control01.ctl';

3. Note the name assigned to the controlfile:

[grid@prsh01 ~]$ asmcmd find --type CONTROLFILE +data *
+data/PRSH01/CONTROLFILE/current.261.1122675497

4. Modify the parameter from sqlplus:

SQL> alter system set control_files='+data/PRSH01/CONTROLFILE/current.261.1122675497','+FRA/PRSH01/control02.ctl' scope=spfile;

5. Restart database

SQL> shutdown immediate;
SQL> startup

6. Double check new location

SQL> show parameter control_files

NAME				     TYPE	 VALUE
------------------------------------ ----------- ------------------------------
control_files			     string	 +DATA/PRSH01/CONTROLFILE/curre
						 nt.261.1122675497, +FRA/PRSH01
						 /control02.ctl

Move Oracle datafiles from filesystem to ASM using RMAN

1. Connect to the database using RMAN and move database into mount mode:

$ rman target /
RMAN> shutdown immediate
RMAN> startup mount

2. Get information about target datafiles and tempfiles:

RMAN> report schema;

using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name PRSH01

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    1020     SYSTEM               YES     /u01/app/oracle/oradata/PRSH01/system01.dbf
3    490      SYSAUX               NO      /u01/app/oracle/oradata/PRSH01/sysaux01.dbf
4    850      UNDOTBS1             YES     /u01/app/oracle/oradata/PRSH01/undotbs01.dbf
7    5        USERS                NO      /u01/app/oracle/oradata/PRSH01/users01.dbf

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    114      TEMP                 32767       /u01/app/oracle/oradata/PRSH01/temp01.dbf

3. Copy datafiles from filesystem to ASM diskgroup, in our case to +DATA:

RMAN> copy datafile 1 to '+DATA';
RMAN> copy datafile 3 to '+DATA';
RMAN> copy datafile 4 to '+DATA';
RMAN> copy datafile 7 to '+DATA';
RMAN> switch datafile 1 to copy;
RMAN> switch datafile 3 to copy;
RMAN> switch datafile 4 to copy;
RMAN> switch datafile 7 to copy;

4. Place tempfile on +DATA:

RMAN> run{
2> set newname for tempfile 1 to '+DATA';
3> switch tempfile 1;
4> }

executing command: SET NEWNAME
renamed tempfile 1 to +DATA in control file

5. Double check that all files have a new name

RMAN> report schema;

Report of database schema for database with db_unique_name PRSH01

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    1020     SYSTEM               ***     +DATA/PRSH01/DATAFILE/system.256.1122674191
3    570      SYSAUX               ***     +DATA/PRSH01/DATAFILE/sysaux.257.1122674225
4    910      UNDOTBS1             ***     +DATA/PRSH01/DATAFILE/undotbs1.258.1122674235
7    5        USERS                ***     +DATA/PRSH01/DATAFILE/users.259.1122674253

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    114      TEMP                 32767       +DATA

6. Open database:

RMAN> alter database open;

7. Check new file names. Note tempfile name will also be generated:

RMAN> report schema;

Report of database schema for database with db_unique_name PRSH01

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    1020     SYSTEM               YES     +DATA/PRSH01/DATAFILE/system.256.1122674191
3    570      SYSAUX               NO      +DATA/PRSH01/DATAFILE/sysaux.257.1122674225
4    910      UNDOTBS1             YES     +DATA/PRSH01/DATAFILE/undotbs1.258.1122674235
7    5        USERS                NO      +DATA/PRSH01/DATAFILE/users.259.1122674253

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    114      TEMP                 32767       +DATA/PRSH01/TEMPFILE/temp.260.1122674673

DBCA Fails With INS-00001 and DBT-00007

Problem:

While trying to run dbca from Oracle user I received the following error:

Cause - No additional information available.  
Action - Refer to the logs or contact Oracle Support Services  

Summary  
- [DBT-00007] User does not have the appropiate write privileges.  
- [DBT-00007] User does not have the appropiate write privileges.

Screenshot:

Solution:

Make sure the directory has the correct owner and permissions, otherwise set:

# chown oracle:oinstall $ORACLE_BASE/cfgtoollogs/dbca
# chmod 750 $ORACLE_BASE/cfgtoollogs/dbca

Rerun dbca.

CRS-2549: Resource ‘ora.asmgroup’ cannot be placed on ‘rac1’ as it is not a valid candidate as per the placement policy

Problem:

After failed JDK patching on the 1st node, we tried troubleshooting and saw that ASM was not able to start:

# su - grid
$ sqlplus / as sysasm
SQL> startup nomount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ORA-39511: Start of CRS resource for instance '223' failed with error:[CRS-2549: Resource 'ora.asmgroup' cannot be placed on 'rac1' as it is not a valid candidate as per the placement policy
CRS-0223: Resource 'ora.asm' has placement error.
clsr_start_resource:260 status:223
clsrapi_start_asm:start_asmdbs status:223

Reason:

Prepatch modified RESOURCE_USE_ENABLED=0 for rac1 node:

[grid@rac1 ~]$ crsctl stat server -f

NAME=rac1
MEMORY_SIZE=63465
CPU_COUNT=8
CPU_CLOCK_RATE=2499
CPU_HYPERTHREADING=1
CPU_EQUIVALENCY=1000
DEPLOYMENT=other
CONFIGURED_CSS_ROLE=hub
RESOURCE_USE_ENABLED=0
SERVER_LABEL=
PHYSICAL_HOSTNAME=
CSS_CRITICAL=no
CSS_CRITICAL_TOTAL=0
RESOURCE_TOTAL=0
SITE_NAME=stsfilive
STATE=ONLINE
ACTIVE_POOLS=Free
STATE_DETAILS=
ACTIVE_CSS_ROLE=hub

NAME=rac2
MEMORY_SIZE=63465
CPU_COUNT=8
CPU_CLOCK_RATE=2499
CPU_HYPERTHREADING=1
CPU_EQUIVALENCY=1000
DEPLOYMENT=other
CONFIGURED_CSS_ROLE=hub
RESOURCE_USE_ENABLED=1
….

Solution:

Connect to the failing node and run:

[root@rac1 ~]# crsctl set resource use 1

Start ASM.

Installing linux.x64_193000_client gets java.library.path system variable is missing or invalid

Problem:

While installing linux.x64_193000_client.zip on Linux, getting the following error:

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-06-15_07-48-19AM. Please wait ...[oracle@mariclient client]$ Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/u01/tmp
The java.library.path system variable is missing or invalid. Please set java.library.path with a correct value and retry the operation.
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class oracle.sysman.oii.oiip.oiipg.OiipgPropertyLoader
	at oracle.sysman.oii.oiip.oiipg.OiipgBootstrap.isCleanMachine(OiipgBootstrap.java:484)
	at oracle.sysman.oii.oiii.OiiiInstallAreaControl.isCleanMachine(OiiiInstallAreaControl.java:3796)

Solution:

Install the following rpm and retry:

# yum install libnsl.x86_64

RMAN backup on NFS v3 takes too much time

Problem:

I was trying to test RMAN backup on Azure blob storage with NFS v3 access. I have noticed a huge delay even when backing up a small controlfile.

The same backup on disk took 7sec, but on NFS 15min. While watching IO on NFS mountpoint using nfsiostat I saw that initial 13min there was no IO at all and actual backup was taken only at the last minute.

Another interesting thing while watching netstat output was the following:

[root@rac1 dbbackup2]# netstat -na|grep 10.0.0.
tcp 0 0 10.0.0.5:875 10.0.0.16:2048 ESTABLISHED
tcp 0 1 10.0.0.5:54997 10.0.0.16:2049 SYN_SENT

You can also turn debug on before running backup in RMAN and check the output:

RMAN> RUN
{
  ALLOCATE CHANNEL disk1 DEVICE TYPE DISK FORMAT '/dataload/%U';
  debug on;
  BACKUP current controlfile;
  debug off;
}

Solution:

Disable dNFS.

$ cd $ORACLE_HOME/rdbms/lib
$ make -f ins_rdbms.mk dnfs_off

Retry RMAN backup, in my case it took only 1sec:

...
piece handle=/dbbackup/0q0rruhd_26_1_1 tag=TAG20220425T202309 comment=NONE
channel disk1: backup set complete, elapsed time: 00:00:01
Finished backup at 25-APR-22

Database Express Setup: This site can not be reached

Note: These steps are for TEST non-production databases only.

Configuring/enabling EM express on HTTPS should be simple, by running only:

SQL> exec DBMS_XDB_CONFIG.SETHTTPSPORT(5500);

But after configuring it you may still not be able to access the web page and show the error “This site can’t be reached”

The reason for this can be permissions on the wallet files:

[grid@rac1 ~]$ ll /u01/app/oracle/product/19.3.0/dbhome_1/admin/orcl/xdb_wallet
total 8
-rw------- 1 oracle asmadmin 3864 Apr 15 17:40 cwallet.sso
-rw------- 1 oracle asmadmin 3819 Apr 15 17:40 ewallet.p12

The correct permission is 600, but when database is on top of ASM with role separation, then group should also have read permission on these files:

[root@rac1 ~]# chmod 640 /u01/app/oracle/product/19.3.0/dbhome_1/admin/orcl/xdb_wallet/*

After changing it, the web page is displayed.

Monitoring ASM disk performance using IOSTAT

iostat in asmcmd displays I/O statistics for Oracle ASM disks in mounted disk groups.

Connect to the database node via GI owner:

# su - grid

Run iostat with the following options (Reads & Writes are in bytes):

# asmcmd
ASMCMD> iostat -t -G FRA 5
Group_Name  Dsk_Name   Reads      Writes    Read_Time  Write_Time
FRA         RAC1$LUN3  585083392  98942464  94.659862  4.03044
FRA         RAC2$LUN3  1847296    98942464  .054822    4.134049
FRA         RACQ$LUN4  57344      24576     .035944    .018594

Group_Name  Dsk_Name   Reads      Writes  Read_Time  Write_Time
FRA         RAC1$LUN3  368640.00  0.00    0.01       0.00
FRA         RAC2$LUN3  0.00       0.00    0.00       0.00
FRA         RACQ$LUN4  0.00       0.00    0.00       0.00

Where
-t displays time statistics (Read_Time, Write_Time)
-G FRA displays statistics for the FRA diskgroup, change the diskgroup name according to your needs.
5 is a refresh interval. When the interval is specified then the value displayed (bytes or I/Os) is the difference between the previous and current values, not the total value. But if a refresh interval is not specified, the number displayed represents the total number of bytes or I/Os.

For synopsis and description about all available iostat options, run help:

ASMCMD> help iostat
iostat
        Displays I/O statistics for Oracle ASM disks in mounted disk groups.

Synopsis
        iostat [-et][--io] [--suppressheader] [--region] [-G <diskgroup>] [<interval>]

Description
        iostat lists disk group statistics using the V$ASM_DISK_STAT view.
        The options for the iostat command are described below.
        -e		- Displays error statistics (Read_Err, Write_Err).
        -G diskgroup	- Displays statistics for the disk group name.
        --suppressheader	- Suppresses column headings.
        --io		- Displays information in number of I/Os, instead
                          of bytes.
        -t		- Displays time statistics (Read_Time, Write_Time).
        --region	- Displays information for cold and hot disk regions
                          (Cold_Reads, Cold_Writes, Hot_Reads, Hot_Writes).
        interval	- Refreshes the statistics display based on the
                          interval value (seconds).
        The attribute descriptions for iostat command output are described
	below. To view the complete set of statistics for a disk group,
	use the V$ASM_DISK_STAT view.
        Group_Name	        Name of the disk group.
        Dsk_Name	        Name of the disk.
        Reads	        	Total number of bytes read from the disk.
				If the --io option is entered, then the value
				is displayed as number of I/Os.
        Writes	        	Total number of bytes written to the disk.
				If the --io option is entered, then the value
				is displayed as number of I/Os.
        Cold_Reads	        Total number of bytes read from the cold disk
				region. If the --io option is entered, then
				the value is displayed as number of I/Os.
        Writes	        	Total number of bytes written to the disk.
        Cold_Writes	        Total number of bytes written to the cold
				disk region. If the --io option is entered,
				then the value is displayed as number of I/Os.
        Hot_Reads	        Total number of bytes read from the hot
				disk region. If the --io option is entered,
				then the value is displayed as number of I/Os.
        Writes	        	Total number of bytes written to the disk.
        Cold_Writes	        Total number of bytes written to the cold
        Hot_Writes	        Total number of bytes written to the hot disk
				region. If the --io option is entered, then the
				value is displayed as number of I/Os.
        Read_Err	        Total number of failed I/O read requests for
				the disk.
        Write_Err	        Total number of failed I/O write requests for
				the disk.
        Read_Time	        Total I/O time (in seconds) for
				read requests for the disk if the
				TIMED_STATISTICS initialization parameter is
				set to TRUE (0 if set to FALSE).
        Write_Time	        Total I/O time (in seconds) for
				write requests for the disk if the
				TIMED_STATISTICS initialization parameter is
				set to TRUE (0 if set to FALSE).
        Writes	        	Total number of bytes written to the disk.
        Cold_Writes	        Total number of bytes written to the cold
        Hot_Writes	        Total number of bytes written to the hot disk
        If a refresh interval is not specified, the number displayed represents
        the total number of bytes or I/Os.  Ifa refresh interval is specified,
        then the value displayed (bytes or I/Os) is the difference between the
        previous and current values, not the total value.

Examples
        The following are examples of the iostat command. The first example
        displays disk I/O statistics for the data disk group in total number
        of bytes. The second example displays disk I/O statistics for the data
        disk group in total number of I/O operations.
        ASMCMD [+] > iostat -G data
        Group_Name  Dsk_Name   Reads       Writes
        DATA        DATA_0000  180488192   473707520
        DATA        DATA_0001  1089585152  469538816
        DATA        DATA_0002  191648256   489570304
        DATA        DATA_0003  175724032   424845824
        DATA        DATA_0004  183421952   781429248
        DATA        DATA_0005  1102540800  855269888
        DATA        DATA_0006  171290624   447662592
        DATA        DATA_0007  172281856   361337344
        DATA        DATA_0008  173225472   390840320
        DATA        DATA_0009  288497152   838680576
        DATA        DATA_0010  196657152   375764480
        DATA        DATA_0011  436420096   356003840
        ASMCMD [+] > iostat --io -G data
        Group_Name  Dsk_Name   Reads  Writes
        DATA        DATA_0000  2801   34918
        DATA        DATA_0001  58301  35700
        DATA        DATA_0002  3320   36345
        DATA        DATA_0003  2816   10629
        DATA        DATA_0004  2883   34850
        DATA        DATA_0005  59306  38097
        DATA        DATA_0006  2151   10129
        DATA        DATA_0007  2686   10376
        DATA        DATA_0008  2105   8955
        DATA        DATA_0009  9121   36713
        DATA        DATA_0010  3557   8596
        DATA        DATA_0011  17458  9269

DBT-06103 The port (1,521) is already used

Thanks Tornike Kupatadze for this testing case!

Problem:

During my OCA class, after successful 19c database software installation, we were creating a database using dbca and got the following error:

If the listener had already been configured we would have had an error DBT-06103 The port (5,500) is already used while configuring EM express. But still, the solution is the same.

We have checked and the port was not used:

# netstat -a |grep 1521

Reason:

The hostname is not reachable. The reason in our case was that /etc/hosts did not contain entries about this server.

Solution:

Qualify the hostname into the /etc/hosts:

After adding the above entry, we were able to continue.