Add new virtual machine in VBox and install Oracle Linux

Intro:

This blog post belongs to my student at Business and Technology University Ivane Metreveli, thank you Ivane for participating in this project.

  1. First of all, you need to download Oracle Linux iso file from edelivery.oracle.com or from oracle.com. After that, run VirtualBox, click New button and create new virtual machine:

2. Set Name of the Virtual Machine and select operation system as follows, click Next

3. Select appropriate RAM amount, 3GB RAM is recommended for normal processing, click on Next button and jump to next step

4. Now, Select Create a virtual hard disk now option and click Create button

5. Select VDI(virtualbox Disk image)

6. Select Dynamically allocated if you don’t want take hard disk space immediately

7. Select file size (disk size for VB) and the location, click Create button to finish virtual machine creation process

8. Virtual machine is already is created. Before we open/start VM, we load iso file in the machne, click Settings and follow me

9. Navigate to Storage and click CD icon,  on the right side of the window, under attributes, click CD icon and add virtual machine’s .iso file.

10. After that, you can click start button

11. Select .iso files or click folder icon and open folder where .iso file is located, select it and click start

12. Next step is OS installation process, here you select Install Oracle linux 7.6 and click enter to start installation process:

13. Select system language and click continue

14. Select installation destiantion

15. Select the disk where you want to install system. You can select virtual disk, that you have created in the previous step or add a new one. Select disk and click Done button;

16. Now all parameter is ready. Click Begin Installation and wait for finishing the process

17. Set password and click Done

18. Installation is in progress, need to wait more

19. Installation proess is finished, click Roboot button and move to the next step:

20. Installation is finised now, you can start working with Oracle Linux:

Install Linux in Virtual Box

Intro:

This blog post belongs to my student at Business and Technology University Saba Lapanashvili, thank you Saba for participating in this project.

Requirements:

– VirtualBox
– Linux iso file ( For example Linux Mint 15 )

Step 1: Choose System Type

– After installing VirtualBox, click New
– Fill the Name field: e.g Linux Mint 15
– Select Type: Linux
– Select Version: Ubuntu

Step 2: Select the Amount of RAM

– Select the amount of RAM, e.g 2048 MB = 2 GB

Step 3: Configure Hard Disk Settings

– Choose Create a virtual hard drive now, to make a virtual disk space
– Select the VDI 
– Choose Dynamically allocated
– Select the amount of hard drive size

Step 4: Choose Linux ISO File

Now we have done hardware settings

– Click Start to launch system
– Choose your system iso file from your computer, for example my system iso file is (linuxmint-15-cinnamon-dvd-64bit.iso)

Step 5: Install Linux and Make Account

– Click on the Install Linux Mint
– And select Erase disk and install Linux Mint
– Then press Install Now
– Make your account
– Press Continue

Step 6: Congratulations

Congratulations now you have Linux on your Windows.

Python: ImportError: No module named typing

Problem:

Installing joblib module is throwing error:

# pip install joblib
...
ImportError: No module named typing

Trying to install typing, returns the same error:

# pip install typing
...
ImportError: No module named typing

Solution:

Install python-typing using yum:

# yum install python-typing

Install azcli on Linux

0. Import gpgkey

$ sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
  1. Create local azure-cli repository information.
# sh -c 'echo -e "[azure-cli]
name=Azure CLI
baseurl=https://packages.microsoft.com/yumrepos/azure-cli
enabled=1
gpgcheck=1
gpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/azure-cli.repo'


2. Install with the yum install command.

$ yum install azure-cli

3. Run the login command.

$ az login

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code MYCODEHERE to authenticate.

4. Go to https://microsoft.com/devicelogin and enter the mentioned code

5. Click Next -> choose your account.

Python: Manipulation on file content located in s3 archived as tar.gz without downloading

Problem:

Need to analyze several values from multiple files that are archived as tar.gz and located on s3. This operation must be performed without downloading or extracting tar.gz

HARBOR: I am neither the Python expert nor the developer, so it is assumed that I am having mistakes in it or script could be written shorter and easier way than I did.
But it satisfies my needs. So please use it as an example only and investigate the content of it.

Hierarcy of the tar.gz file is the following (sample):

-myfile.tar.gz
—folder1.tar.gz
—–flashgrid_cluster
—–files/node_monitor_error.log
—folder2.tar.gz
—–flashgrid_cluster
—–files/node_monitor_error.log

  1. Create extracts3tar.py file with the following content and grant executable permission to that file:

    Note: Update the following entries in the file according to your environment.
AWS_ACCESS_KEY_ID = "my key goes here"    
AWS_SECRET_ACCESS_KEY = "my secret key goes here"

Content of extracts3tar.py:

#!/usr/bin/python2.7
import boto3
import tarfile
import joblib
import io
import sys

class S3Loader(object):
    AWS_ACCESS_KEY_ID = "my key goes here"
    AWS_SECRET_ACCESS_KEY = "my secret key goes here"
    AWS_REGION_NAME = "us-east-1"
    AWS_STORAGE_BUCKET_NAME = "flashgrid-support"
    def __init__(self):
        self.s3_client = boto3.client("s3",
                                     aws_access_key_id=self.AWS_ACCESS_KEY_ID,
                                     aws_secret_access_key=self.AWS_SECRET_ACCESS_KEY)

    def load_tar_file_s3_into_object_without_download(self, s3_filepath):

        # Describing variables search pattern
        match = ("Disk latency above threshold")
        notmatch = (".lun")

        s3_object = self.s3_client.get_object(Bucket=self.AWS_STORAGE_BUCKET_NAME, Key=s3_filepath)
        wholefile = s3_object['Body'].read()
        fileobj = io.BytesIO(wholefile)

        # Opening first tar.gz file
        tar = tarfile.open(fileobj=fileobj)

        # Searching nested tar.gz files
        childgz = [f.name for f in tar.getmembers() if f.name.endswith('.gz')]

        # Extracting file named flashgrid_cluster which is located in the first tar.gz
        node1gz = tarfile.open(fileobj=tar.extractfile(childgz[0]))
        fgclustername = [f.name for f in node1gz.getmembers() if f.name.endswith('flashgrid_cluster')]
        fgclusternamecontent = node1gz.extractfile(fgclustername[0])

        # Extracting text that contains string "Cluster Name:"
        for fgclusternameline in fgclusternamecontent:
           if "Cluster Name:" in fgclusternameline:
             clustername=fgclusternameline
#        print(len(childgz))
#        print(clustername)
#        print(childgz)
#        nodegzlist=list('')
#        nodemonfilelist=list('')

# Extracting file node_monitor_error.log from all nested tar.gz files
        for i in childgz:
#          nodegzlist.append(tarfile.open(fileobj=tar.extractfile(i)))
           cur_gz_file_extracted = tarfile.open(fileobj=tar.extractfile(i))
#           print(tarfile.open(fileobj=tar.extractfile(i)).getmembers())
           cur_node_mon_file = [f.name for f in cur_gz_file_extracted.getmembers() if f.name.endswith('node_monitor-error.log')]

# Path to node_monitor_error.log contains hostname inside so extracting string that is the hostname
           cur_node_name = cur_node_mon_file[0].split("/")[0]
#           print(cur_node_name)
#           nodemonfilelist.append([f.name for f in curfile.getmembers() if f.name.endswith('node_monitor-error.log')])
#           print(nodemonfilelist[0],nodemonfilelist[1],nodemonfilelist[2])

# Extracting content of node_monitor_error.log file
           cur_node_mon_file_content = cur_gz_file_extracted.extractfile(cur_node_mon_file[0])
#           print(cur_node_mon_file_content)
#        fgclusternamecontent = node1gz.extractfile(fgclustername[0])

#        for fgclusternameline in fgclusternamecontent:
#           if "Cluster Name:" in fgclusternameline:
#             clustername=fgclusternameline

# Selecting lines from the extracted file and filtering based on match criteria (match, notmatch variables)
           for cur_node_mon_file_content_line in cur_node_mon_file_content:
            if match in cur_node_mon_file_content_line and not (notmatch in cur_node_mon_file_content_line):
               # Extracting time from the string, knowing the exact position
               time = cur_node_mon_file_content_line.split(" ")[0] + " " + cur_node_mon_file_content_line.split(" ")[1]
               cur_node_mon_file_line_splitted = cur_node_mon_file_content_line.split(" ")
               # Extracting necessary values after spliting the content by delimiter " "
               print(clustername.strip(),cur_node_name,cur_node_mon_file_line_splitted[8] , time,  cur_node_mon_file_line_splitted[17] + " " + cur_node_mon_file_line_splitted[18].strip())
#               print(nodemonfileline)

if __name__ == "__main__":
    s3_loader = S3Loader()
    try:

     # Script takes 1 argument
      s3_loader.load_tar_file_s3_into_object_without_download(s3_filepath=str(sys.argv[1]))

    except:
     pass

2. Run .py file and pass path of the tar.gz file

# ./extracts3tar.py "mybucketname/myfoldername/myfile.tar.gz"

So the search is happening for flashgrid_cluster and node_monitor_error.log file content, for which two nested tar.gz should be analyzed.

RMAN restore on ASM fails ORA-12547: TNS:lost contact, WARNING: ASMB0 exiting with error

Problem:

Our client was not able to restore controlfile on +FRA diskgroup:

RMAN> restore controlfile to '+FRA' from '<backup file location>';
....
RMAN-03002 failure of restore command at 2021/05/24 19:12:19
ORA-19870: error while restoring backup piece 
ORA-19504: failed to create file '+FRA'
ORA-17502: ksfdcre:4 Failed to create file +FRA
ORA-15001: diskgroup "FRA" does not exist or is not mounted
ORA-12547: TNS:lost contact
ORA-12547: TNS:lost contact

Reason:

oracle binary under GI home did not have correct permissions:

[grid@rac1 bin]$ ll oracle
-rwxr-x--x 1 grid oinstall 420332360 Oct 13  2020 oracle

Solution:

Change permissions to 6751 and retry the restore:

[grid@rac1 bin]$ chmod 6751 oracle

[grid@rac1 bin]$ ll oracle
-rwsr-s--x 1 grid oinstall 420332360 Oct 13  2020 oracle

rpm -qa gets thread died in Berkeley DB library

Problem:

After checking if flashgrid-clan package was installed, got this error:

error: rpmdb: BDB0113 Thread/process 2884/140438918064192 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 - (-30973)
error: cannot open Packages database in /var/lib/rpm
error: rpmdb: BDB0113 Thread/process 2884/140438918064192 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages database in /var/lib/rpm
package flashgrid-clan is not installed

Reason:

If you see rpmdb errors during package management (rpm, yum), it means that the RPM database is corrupted.

Solution:

# mkdir /var/lib/rpm/backup
# cp -a /var/lib/rpm/__db* /var/lib/rpm/backup/
# rm -f /var/lib/rpm/__db.[0-9][0-9]*
# rpm --quiet -qa
# rpm --rebuilddb
# yum clean all

SQL Developer ORA-00604: error occurred at recursive SQL level 1 ORA-12705: Cannot access NLS data files or invalid environment specified

Problem:

Connection from the SQL Developer fails with the following error:

ORA-00604: error occurred at recursive SQL level 1 ORA-12705: Cannot access NLS data files or invalid environment specified

Solution:

Find sqldeveloper.conf file:

$ find / -name sqldeveloper.conf 2>>/dev/null

^@/System/Volumes/Data/Applications/SQLDeveloper.app/Contents/Resources/sqldeveloper/sqldeveloper/bin/sqldeveloper.conf
/Applications/SQLDeveloper.app/Contents/Resources/sqldeveloper/sqldeveloper/bin/sqldeveloper.conf

Add the following two parmeters in /Applications/SQLDeveloper.app/Contents/Resources/sqldeveloper/sqldeveloper/bin/sqldeveloper.conf

AddVMOption -Duser.language=en
AddVMOption -Duser.country=US

Restart SQL Developer and try connection again.

Backup and restore LUN0 and OS disk in Azure

==================================Backup process


1. Configure backup for rac2 which includes only LUN0 (+ OS disk, by default)

Backup configuration script from CLI: 

az backup protection enable-for-vm --resource-group marirac --vault-name vault485 --vm rac2 --policy-name  mkpolicy --disk-list-setting include --diskslist 0

Check settings from Portal: 


2. Create test files on / and /u01

[root@rac2 ~]# touch /u01/mari_u01
[root@rac2 ~]# ll /u01/mari_u01
-rw-r–r– 1 root root 0 Jun  2 15:24 /u01/mari_u01

[root@rac2 ~]# touch /mari_root
[root@rac2 ~]# ll /mari_root
-rw-r–r– 1 root root 0 Jun  2 15:24 /mari_root



==================================Restore process

1. Find the latest restore point from Azure CLI:

az backup recoverypoint list --vault-name vault485 --resource-group marirac -c rac2 -i rac2 --backup-management-type AzureIaasVM


2. Restore OS and LUN0 disks:

az backup restore restore-disks --resource-group marirac --vault-name vault485 -c rac2 -i rac2  --target-resource-group marirac --storage-account mzx41qmarivol2asrcache --diskslist 0 --rp-name 5638874691908752817


The above script restores them as managed disks, that are not attached to anything. Note restore did not replace the disk, it just restores as a new disk. 


3. To replace disks do the following:

Detach old LUN0 -> Save -> Attach restored disk as Lun 0. 

To replace existing OS disk with restored one, click Switch OS disk and then start the VM. 



4. Check that files are not there anymore:

[root@rac2 ~]#  ll /mari_root
ls: cannot access /mari_root: No such file or directory

[root@rac2 ~]# ll /u01/mariu01
ls: cannot access /u01/mariu01: No such file or directory

Find 5 biggest files in Linux

I have used this command many times, but the interval between each usage is so big that I almost always forget the syntax.

So here it is:

# du -a / | sort -n -r | head -n 5

51190272	/
37705424	/root
33040524	/root/apache-tomcat-7.0.53
32802516	/root/apache-tomcat-7.0.53/logs
32802440	/root/apache-tomcat-7.0.53/logs/catalina.out