Add new virtual machine in VBox and install Oracle Linux


This blog post belongs to my student at Business and Technology University Ivane Metreveli, thank you Ivane for participating in this project.

  1. First of all, you need to download Oracle Linux iso file from or from After that, run VirtualBox, click New button and create new virtual machine:

2. Set Name of the Virtual Machine and select operation system as follows, click Next

3. Select appropriate RAM amount, 3GB RAM is recommended for normal processing, click on Next button and jump to next step

4. Now, Select Create a virtual hard disk now option and click Create button

5. Select VDI(virtualbox Disk image)

6. Select Dynamically allocated if you don’t want take hard disk space immediately

7. Select file size (disk size for VB) and the location, click Create button to finish virtual machine creation process

8. Virtual machine is already is created. Before we open/start VM, we load iso file in the machne, click Settings and follow me

9. Navigate to Storage and click CD icon,  on the right side of the window, under attributes, click CD icon and add virtual machine’s .iso file.

10. After that, you can click start button

11. Select .iso files or click folder icon and open folder where .iso file is located, select it and click start

12. Next step is OS installation process, here you select Install Oracle linux 7.6 and click enter to start installation process:

13. Select system language and click continue

14. Select installation destiantion

15. Select the disk where you want to install system. You can select virtual disk, that you have created in the previous step or add a new one. Select disk and click Done button;

16. Now all parameter is ready. Click Begin Installation and wait for finishing the process

17. Set password and click Done

18. Installation is in progress, need to wait more

19. Installation proess is finished, click Roboot button and move to the next step:

20. Installation is finised now, you can start working with Oracle Linux:

Install Linux in Virtual Box


This blog post belongs to my student at Business and Technology University Saba Lapanashvili, thank you Saba for participating in this project.


– VirtualBox
– Linux iso file ( For example Linux Mint 15 )

Step 1: Choose System Type

– After installing VirtualBox, click New
– Fill the Name field: e.g Linux Mint 15
– Select Type: Linux
– Select Version: Ubuntu

Step 2: Select the Amount of RAM

– Select the amount of RAM, e.g 2048 MB = 2 GB

Step 3: Configure Hard Disk Settings

– Choose Create a virtual hard drive now, to make a virtual disk space
– Select the VDI 
– Choose Dynamically allocated
– Select the amount of hard drive size

Step 4: Choose Linux ISO File

Now we have done hardware settings

– Click Start to launch system
– Choose your system iso file from your computer, for example my system iso file is (linuxmint-15-cinnamon-dvd-64bit.iso)

Step 5: Install Linux and Make Account

– Click on the Install Linux Mint
– And select Erase disk and install Linux Mint
– Then press Install Now
– Make your account
– Press Continue

Step 6: Congratulations

Congratulations now you have Linux on your Windows.

Python: ImportError: No module named typing


Installing joblib module is throwing error:

# pip install joblib
ImportError: No module named typing

Trying to install typing, returns the same error:

# pip install typing
ImportError: No module named typing


Install python-typing using yum:

# yum install python-typing

Python: Manipulation on file content located in s3 archived as tar.gz without downloading


Need to analyze several values from multiple files that are archived as tar.gz and located on s3. This operation must be performed without downloading or extracting tar.gz

HARBOR: I am neither the Python expert nor the developer, so it is assumed that I am having mistakes in it or script could be written shorter and easier way than I did.
But it satisfies my needs. So please use it as an example only and investigate the content of it.

Hierarcy of the tar.gz file is the following (sample):


  1. Create file with the following content and grant executable permission to that file:

    Note: Update the following entries in the file according to your environment.
AWS_ACCESS_KEY_ID = "my key goes here"    
AWS_SECRET_ACCESS_KEY = "my secret key goes here"

Content of

import boto3
import tarfile
import joblib
import io
import sys

class S3Loader(object):
    AWS_ACCESS_KEY_ID = "my key goes here"
    AWS_SECRET_ACCESS_KEY = "my secret key goes here"
    AWS_REGION_NAME = "us-east-1"
    AWS_STORAGE_BUCKET_NAME = "flashgrid-support"
    def __init__(self):
        self.s3_client = boto3.client("s3",

    def load_tar_file_s3_into_object_without_download(self, s3_filepath):

        # Describing variables search pattern
        match = ("Disk latency above threshold")
        notmatch = (".lun")

        s3_object = self.s3_client.get_object(Bucket=self.AWS_STORAGE_BUCKET_NAME, Key=s3_filepath)
        wholefile = s3_object['Body'].read()
        fileobj = io.BytesIO(wholefile)

        # Opening first tar.gz file
        tar =

        # Searching nested tar.gz files
        childgz = [ for f in tar.getmembers() if'.gz')]

        # Extracting file named flashgrid_cluster which is located in the first tar.gz
        node1gz =[0]))
        fgclustername = [ for f in node1gz.getmembers() if'flashgrid_cluster')]
        fgclusternamecontent = node1gz.extractfile(fgclustername[0])

        # Extracting text that contains string "Cluster Name:"
        for fgclusternameline in fgclusternamecontent:
           if "Cluster Name:" in fgclusternameline:
#        print(len(childgz))
#        print(clustername)
#        print(childgz)
#        nodegzlist=list('')
#        nodemonfilelist=list('')

# Extracting file node_monitor_error.log from all nested tar.gz files
        for i in childgz:
#          nodegzlist.append(
           cur_gz_file_extracted =
#           print(
           cur_node_mon_file = [ for f in cur_gz_file_extracted.getmembers() if'node_monitor-error.log')]

# Path to node_monitor_error.log contains hostname inside so extracting string that is the hostname
           cur_node_name = cur_node_mon_file[0].split("/")[0]
#           print(cur_node_name)
#           nodemonfilelist.append([ for f in curfile.getmembers() if'node_monitor-error.log')])
#           print(nodemonfilelist[0],nodemonfilelist[1],nodemonfilelist[2])

# Extracting content of node_monitor_error.log file
           cur_node_mon_file_content = cur_gz_file_extracted.extractfile(cur_node_mon_file[0])
#           print(cur_node_mon_file_content)
#        fgclusternamecontent = node1gz.extractfile(fgclustername[0])

#        for fgclusternameline in fgclusternamecontent:
#           if "Cluster Name:" in fgclusternameline:
#             clustername=fgclusternameline

# Selecting lines from the extracted file and filtering based on match criteria (match, notmatch variables)
           for cur_node_mon_file_content_line in cur_node_mon_file_content:
            if match in cur_node_mon_file_content_line and not (notmatch in cur_node_mon_file_content_line):
               # Extracting time from the string, knowing the exact position
               time = cur_node_mon_file_content_line.split(" ")[0] + " " + cur_node_mon_file_content_line.split(" ")[1]
               cur_node_mon_file_line_splitted = cur_node_mon_file_content_line.split(" ")
               # Extracting necessary values after spliting the content by delimiter " "
               print(clustername.strip(),cur_node_name,cur_node_mon_file_line_splitted[8] , time,  cur_node_mon_file_line_splitted[17] + " " + cur_node_mon_file_line_splitted[18].strip())
#               print(nodemonfileline)

if __name__ == "__main__":
    s3_loader = S3Loader()

     # Script takes 1 argument


2. Run .py file and pass path of the tar.gz file

# ./ "mybucketname/myfoldername/myfile.tar.gz"

So the search is happening for flashgrid_cluster and node_monitor_error.log file content, for which two nested tar.gz should be analyzed.

rpm -qa gets thread died in Berkeley DB library


After checking if flashgrid-clan package was installed, got this error:

error: rpmdb: BDB0113 Thread/process 2884/140438918064192 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 - (-30973)
error: cannot open Packages database in /var/lib/rpm
error: rpmdb: BDB0113 Thread/process 2884/140438918064192 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages database in /var/lib/rpm
package flashgrid-clan is not installed


If you see rpmdb errors during package management (rpm, yum), it means that the RPM database is corrupted.


# mkdir /var/lib/rpm/backup
# cp -a /var/lib/rpm/__db* /var/lib/rpm/backup/
# rm -f /var/lib/rpm/__db.[0-9][0-9]*
# rpm --quiet -qa
# rpm --rebuilddb
# yum clean all

Find 5 biggest files in Linux

I have used this command many times, but the interval between each usage is so big that I almost always forget the syntax.

So here it is:

# du -a / | sort -n -r | head -n 5

51190272	/
37705424	/root
33040524	/root/apache-tomcat-7.0.53
32802516	/root/apache-tomcat-7.0.53/logs
32802440	/root/apache-tomcat-7.0.53/logs/catalina.out

Create shortcuts for frequently accessed servers

Life is too short, that’s why it’s mandatory to use shortcuts… Instead of typing frequently used ssh client options such as port, user, hostname, identity-file and so on, you can save that information in sshd config file and then access it with defined alias.

  • System wide config file location is /etc/ssh/ssh_config
  • User specific config file location is ~/.ssh/config same as $HOME/.ssh/config

Instead of connecting to the server everytime using the following command:

# ssh root@ -i ~/.ssh/my_id_rsa

Save the following entries in ~/.ssh/config file:

# vim ~/.ssh/config
Host my_db
IdentityFile ~/.ssh/my_id_rsa
User root

And connect to the server using this simple way:

# ssh my_db

For other options check

‘udev’ rules continuously being reloaded resulted in ASM nvme disks going offline


Linux Server release 7.2


When Oracle processes are opening the device for writing and then closing it, this synthesizes a change event. And udev rules having  ACTION=="add|change" gets reloaded. This behavior causes ASM nvme disks to go offline:

Thu Jul 09 16:33:16 2020
WARNING: Disk 18 (rac1$disk1) in group 2 mode 0x7f is now being offlined

Fri Jul 10 10:04:34 2020
WARNING: Disk 19 (rac1$disk5) in group 2 mode 0x7f is now being offlined

Fri Jul 10 13:45:45 2020
WARNING: Disk 15 (rac1$disk8) in group 2 mode 0x7f is now being offlined


To suppress the false positive change events disable the inotify watch for devices used for Oracle ASM using following steps:

  1.  Create /etc/udev/rules.d/96-nvme-nowatch.rules file with just one line in it:
ACTION=="add|change", KERNEL=="nvme*", OPTIONS:="nowatch"

2. After creating the file please run the following to activate the rule:

# udevadm control --reload-rules
# udevadm trigger --type=devices --action=change

The above command will reload the complete udev configuration and will trigger all the udev rules. On a busy production system this could disrupt ongoing operations, applications running on the server. Please use the above command during a scheduled maintenance window only.

Source: + our experience with customers.

Retrieving the Public Key for Key Pair on Linux or Mac OS

ssh-keygen command can be used on Linux or Mac OS to retrieve the public key from the private SSH key:

$ ssh-keygen -y -f MyKeyPair.pem 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQChauLwkBK/vIiFY/t7uY6lzxESqZkZNvCAA3L42OH2fWzKptqGF+N32zjmLLSPFpYjoEHoHpi5e7yypTmiljtHcKUTwJTs3xclQrApCQvR+LneOi/5P5WaYl61G76osJesXiunLTa+RVr3LDR96LjPcql7JDnuh1RFhDqZ87nDfcGmXGV8iG7w3bk3R/2LuzzMYTgEVdv91S1OF1roH1baPXSV8MaYbOKhMUqV61+eP6/F5ZhT5Gk0BKX1KnQ3/gbgMqjMWRMZzYUeVjUbC52lYwrrBTQX5tHphAJtOTNJ/CpyuEuZ7ED+XYhX9Q1DNOZ47K51xbg5lsnyOBYSUqHz

-y – This option will read a private OpenSSH format file and print its public key.
-f – Specifies the filename of the key file.

Boot in single user mode and rescue your RHEL7


One of our customer incorrectly changed fstab file and rebooted the OS. As a result, VM was not able to start. Fortunately, cloud where this VM was located supported serial console.


We booted in single user mode through serial console and reverted the changes back. To boot in single user mode and update necessary file, do as follows:

Connect to the serial console and while OS is booting in a grub menu press e to edit the selected kernel:

Find line that starts with linux16 ( if you don’t see it press arrow down ), go to the end of this line and type rd.break.

Press ctrl+x.

Wait for a while and system will enter into single user mode:

During this time /sysroot is mounted in read only mode, you need to remount it in read write:

switch_root:/# mount -o remount,rw /sysroot
switch_root:/# chroot /sysroot

You can revert any changes back by updating any file, in our case we updated fstab:

sh-4.2# vim /etc/fstab

You are a real hero, because you rescued your system!