Postfix: flush emails from the mail queue


The customer was getting a lot system alerts even after disabling it. There were a lot of emails in the queue:

# mailq
-Queue ID-  --Size-- ----Arrival Time---- -Sender/Recipient-------
DB13ED78461   21597 Thu May 26 05:39:26  flashgrid@localhost.localdomain
(delivery temporarily suspended: connect to[~]:25: Connection timed out)

DA31AD9C4B5   21503 Thu May 26 10:32:16  flashgrid@localhost.localdomain
(delivery temporarily suspended: connect to[~]:25: Connection timed out)

DA701D9AC16   21601 Thu May 26 07:01:06  flashgrid@localhost.localdomain
(delivery temporarily suspended: connect to[~]:25: Connection timed out)

DA3B9D9C488   21503 Thu May 26 09:31:36  flashgrid@localhost.localdomain
(connect to[~]:25: Connection timed out)



  • To remove all mail from all the queues ( hold, incoming, active and deferred ) , run :
# postsuper -d ALL
  • To remove all mails in the deferred queue only, run :
# postsuper -d ALL deferred

Terminal window xterm is not displayed in VNC


After installing/starting the tiger VNC server, and connecting using VNC Viewer terminal window is not displayed.

# yum install tigervnc-server


Install xterm:

# yum install xterm

Kill old VNC process and start again:

$ vncserver -kill :1
$ vncserver -geometry 1024x1024

Reconnect using VNC viewer, terminal should be displayed automatically.

Install Google Chrome on Linux 7.9 using terminal

There are several ways to do that, I found the simplest (I hope so) and want to share it with you:

0. Create repo file:

# vi /etc/yum.repos.d/google-chrome.repo


1. Enable repo ol7_optional_latest for vulkan dependency:

# yum-config-manager --enable ol7_optional_latest

2. Install google-chrome-stable package:

# yum install google-chrome-stable -y

3. Run:

$ google-chrome

Or in the background:

$ google-chrome &

The window will come up in VNC or X Window whichever you’ve configured before.

Change default kernel using grubby Tool

There are several ways to fulfill the same task, I am providing one of them.

  1. Check the information about currently loaded kernel:
# uname -r

2. Find all available kernels in your system and locate their index number:

# grubby --info=ALL
args="ro console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 numa=off transparent_hugepage=never net.ifnames=0"
title=Oracle Linux Server 7.9, with Unbreakable Enterprise Kernel 5.4.17-2036.101.2.el7uek.x86_64

args="ro console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 numa=off transparent_hugepage=never net.ifnames=0"
title=Oracle Linux Server 7.9, with Linux 3.10.0-1160.42.2.el7.x86_64

args="ro console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 numa=off transparent_hugepage=never net.ifnames=0"

3. Check currently loaded kernel index using grubby tool (actually, we could find the same from 1st and 2nd steps, but let’s do one more time):

# grubby --default-index

4. Change the default kernel, in my case I want to set it to vmlinuz-3.10.0-1160.42.2.el7.x86_64 and it’s index number is 1:

# grubby --set-default 1

5. Reboot the system and check the kernel again:

# reboot
# uname -r

Useful tools for analyzing OS diagnostic data

Can fs.aio-max-nr be changed on a live system?

Short answer:

Yes, it can be changed without impacting the business operations.
Oracle source: Doc ID 2269728.1
RHEL source:

========================Additional information========================

Additional explanation, why you may need to change that value.


Alert log shows:

ORA-27090: Unable to reserve kernel resources for asynchronous disk I/O
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Additional information: 128
Additional information: 139817340277512


The problem is caused by a lower than recommended value for aio-max-nr. The current value is 1048576, but recommended is 3145728.


Set fs.aio-max-nr to 3145728 in /etc/sysctl.conf.

# grep fs.aio-max-nr /etc/sysctl.conf
fs.aio-max-nr = 3145728

Make a new value effective:

# sysctl -p  /etc/sysctl.conf

Verify the new value:

# sysctl -a|grep fs.aio-max-nr
fs.aio-max-nr = 3145728



Connecting via ssh to the newly created host causes error:

Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in /Users/mari/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/mari/.ssh/known_hosts:315
ECDSA host key for has changed and you have requested strict checking.
Host key verification failed.


I had another server with the same Public IP, so when I connected to the old saver the host identification has been saved in known_hosts. After a while I have removed old server and created a new one and assigned the PIP. The host identification has changed, but old entries were still saved in known_hosts.


Open /Users/mari/.ssh/known_hosts and delete only the line containing mentioned IP ( in my case), save file and retry the connection.
It should work now.

One of the solutions for ORA-27300: OS system dependent operation:fork failed with status: 11


Databases were crashed and alert logs were showing errors:

Fri Nov 12 13:23:39 2021
Process startup failed, error stack:
Errors in file /app/oracle/diag/rdbms/orcl/orcl/trace/orcl_psp0_25852.trc:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn5

We’ve implemented a procedure to take process list (by ps -ef), so output during that time was the following:

oracle    592    1  0 13:57 ?        00:00:00 [oracle] <defunct>
oracle    593    1  0 13:55 ?        00:00:00 [oracle] <defunct>
oracle    615    1  0 13:57 ?        00:00:00 [oracle] <defunct>
oracle    618    1  0 13:57 ?        00:00:00 [oracle] <defunct>

Not only Oracle, but sshd and some other processes were also experiencing the same:

oracle  22335 22331  0 13:52 ?        00:00:00 [ps] <defunct>
oracle  22336 22331  0 13:52 ?        00:00:00 [grep] <defunct>
oracle  22338 22331  0 13:52 ?        00:00:00 [grep] <defunct>
oracle  14389    1  0 13:24 ?        00:00:00 [sshd] <defunct>
oracle  15852    1  0 13:23 ?        00:00:00 [sshd] <defunct>


A large amount of Zombie processes, causing applications to fail.


You may find a lot of recommendations about increasing kernel.pid_max, similarly ORA-27300: OS System Dependent Operation:fork Failed With Status: 11 (Doc ID 1546393.1). Of course, you can make this parameter unlimited, but this will not solve the problem, it will just postpone it.

The reason for high number of defunct processes is described here,

The parent process for our defunct processes was systemd (pid=1) and the version of it was systemd-219-19.el7.x86_64.

The solution is to update systemd to the latest version.

Print the content of multiple differently named files in Linux

If the number of files you are working on is big, then you need automation as soon as possible.
This post describes find -o option, which helps you work on differently named files when their number is big.

For example, if you want to output the content of files physical_block_size and logical_block_size located under /sys/block/*/queue, run the following:

# find /sys/block/*/queue -name physical_block_size -o -name logical_block_size | while read f ; do echo "$f $(cat $f)" ; done

/sys/block/dm-0/queue/physical_block_size 4096
/sys/block/dm-0/queue/logical_block_size 512
/sys/block/dm-1/queue/physical_block_size 512

Where -o means OR.

Useful when working on ASM disks.

Draw graph for Linux sar output using ksar

I’ve recently heard about this tool , as it is said we are learning things until the death (:

Our company is saving sar output in a text file periodicly and after performance or other issues we need to analyze it’s output to find out which resource was busy and when.. analyzing text file is time-consuming and can also cause eye tension.

Output in sar:

00:00:01        CPU      %usr     %nice      %sys   %iowait    %steal      %irq     %soft    %guest    %gnice     %idle
00:10:01        all      3.14      0.00      2.43      1.64      0.00      0.00      0.60      0.00      0.00     92.20
00:10:01          0      3.64      0.00      2.33      4.10      0.00      0.00      1.10      0.00      0.00     88.83

00:00:01      scall/s badcall/s  packet/s     udp/s     tcp/s     hit/s    miss/s   sread/s  swrite/s saccess/s sgetatt/s
00:10:01         0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

00:00:01       totsck    tcpsck    udpsck    rawsck   ip-frag    tcp-tw
00:10:01         5682       656      1783         0         6       502
00:20:01         5651       668      1748         0         0       804

CPU, Network, Disk I/O, etc. activities are logged.

Same text file analyzed by ksar tool and graphycally displayed is the following:

Full list of items that can be seen graphycally are the following:

Now I will show all necessary information that is necessary to use this tool:

1. Download a pre-built jar from GitHub releases page.

2. Run jar on your computer:

   java -jar ksar-5.2.4-b396_gf0680721-SNAPSHOT-all.jar

3. Click Data -> Load from a file…and choose output of sar in a text file

Full information about this tool: