Oracle&Linux 4ever
August 1, 2020 Leave a comment
There are several ways to identify that, I will suggest one of them using rpm -qf, that finds out what package a file belongs to:
Oracle Linux:
# rpm -qf /etc/redhat-release oraclelinux-release-7.8-1.0.7.el7.x86_64
RHEL:
# rpm -qf /etc/redhat-release redhat-release-server-7.8-2.el7.x86_64
August 26, 2019 Leave a comment
Problem:
One of our customer incorrectly changed fstab file and rebooted the OS. As a result, VM was not able to start. Fortunately, cloud where this VM was located supported serial console.
Solution:
We booted in single user mode through serial console and reverted the changes back. To boot in single user mode and update necessary file, do as follows:
Connect to the serial console and while OS is booting in a grub menu press e to edit the selected kernel:
Find line that starts with linux16 ( if you don’t see it press arrow down ), go to the end of this line and type rd.break.
Press ctrl+x.
Wait for a while and system will enter into single user mode:
During this time /sysroot is mounted in read only mode, you need to remount it in read write:
switch_root:/# mount -o remount,rw /sysroot switch_root:/# chroot /sysroot
You can revert any changes back by updating any file, in our case we updated fstab:
sh-4.2# vim /etc/fstab
You are a real hero, because you rescued your system!
August 12, 2019 Leave a comment
Problem:
Oracle 18c GI configuration precheck was failing with the following error:
Summary of node specific errors primrac2 - PRVG-11069 : IP address "169.254.0.2" of network interface "idrac" on the node "primrac2" would conflict with HAIP usage. - Cause: One or more network interfaces have IP addresses in the range (169.254..), the range used by HAIP which can create routing conflicts. - Action: Make sure there are no IP addresses in the range (169.254..) on any network interfaces. primrac1 - PRVG-11069 : IP address "169.254.0.2" of network interface "idrac" on the node "primrac1" would conflict with HAIP usage. - Cause: One or more network interfaces have IP addresses in the range (169.254..), the range used by HAIP which can create routing conflicts. - Action: Make sure there are no IP addresses in the range (169.254..) on any network interfaces.
On each node additional network interface – named idrac was started with the ip address 169.254.0.2. I tried to set static ip address in /etc/sysconfig/network-scripts/ifcfg-idrac , also tried to bring the interface down – but after some time interface was starting up automatically and getting the same ip address.
Cluster nodes were DELL servers with Dell Remote Access Controller(iDRAC) Service Module installed. For more information about this module installation/deinstallation… can be found here https://topics-cdn.dell.com/pdf/idrac-service-module-v32_users-guide_en-us.pdf
Servers were configured by system administrator and was not clear why this module was there, we are not using iDRAC module and the only option that we had was to remove/uninstall that module. (configuring module should also be possible to avoid such situation, but we keep our servers as clean as possible without having unsed services)
Solution:
Uninstalled iDRAC module (also expained in the above pdf):
# rpm -e dcism
After uninstalling it idrac interface did not started anymore, so we could continue GI configuration.
July 22, 2019 Leave a comment
Problem:
During my previous installations I used the following udev rule on multipath devices:
KERNEL=="dm-[0-9]*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="360050768028200a9a40000000000001c", NAME="oracleasm/asm-disk1", OWNER="oracle", GROUP="asmadmin", MODE="0660"
So to identify the exact disk I used PROGRAM option. The above script looks through `/dev/dm-*` devices and if any of them satisfy the condition, for example:
# scsci_id -gud /dev/dm-3 360050768028200a9a40000000000001c
then device name will be changed to /dev/oracleasm/asm-disk1, owner:group to grid:asmadmin and permission to 0660
But on my new servers same udev rule was not working anymore. (Of course, it needs more investigation, but our time is really valuable and never enough and if we know another solution that works and is acceptable- let’s just use it)
Solution:
I used udevadm command to identify other properties of these devices and wrote new udev rule (to see all properties, just remove grep):
# udevadm info --query=property --name /dev/mapper/asm1 | grep DM_UUID DM_UUID=mpath-360050768028200a9a40000000000001c
New udev rule looks like this:
# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
ENV{DM_UUID}=="mpath-360050768028200a9a40000000000001c", SUBSYSTEM=="block", NAME="oracleasm/asm-disk1", OWNER="grid", GROUP="asmadmin", MODE="0660"
Trigger udev rules:
# udevadm trigger
Verify that name, owner, group and permissions are changed:
# ll /dev/oracleasm/
total 0
brw-rw---- 1 grid asmadmin 253, 3 Jul 17 17:33 asm-disk1
June 19, 2019 Leave a comment
Problem:
There are a lot of warnings “kernel: serial8250: too much work for irq4 ” in /var/log/messages and are likely your system experiences stability problems. And can lead to Oracle cluster node evictions.
Cause:
The problem was related to Azure OAM Agent pushing very large messages to serial console. The problem was introduced by the latest update of the Azure OMS agent.
Temporary Solution:
Temporarily remove OMS Linux Agent Extension until Microsoft resolves this bug:
1. On Azure portal click the link of the affected VM.
2. Click the “Extensions” section.
3. Click the OMS Linux Agent in the list.
4. Click the “Uninstall” button at the top
When you make sure that OMS agent bug is fixed (should be verified with Microsoft support), then you can reinstall the pluggin.
June 13, 2019 1 Comment
Problem:
I am not able to receive email alerts from database server. Because message transfer agent is trying to connect to the Google SMTP via IPv6, which fails.
# tail /var/log/maillog Jun 12 15:35:10 rac1 postfix/smtp[19725]:connect to gmail-smtp-in.l.google.com [2607:f8b0:400c:c0b::1a]:25: Network is unreachable
Solution:
Configure Postfix not to use IPv6 by editing /etc/postfix/main.cf with the following:
[root@rac1 ~]# cat /etc/postfix/main.cf | grep inet_protocols inet_protocols = ipv4
Restart Postfix and check the status:
[root@rac1 ~]# systemctl restart postfix
[root@rac1 ~]# systemctl status postfix
● postfix.service - Postfix Mail Transport Agent
Loaded: loaded (/usr/lib/systemd/system/postfix.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2019-06-13 10:20:48 UTC; 52s ago
Process: 17431 ExecStop=/usr/sbin/postfix stop (code=exited, status=0/SUCCESS)
Process: 17449 ExecStart=/usr/sbin/postfix start (code=exited, status=0/SUCCESS)
Process: 17445 ExecStartPre=/usr/libexec/postfix/chroot-update (code=exited, status=0/SUCCESS)
Process: 17442 ExecStartPre=/usr/libexec/postfix/aliasesdb (code=exited, status=0/SUCCESS)
Main PID: 17520 (master)
Memory: 3.0M
CGroup: /system.slice/postfix.service
├─17520 /usr/libexec/postfix/master -w
├─17521 pickup -l -t unix -u
└─17522 qmgr -l -t unix -u
Jun 13 10:20:48 rac1.example.com systemd[1]: Starting Postfix Mail Transport Agent…
Jun 13 10:20:48 rac1.example.com postfix/postfix-script[17518]: starting the Postfix mail system
Jun 13 10:20:48 rac1.example.com postfix/master[17520]: daemon started -- version 2.10.1, configuration /etc/postfix
Jun 13 10:20:48 rac1.example.com systemd[1]: Started Postfix Mail Transport Agent
December 16, 2016 2 Comments
In our database there is turned on auditing on some operations and audit records go to OS.
SYS> show parameter audit_file_dest NAME TYPE VALUE ------------------ ----------- ------------------------------ audit_file_dest string /u01_log/audit/orcl SYS > show parameter audit_trail NAME TYPE VALUE ------------- ----------- ----------- audit_trail string OS
Our security administrators are using SIEM to monitor suspicious activities and they want database to send audit records to this third party tool.
I thought that I could somehow indicate directory “/u01_log/audit/orcl” from where *.aud files would be uploaded to SIEM, but I was wrong. Some tools may be able to use these *.aud files but not SIEM and let’s configure our database to be able to send audit records to it.
1. Connect to a database instance as sysdba user
SQL> connect / as sysdba2. Set audit trail to OS
SQL> alter system set audit_trail=OS;3. Enable auditing for system users if you need to audit activities of sys user(optional)
SQL> alter system set audit_sys_operations=TRUE;4. Set rsyslog facility and severity(needs database restart)
SQL> alter system set audit_syslog_level=local5.info scope=spfile sid='*';
5. Restart database
SQL> shutdown immediate;
SQL> startup;
6. Edit rsyslog.conf file
#Saving oracle database audit recordslocal5.info /u01_log/audit/RSYSLOG/dbaudit.log#Send oracle database audit trail to remote rsyslog serverlocal5.info @192.168.0.15
7. Restart rsyslog service
# service rsyslog restart
Shutting down system logger: [ OK ]
Starting system logger: [ OK ]
8. It is better to limit the size for audit log, or it may fill the space:
# vi /etc/logrotate.d/oracle.audit
#Created by MariK
/u01_log/audit/RSYSLOG/dbaudit.log {
rotate 3
compress
missingok
notifempty
size 40G
postrotate
service rsyslog restart
endscript
}
To check the syntax run :
# logrotate /etc/logrotate.d/oracle.audit
It will say if you have an error. If syntax is ok then output is nothing.
September 29, 2015 Leave a comment
Windoooooowwws , it makes me crazy some times 🙂
Sometimes(it’s my third time) I have problems with windows vpn, getting error 720.
It has a “simple”(when you know(:) solution but every time I come across this problem I forget how I solved it before and decided to write solution here.
In Device Manager under “Network adapters” , there is yellow exclamation mark over “WAN Miniport (IP)” and “WAN Miniport(Network Monitor)”
Let’s uninstall it.
Uninstalling these miniports are not allowed. So first of all you should change it with different driver.
Locate your mouse on these driver and right click -> Update Driver Software->Browse my computer for driver software->Let me pick from a list of …-> uncheck “Show compatible hardware” and then choose for example Philips ->
When instead of WAN Miniport (IP) there appears the above driver , you can uninstall it.
Do the same steps for Network Monitor driver.
Now we do not have these drivers with exclamation marks.. because we do not have them at all 🙂
Now we are going to install these drivers with devcon.exe utility.
Download devcon 64bit version from http://static.miklos.ca/devcon.zip
Run CMD with administrator privileges, go to the directory where devcon.exe file exist and run:
devcon.exe install c:\windows\inf\netrasa.inf MS_NDISWANIP
Good Luck, it should now be working.
April 2, 2013 2 Comments
1. Check if you have already installed device-mapper-multipath rpm, if not then install it.
rpm -qa device-mapper-multipath
2. If /etc/multipath.conf file doesn’t exist, then copy it from /usr/share/doc/device-mapper-multipath-*
cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/multipath.conf
3. Find WWIDs that should be added to multipath configuration.
# scsi_id -g -u /dev/sdb
36001438009b044d90000900000780000
4. Edit the /etc/multipath.conf configuration file
defaults {
user_friendly_names yes
path_grouping_policy failover
}blacklist {
wwid "*"
}blacklist_exceptions {
wwid "36001438009b044d90000900000780000"
}multipaths {
multipath {
wwid "36001438009b044d90000900000780000"
alias asm1
}
}
5. Add module to the Linux kernel:
modprobe dm-multipath
6. Start multipath service:
service multipathd start
7. If you have any syntax errors or any parameters that do not work in your Linux version, the following command will show:
multipath -d
8. Commit the configuration:
multipath -v2
9. The following command must find the paths , or you have a bad configuration in multipath.conf file:
multipath -ll
10. Make devices configured after a reboot:
chkconfig multipathd on
If you have made any mistakes in multipath.conf file then correct them and do the following steps to make changes take affect :
1. edit the /etc/multipath.conf
2. Reload the multipath service:
service multipathd reload
3. Remove all unused multipath devices
multipath -F
4. Check again that syntax is correct:
multipath –d
5. Commit the changes:
multipath –v2
Note that, this configuration is very simple, but it is working also perfectly.
For more multipath options and more sophisticated configuration, see the following documentation.
