Copy a file over SSH without SCP

Problem:

/usr/bin/scp binary was removed from the system. Which has caused the Oracle Patching process to fail.

scp binary is provided by openssh-clients rpm, which was present on the system, but scp binary was missing.

Troubleshooting/Testing:

The workaround is to copy scp binary from a similar healthy server (keep the same version). However, transferring a file to a location where it doesn’t exist can be a bit challenging. Let’s try:

[fg@rac1 ~]$ scp /usr/bin/scp racq:/tmp/scp
bash: scp: command not found
lost connection

We got lost connection, because scp is not on racq node.

Solution:

Need to use ssh and cat commands. For most systems root user login is not enabled, so you need to place the file under /tmp and then relocate to the correct location.

In my example, I have already set up fg user equivalency, so in my case, the format will be the following:

[fg@rac1 ~]$ ssh racq cat < /usr/bin/scp ">" /tmp/scp

Connect to the remote server and copy /tmp/scp to the correct location. Reset permissions.

[root@racq tmp]# cp /tmp/scp /usr/bin/scp
[root@racq tmp]# chmod 755 /usr/bin/scp
[root@racq tmp]# chown root:root /usr/bin/scp

The transfer should be working now:

[fg@rac1 ~]$ scp /usr/bin/scp racq:/tmp/scp
scp      100%   89KB  44.0MB/s   00:00

The process worked for a binary file, so it will work for a text file too.

aad_certhandler: The public key is of type ssh-rsa, not a certificate.

Problem:

I am currently experiencing difficulty connecting to the Azure VM using AD authentication. I am receiving an error message that states:

~ az ssh  vm -n rac1 -g marirac
OpenSSH_9.0p1, LibreSSL 3.3.6
...: Permission denied (publickey).
The OpenSSH server version in the target VM 7.4 is too old. Version incompatible with OpenSSH client version 9.0. Refer to https://bugzilla.mindrot.org/show_bug.cgi?id=3351 for more information.

When I check /var/log/secure log on the server side, it shows:

Jun  6 16:49:54 rac1 sshd[27249]: Connection closed by ... port 51572 [preauth]
Jun  6 16:54:44 rac1 sshd[31386]: nss_aad: This is an Azure machine
Jun  6 16:54:44 rac1 aad_certhandler[31393]: Version: 1.0.022600002; user: ...
Jun  6 16:54:44 rac1 aad_certhandler[31393]: The public key is of type ssh-rsa, not a certificate.
Jun  6 16:54:44 rac1 sshd[31386]: error: AuthorizedKeysCommand /usr/sbin/aad_certhandler ...
Jun  6 16:54:44 rac1 sshd[31386]: Connection closed by ... port 52092 [preauth]

I have intentionally redacted certain portions of the information for security reasons, although errors are still apparent.

Troubleshooting:

Client and server versions are different:

Server:

[root@rac1 ~]# ssh -V
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017

Client:

 ~ ssh -V
OpenSSH_9.0p1, LibreSSL 3.3.6

Workaround:

Pass the following option to ssh when using az ssh Command:

~ az ssh  vm -n rac1 -g marirac -- -o PubkeyAcceptedKeyTypes=+ssh-rsa-cert-v01@openssh.com

Azure CLI install ssh extension

Problem:

To enable AD authentication on a Linux OS Azure VM, you must install Azure CLI and have the SSH extension. However, the SSH extension is not installed automatically after installing Azure CLI.

I can guide you through the simple steps to add this extension.

Solution:

Ensure that the extension is not present:

~ az version
{
  "azure-cli": "2.49.0",
  "azure-cli-core": "2.49.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {}
}

List available extensions:

~ az extension list-available --output table

Name    Version    Summary  Preview  Experimental  Installed
------- --------   -------  -------  ------------  --------------------------
...
ssh     1.1.6      SSH...   False    False         False
...

Add extension:

~ az extension add --name ssh

Ensure that the extension has been added:

~ az version
{
  "azure-cli": "2.49.0",
  "azure-cli-core": "2.49.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "ssh": "1.1.6"
  }
}