Unable to ssh to slave node

Hi JonEtkins,

Thank you for reaching out to Sophos Community.

If 'Block Password Guessing' is enabled under Definitions & Users > Authentication Services > Advanced, then the slave node might have temporarily blocked access from the master node's IP address due to failed login attempts.

Could you please try again and share your observation?

Hi Yash, and thanks for responding.  Yes, I have blocking enabled, but I also have my workstation added to the "Never block networks:, so I doubt that is the problem.  I get the same result regardless of which node is currently active, by the way:

<M> astaro:/root # ha_utils status
- Status -----------------------------------------------------------------------
Current mode: HA MASTER with id 2 in state ACTIVE
-- Nodes -----------------------------------------------------------------------
MASTER: 2 esxi55 198.19.250.2 9.705003 ACTIVE since Tue May 11 00:51:50 2021
SLAVE: 1 esxi3 198.19.250.1 9.705003 ACTIVE since Tue May 11 00:56:50 2021
-- Load ------------------------------------------------------------------------
Node  2: [1m] 0.15  [5m] 0.07  [15m] 0.06
Node  1: [1m] 0.00  [5m] 0.02  [15m] 0.05

- Kernel -----------------------------------------------------------------------
Current mode: enabled master
interface: eth3
Local ID: 198.19.250.2
debug: off
verbose: off
ppp sync: off
port smtp: 25
port pop3: 8110
port ftp: 2121

- IPSec ------------------------------------------------------------------------
000 HA System active on eth3/224.0.0.82. Current mode is Master. Seqdiff in: 256 Seqdiff out: 4096

- PostgreSQL ------------------------------------------------------------------------
 primary | standby |     lag      |  bytelag
---------+---------+--------------+----------
       2 |       1 | 00:00:00.993 |        0
<M> astaro:/root # ha_utils ssh

Connecting to slave 198.19.250.1
loginuser@198.19.250.1's password:
Permission denied, please try again.
loginuser@198.19.250.1's password:

Further information:  this evening, I ssh'd to the UTM and connected to node 2 which was the active node at the time.  As root, I issued a passwd loginuser command to reset the password once more on that node.

I then triggered a takeover via the web UI, and to my surprise, found that my SSH session remained active and connected to node 2, which was now the slave.  I fired up another ssh session to the UTM, which connected to node 1 which was now master, and tried hautils ssh once more, specifying the password I had just set on node 2.

It still failed.

So I reset the password on node2 again, in case it had somehow gotten overwritten when the roles switched, but no, I still get Permission denied when attempting to   ha_utils ssh from the active master node.

It sure seems to me like something ain't working as designed!

<M> astaro:/root # ha_utils status
- Status -----------------------------------------------------------------------
Current mode: HA MASTER with id 2 in state ACTIVE
-- Nodes -----------------------------------------------------------------------
MASTER: 2 esxi55 198.19.250.2 9.705003 ACTIVE since Tue May 11 00:51:50 2021
SLAVE: 1 esxi3 198.19.250.1 9.705003 ACTIVE since Tue May 11 00:56:50 2021
-- Load ------------------------------------------------------------------------
Node  2: [1m] 0.15  [5m] 0.07  [15m] 0.06
Node  1: [1m] 0.00  [5m] 0.02  [15m] 0.05

- Kernel -----------------------------------------------------------------------
Current mode: enabled master
interface: eth3
Local ID: 198.19.250.2
debug: off
verbose: off
ppp sync: off
port smtp: 25
port pop3: 8110
port ftp: 2121

- IPSec ------------------------------------------------------------------------
000 HA System active on eth3/224.0.0.82. Current mode is Master. Seqdiff in: 256 Seqdiff out: 4096

- PostgreSQL ------------------------------------------------------------------------
 primary | standby |     lag      |  bytelag
---------+---------+--------------+----------
       2 |       1 | 00:00:00.993 |        0

<M> astaro:/root # passwd loginuser
Changing password for loginuser.
New Password:
Reenter New Password:
Password changed.

<M> astaro:/root #
<S> astaro:/root # ha_utils status
- Status -----------------------------------------------------------------------
Current mode: HA SLAVE with id 2 in state ACTIVE
-- Nodes --------------------------------------------------- eth0 alive --------
SLAVE: 2 esxi55 198.19.250.2 9.705003 ACTIVE since Wed May 12 02:04:36 2021
MASTER: 1 esxi3 198.19.250.1 9.705003 ACTIVE since Wed May 12 01:59:36 2021
-- Load ------------------------------------------------------------------------
Node  2: [1m] 0.00  [5m] 0.01  [15m] 0.08
Node  1: [1m] 0.01  [5m] 0.04  [15m] 0.09

- Kernel -----------------------------------------------------------------------
Current mode: enabled slave
interface: eth3
Local ID: 198.19.250.2
Master ID: 1
debug: off
verbose: off
ppp sync: off
port smtp: 25
port pop3: 8110
port ftp: 2121

- IPSec ------------------------------------------------------------------------
000 HA System active on eth3/224.0.0.82. Current mode is Slave. Seqdiff in: 256 Seqdiff out: 4096

- PostgreSQL ------------------------------------------------------------------------
 primary | standby |     lag      |  bytelag
---------+---------+--------------+----------
       1 |       2 | 00:00:00.587 |        0
<S> astaro:/root # passwd loginuser
Changing password for loginuser.
New Password:
Reenter New Password:
Password changed.
<S> astaro:/root #

<M> astaro:/root # ha_utils status
- Status -----------------------------------------------------------------------
Current mode: HA MASTER with id 1 in state ACTIVE
-- Nodes -----------------------------------------------------------------------
MASTER: 1 esxi3 198.19.250.1 9.705003 ACTIVE since Wed May 12 01:59:36 2021
SLAVE: 2 esxi55 198.19.250.2 9.705003 ACTIVE since Wed May 12 02:04:37 2021
-- Load ------------------------------------------------------------------------
Node  1: [1m] 0.05  [5m] 0.04  [15m] 0.05
Node  2: [1m] 0.06  [5m] 0.03  [15m] 0.05

- Kernel -----------------------------------------------------------------------
Current mode: enabled master
interface: eth3
Local ID: 198.19.250.1
debug: off
verbose: off
ppp sync: off
port smtp: 25
port pop3: 8110
port ftp: 2121

- IPSec ------------------------------------------------------------------------
000 HA System active on eth3/224.0.0.82. Current mode is Master. Seqdiff in: 256 Seqdiff out: 4096

- PostgreSQL ------------------------------------------------------------------------
 primary | standby |     lag      |  bytelag
---------+---------+--------------+----------
       1 |       2 | 00:00:00.299 |        0
<M> astaro:/root # ha_utils ssh

Connecting to slave 198.19.250.2
loginuser@198.19.250.2's password:
Permission denied, please try again.
loginuser@198.19.250.2's password:

<M> astaro:/root #

Hi JonEtkins,

Did you configure public key authentication under Management > System settings > Shell Access > Authentication? 

To change it to root user, are you using "su -"? 

What is the current system status under Management > High Availability > System Status? 

Thanks,

Hi JonEtkins .

One thing. I doubt that putting your workstation to the "Never block networks" list would help. You could put in the network of the HA connection, because with ha_utils your connection should normally occur using the HA port, as far as I know.

You could give this a try.

But be careful as the HA IP is a public IP you should not use this generally.

I have both password authentication and public key authentication enabled.  I have "Allow root login" set to "Root access but only with SSH key."  I have an authorized key specified for root, but none for loginuser.  When I connect to the active node, I log in directly as root, using that key, so there is no need to su.  HA status is as follows:

Oh, and in case it's relevant, I have the ssh port set to 2222.

Thanks,
  Jon

Hi JonEtkins,

Would it be possible for you to turn off the public key authentication for testing and try to ssh into the secondary unit and let us know if that works. 

Thanks,

Well we're getting somewhere, but I'm not sure where. :)  After disabling key authentication, I am of course unable to log in to the master as root, so I tried logging in as loginuser, and...Access denied.

So it would appear that the problem is not tied to the secondary node ssh feature, but is instead some sort of problem authenticating or authorizing the loginuser account for any ssh access.  I have double-checked that the password I have set is correct, and that it meets all the criteria I have set under Authentication Services - Advanced (length is 8 chars and it contains at least one character from each of the four groups).  The account is not locked, and is not denied in /etc/ssh/sshd_config.

Not sure where to go next.  I don't see any errors in /var/log/login.log or system.log, but I do get email notification of the failed attempt.  I could change the debug level in /etc/ssh/sshd_config, but I don't know how to restart the sshd daemon since I don't see an entry for it in /etc/init.d.

Hi, Harsh.  Not sure if you've seen my update below; is there some further tracing and/or logging I can enable to try to see what's blocking this login?

Hi JonEtkins,

Apologies for the delayed response. 

After removing the public key authentication, you should be able to ssh into the secondary unit, at least I was able to ssh to the secondary unit. 

If you want to restart the sshd service use the following command: 

<M> h_patel:/root # /var/mdw/scripts/sshd restart
:: Stopping SSH done
:: Starting SSH starting SSH daemon done
:: Restarting SSH
<M> h_patel:/root #

Authentication logs are stored in aua.log file. 

Thanks,