Remote SSH to a VPS (Two Ways): Password & SSH Keys

Last updated on 26 Sep 2025

A concise, copy‑paste friendly guide to connect to your VPS over SSH—first with a password, then the recommended SSH key method—plus hardening tips and common troubleshooting steps. Works on Ubuntu/Debian/RHEL/AlmaLinux/Rocky and macOS/Linux/Windows clients.

Password login: quick to get in once—then turn it off.
SSH keys: generate on your laptop, install the public key on the server, and disable passwords.
Harden with: a non‑root sudo user, PermitRootLogin no, PasswordAuthentication no, fail2ban, and a firewall.

Prerequisites

A VPS with a public IP (or a reachable hostname).
You can access the VPS provider’s console (rescue/KVM) in case you lock yourself out.
Your server runs an OpenSSH server (most images do by default).

Default SSH port is 22. If your provider blocks it, use the port they give you.

1) Prepare the server (one-time)

# Ensure OpenSSH is installed and running
# Debian/Ubuntu
sudo apt update && sudo apt install -y openssh-server
sudo systemctl enable --now ssh

# RHEL/AlmaLinux/Rocky
sudo dnf install -y openssh-server
sudo systemctl enable --now sshd

# (Optional) See which port SSH is listening on
sudo ss -ltnp | grep -E ':(22|PORT_YOU_SET)'

Add a non‑root sudo user (safer)

# Replace USER with your name
adduser USER
usermod -aG sudo USER   # Debian/Ubuntu
# or on RHEL-family
usermod -aG wheel USER

Open the firewall (if enabled)

# UFW (Debian/Ubuntu)
sudo ufw allow 22/tcp
sudo ufw enable     # if not enabled yet
sudo ufw status

# firewalld (RHEL-family)
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload

Note: If you plan to change the SSH port later (e.g., 2222), open that port before changing the config.

2) Method A — Connect with a password (quick start)

Client → Server

macOS/Linux:

ssh USER@SERVER_IP

Windows (PowerShell):

ssh USER@SERVER_IP

Enter the password when prompted.

Use this once to get in and set up keys. Password logins are brute‑force targets.

3) Method B — Connect with SSH keys (recommended)

3.1 Generate a key pair on your laptop

macOS/Linux:

ssh-keygen -t ed25519 -C "your_email@example.com"
# Press Enter to accept ~/.ssh/id_ed25519
# Set a passphrase when asked (recommended)

Windows (PowerShell, built‑in OpenSSH):

ssh-keygen -t ed25519 -C "your_email@example.com"

This creates:

Private key: ~/.ssh/id_ed25519
Public key: ~/.ssh/id_ed25519.pub

3.2 Install your public key on the server

Option 1 — ssh-copy-id (macOS/Linux):

ssh-copy-id -i ~/.ssh/id_ed25519.pub USER@SERVER_IP

Option 2 — Manual (all platforms):

On the server (as the target USER):

mkdir -p ~/.ssh && chmod 700 ~/.ssh
echo "<paste your public key line here>" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Show your public key:

cat ~/.ssh/id_ed25519.pub

ssh -i ~/.ssh/id_ed25519 USER@SERVER_IP

If it works, you can disable password logins.

4) Harden your SSH configuration

Edit the SSH daemon config and restart.

Debian/Ubuntu: /etc/ssh/sshd_config
RHEL/Alma/Rocky: /etc/ssh/sshd_config (same path)

Recommended settings (add or update these):

Port 22                    # change only if you know the implications
Protocol 2
PermitRootLogin no
PasswordAuthentication no  # turn OFF after key login works
PubkeyAuthentication yes
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yes
# Allow only specific users (optional)
# AllowUsers USER anotheruser
# Limit auth attempts
MaxAuthTries 3
LoginGraceTime 20
# Modern host keys (default on new systems)
HostKey /etc/ssh/ssh_host_ed25519_key

Restart SSH:

# Debian/Ubuntu
sudo systemctl restart ssh
# RHEL-family
sudo systemctl restart sshd

Tip: Keep your existing SSH session open when restarting, so you don’t lock yourself out.

Optional extras

Change port (e.g., to 2222) and update firewall: ufw allow 2222/tcp or firewall-cmd --add-port=2222/tcp --permanent && firewall-cmd --reload

Fail2ban to block brute force:

sudo apt install -y fail2ban   # Debian/Ubuntu
# RHEL-family: sudo dnf install -y fail2ban
sudo systemctl enable --now fail2ban

5) Quality‑of‑life: `~/.ssh/config` (client side)

On your laptop create/edit ~/.ssh/config:

Host my-vps
    HostName SERVER_IP_OR_HOSTNAME
    User USER
    IdentityFile ~/.ssh/id_ed25519
    Port 22
    ServerAliveInterval 60
    ServerAliveCountMax 3

Then connect with:

ssh my-vps

6) Common errors & fixes

❌ Permission denied (publickey)

Wrong user? Try ssh -v USER@SERVER_IP to see details.
authorized_keys permissions must be: ~/.ssh 700, authorized_keys 600.
On SELinux systems: restorecon -R -v ~/.ssh if contexts are wrong.

❌ Connection refused / timed out

SSH service running? systemctl status ssh|sshd
Firewall open? ufw status or firewall-cmd --list-all
Provider blocks port 22? Try a different port (and open it).

❌ Host key verification failed (server reinstalled / IP reassigned)

You changed servers behind the same IP. Remove the old key:

ssh-keygen -R SERVER_IP
# or edit ~/.ssh/known_hosts and delete the matching line

❌ Locked out after config change

Use your provider’s console/KVM to revert /etc/ssh/sshd_config.
Re‑enable PasswordAuthentication yes temporarily, fix keys, then disable again.

7) Bonus: Windows PuTTY (if you prefer GUI)

Generate keys with PuTTYgen → save private key (.ppk) and copy the public key to server’s authorized_keys.
Connect using PuTTY → Host Name = SERVER_IP, specify the .ppk under Connection → SSH → Auth → Credentials.

Security checklist (copy/paste)

Non‑root sudo user created
SSH keys working for that user
PermitRootLogin no
PasswordAuthentication no
Firewall allows only necessary ports
Fail2ban enabled (optional but recommended)
Backups & provider console access tested

Reusable snippets

Create user and add SSH key quickly:

USER=john
sudo adduser "$USER"
sudo usermod -aG sudo "$USER"  # or wheel on RHEL
sudo -u "$USER" bash -c 'mkdir -p ~/.ssh && chmod 700 ~/.ssh'
cat <<'PUB' | sudo tee -a /home/$USER/.ssh/authorized_keys >/dev/null
<YOUR_PUBLIC_KEY_LINE_HERE>
PUB
sudo chown -R "$USER":"$USER" /home/$USER/.ssh
sudo chmod 600 /home/$USER/.ssh/authorized_keys

Safe hardening (after key login verified):

sudo cp /etc/ssh/sshd_config{,.bak}
sudo sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo systemctl restart ssh || sudo systemctl restart sshd

That’s it!

You now have both methods to reach your VPS and a hardened setup for everyday use. Copy this into your blog and tweak the names/ports to match your environment.