VPS Firewall and SSH Hardening: A Practical Checklist

Windows Server course at Orbit Training Center, Dubai

A brand-new VPS, straight out of the box from any hosting provider, is not secure by default. It’s reachable, it’s functional, and it is also – within minutes of going live, according to widely reported patterns in server security guidance – already being scanned by automated bots probing for weak SSH credentials and open ports. The gap between “the server works” and “the server is reasonably secure” is a short, well-defined list of steps. Most of them take minutes. Almost all of the serious damage from compromised VPS instances traces back to one of a small handful of skipped basics, not some exotic attack.

Key Takeaways

  • Order matters: create a non-root user and confirm SSH key login works before you touch the firewall or disable password authentication – doing it in the wrong order is the single most common way people lock themselves out of their own server.
  • Key-based SSH authentication, with password authentication disabled only after key login is confirmed working, is the single highest-value hardening step.
  • A default-deny firewall that only opens the ports you actually need (typically SSH, HTTP, and HTTPS) closes off the vast majority of casual scanning attempts immediately.
  • Fail2ban automatically blocks brute-force login attempts and is simple to set up once SSH and the firewall are in place.
  • Automatic security updates mean known vulnerabilities get patched even if you don’t log in to the server every day.

The One Rule That Prevents Locking Yourself Out

Before any specific step: do these in the right order. Server-hardening guides consistently warn about the same mistake – enabling a strict firewall before confirming SSH access still works through it, or disabling password login before confirming key-based login actually works, are the two most common ways people accidentally lock themselves out of a server they’ve just started hardening. The safe sequence is: set up a non-root user and SSH keys first, confirm key login works, harden SSH configuration next, and only then lock down the firewall – checking at each step that you still have a working way in before removing the previous method of access.

Step 1: Create a Non-Root User With Sudo Access

Logging in and operating as root for everyday work is a long-standing bad practice: any mistake, compromised session, or malicious script runs with full, unrestricted system privileges. The standard first step on a new VPS is creating a separate user account with sudo privileges (the ability to run specific commands as root when explicitly needed, rather than operating as root by default), and doing all routine work from that account instead.

Step 2: Set Up SSH Key-Based Authentication

SSH keys are a cryptographic key pair – a private key that stays on your own computer and a public key placed on the server – that let you log in without typing a password at all, and that are dramatically harder to brute-force than any password. Generate a key pair, copy the public key onto the server for your new non-root user, and test logging in with the key before changing anything else. This is consistently identified across hardening guides as the single most effective SSH security improvement available.

Step 3: Disable Root Login and Password Authentication – But Only in the Right Order

Once key-based login is confirmed working for your non-root user, two SSH configuration changes meaningfully reduce attack surface: disabling direct root login over SSH entirely (forcing anyone – including you – to log in as a regular user and use sudo for elevated tasks), and disabling password authentication entirely, so only key-based login is accepted. The critical caveat repeated across hardening guidance: do not disable password authentication until you’ve actually confirmed key-based login works. Making this change before confirming keys work is one of the most common ways people get locked out of a fresh VPS with no working way back in.

Why Key-Based Authentication Is So Much Stronger Than a Password

It’s worth understanding why this one step carries so much weight. A password, no matter how long, is a single piece of information that can theoretically be guessed, intercepted, or brute-forced given enough attempts – and automated bots make an enormous number of attempts continuously against any server with SSH exposed to the internet. An SSH key pair works differently: the private key never leaves your own computer and is never transmitted anywhere during login, so there’s no password being sent that could be intercepted or guessed, and the mathematical problem an attacker would need to solve to forge a valid key without it is computationally infeasible with current technology. This is also why protecting your own private key file (not sharing it, optionally protecting it with its own passphrase) matters – it’s the one thing standing between anyone who obtains a copy of it and full access to every server it’s registered on.

Step 4: Configure a Default-Deny Firewall

A firewall configured to deny all traffic by default, with only explicitly necessary ports opened, closes off the vast majority of automated scanning and opportunistic attacks immediately. For a typical web-facing VPS, that usually means opening only the SSH port, and the standard web ports (80 for HTTP, 443 for HTTPS) – and nothing else, unless a specific application genuinely needs another port open. Tools like UFW (Uncomplicated Firewall) on Ubuntu/Debian systems make this straightforward to configure without needing to write raw iptables rules directly.

Important sequencing note: enable the firewall rule allowing SSH traffic before you turn the firewall on – if you enable a default-deny firewall without first explicitly permitting the SSH port, you’ll cut off your own remote access immediately.

Step 5: Set Up Fail2ban for Brute-Force Protection

Fail2ban monitors login attempts and automatically, temporarily bans IP addresses that show patterns of repeated failed login attempts – most commonly configured to watch SSH login attempts specifically. This catches the automated brute-force bots that firewalls alone don’t stop (since the SSH port still has to stay open for legitimate access), by recognizing the attack pattern and blocking the source rather than the port.

Step 6: Consider Rate-Limiting SSH at the Firewall Level Too

Beyond fail2ban, UFW and similar tools support rate-limiting specific ports directly (commonly referenced as `ufw limit` for SSH), which adds a second layer that throttles repeated connection attempts to the SSH port itself, independent of fail2ban’s log-watching approach. The two aren’t redundant – they catch slightly different attack patterns – and using both is common in more thorough hardening checklists.

Step 7: Enable Automatic Security Updates

Known vulnerabilities in server software get patched regularly, but only if updates are actually applied. Automatic security updates (on Ubuntu/Debian systems, commonly configured via the unattended-upgrades package) apply critical security patches without requiring you to manually log in and run updates on a schedule you might forget. This doesn’t replace checking on your server periodically, but it closes the gap of a known, published vulnerability sitting unpatched for weeks because nobody remembered to update.

A Note on SSH Configuration File Changes

Most of the SSH-related steps above involve editing the SSH daemon’s configuration file directly (commonly `/etc/ssh/sshd_config` on Linux systems) rather than a graphical settings panel – there typically isn’t one on a bare VPS. Each setting (disabling root login, disabling password authentication, changing the listening port) is a specific line in that file, and the SSH service needs to be restarted for a change to take effect. The practical implication for the verification habit described above: after editing this file and restarting the SSH service, that is exactly the moment to open a second terminal and test before closing your current session, since a typo or mistaken setting in this one file is specifically what causes most SSH-related lockouts.

Step 8: Consider Changing the Default SSH Port (Debated, Minor Value)

Moving SSH off its default port 22 to a non-standard port is a commonly suggested step, though its actual security value is debated among practitioners – it reduces noise from the most basic automated scans that only check port 22, but doesn’t meaningfully stop a more targeted attacker who simply scans all ports. Treat this as a minor, optional layer, not a substitute for the steps above.

StepWhat it protects againstPriority
Non-root user with sudoFull-system compromise from any single mistake or exploitEssential – do first
SSH key authenticationPassword brute-forcing and credential guessingEssential – highest-value single step
Disable root login + password authDirect root compromise and password-based attacksEssential – only after key login confirmed
Default-deny firewall (UFW)Opportunistic scanning of unused ports/servicesEssential
Fail2banAutomated brute-force login attemptsHigh – set up right after firewall
Automatic security updatesKnown, published vulnerabilities going unpatchedHigh – ongoing protection
Non-standard SSH portBasic automated scans targeting port 22 specificallyOptional, minor value

A Note on SFTP and File Transfer Security

Hardening SSH access also hardens file transfer, if you’re transferring files the right way. SFTP (SSH File Transfer Protocol) runs over the same encrypted SSH connection and benefits from every hardening step above automatically – once SSH is locked down with key-based auth and the other steps on this checklist, SFTP access inherits that same protection. The mistake to avoid is falling back to plain, unencrypted FTP for convenience, which transmits credentials and file contents in plain text and undoes much of the security benefit gained from hardening SSH itself. If a workflow or tool still defaults to plain FTP, switching it to SFTP is a small, high-value addition to this checklist.

Beyond the Basics

Once the core checklist above is in place, more advanced hardening steps commonly referenced in thorough security guidance include intrusion detection tooling, kernel-level security parameter tuning, centralized log monitoring (so suspicious patterns are visible rather than buried in local logs nobody checks), and – critically – encrypted, tested backups stored off the server itself. None of these replace the basics; they’re the next layer once the fundamentals are solid.

Verifying Each Step Actually Worked Before Moving to the Next

The single most important discipline in this entire checklist, repeated across virtually every serious hardening guide, is this: after any change that could affect your ability to log in, open a brand-new second terminal window or connection and test the new access method, while keeping your original, currently-working session open. Only close the original session once the new one is confirmed working. This one habit – never closing your only working connection until a new one is verified – prevents the overwhelming majority of self-inflicted lockouts, regardless of which specific step you’re on.

Concretely: after setting up SSH keys, open a second terminal and log in with the key before closing your original password-based session. After changing firewall rules, open a second connection and confirm you can still reach the server before trusting that the change is safe. This costs almost nothing in time and eliminates the single most common way people get hardening wrong.

Common Mistakes Beyond the Basic Lockout Risk

A few other patterns show up repeatedly in real-world hardening mistakes, beyond the core sequencing issue:

  • Reusing the same SSH key pair across many unrelated servers. If one server or one copy of your private key is ever compromised, every server using that same key pair is now at risk simultaneously. Using distinct keys per server, or at minimum per client/project, limits the damage from any single key being exposed.
  • Setting up fail2ban but never checking its logs. Fail2ban working silently in the background is good, but periodically reviewing what it’s actually been blocking gives you real visibility into how much active scanning and attack traffic your server receives – useful both for peace of mind and for spotting unusual patterns worth investigating further.
  • Treating hardening as a one-time setup task. New vulnerabilities get discovered in server software on an ongoing basis. A server hardened correctly on day one and never touched again gradually becomes less secure over time as new issues emerge in unpatched software – which is exactly why automatic security updates matter as an ongoing practice, not a box to check once.
  • Opening a port “just to test something” and forgetting to close it. Temporary firewall exceptions opened during debugging are easy to forget about once the immediate problem is solved. Treating every firewall change as something to explicitly review and clean up afterward avoids a server slowly accumulating unnecessary open ports over its lifetime.

A Short Worked Walkthrough

To make the sequence concrete: on a fresh Ubuntu VPS, a typical first session looks like logging in as root (the only option available initially), creating a new user and granting it sudo access, generating an SSH key pair on your own computer, copying the public key to the new user’s account, and opening a second terminal to confirm you can log in as the new user with the key – all before changing a single firewall or SSH configuration setting. Only once that’s confirmed working do you move to disabling root SSH login and password authentication, then to firewall configuration (allowing SSH explicitly before enabling default-deny), then to fail2ban and automatic updates. Each stage builds on the previous one being verified, not assumed.

Why Order and Discipline Matter More Than Any Single Step

No individual item on this checklist is exotic or hard to understand. What actually separates a properly hardened VPS from a vulnerable one is doing these steps in the right order, verifying each one works before removing the previous safety net, and treating hardening as a baseline you set up once and then maintain – not a one-time task you do and forget. A structured VPS Server Administration course in Dubai walks through this exact sequence hands-on, including the lockout mistakes to avoid, which is a meaningfully safer way to learn this than discovering the hard way, on a live client server, what happens when the order gets reversed.

Frequently Asked Questions

What’s the very first thing I should do on a brand-new VPS?

Create a non-root user with sudo privileges and set up SSH key-based authentication for that user, confirming key login actually works, before touching the firewall or disabling any existing access method.

Is it safe to disable SSH password login immediately?

Only after you’ve confirmed SSH key-based login works successfully. Disabling password authentication before confirming keys work is one of the most common ways people lock themselves out of a new server entirely.

Do I need both fail2ban and firewall rate-limiting, or is one enough?

They’re not redundant – fail2ban watches login attempt patterns in logs and bans offending IPs, while firewall-level rate-limiting throttles connection attempts directly at the network level. Thorough hardening checklists commonly use both together.

Which ports should I actually open on my firewall for a typical website?

For a standard web-facing server, typically just SSH (for your own access) and the two standard web ports, 80 (HTTP) and 443 (HTTPS). Open additional ports only if a specific, identified application genuinely requires one.

Is changing the default SSH port worth doing?

It has some minor value against the most basic automated scans, but it’s not a real substitute for key-based authentication, a proper firewall, and fail2ban. Treat it as an optional extra layer, not a core security measure.

Table of Contents