Setup & Harden a Linux Server in 2026

If you spin up a fresh Linux box today, it's not "new" for long. The moment it's reachable, it's being scanned, prodded, and occasionally hammered by automated traffic. That's not paranoia, it's just the background noise of the modern internet. Honeypot data has been consistent for years: a brand-new instance with a public IP will typically see its first SSH probe within minutes, and credential-stuffing attempts shortly after.

Cloud providers have made it trivially easy to create compute instances. What they haven't done is make them safe by default. You still get a general-purpose system with a wide attack surface, and it's on you to reduce that surface before you put anything meaningful on it.

This guide is a pragmatic baseline. Not a full security model, not compliance theatre, just the first set of moves you should make on any new Linux host:

get it updated
stop logging in as root
lock down SSH properly (or remove it from the public internet altogether)
put a firewall in place - ideally two
add some basic intrusion protection
remove anything you don't explicitly need

There's nothing especially novel here, and that's the point. Most compromises aren't zero-days, they're neglected basics. If you do these steps consistently, you eliminate a large class of avoidable problems.

Assumptions are minimal: you've got shell access, you're comfortable editing config files, and you want something you can repeat across machines without ceremony.

Everything below works on a typical Debian or Fedora family system, with notes where they differ. Adjust to taste, but don't skip the intent behind each step.

The ideal: minimize your public attack surface

Before any of the steps below, ask the obvious question: does this box need to expose anything to the public internet at all?

The ideal posture for a modern server is brutally simple:

Only 443/tcp open to the world (and 80/tcp if you're terminating ACME challenges or doing HTTP->HTTPS redirects).
Everything else - SSH, databases, admin panels, metrics endpoints - reachable only over a private overlay network.

If you can get away with that, do. A port that isn't listening on the public internet can't be brute-forced, can't leak a banner, and can't get caught in someone else's exploit chain. The cheapest packet to drop is the one your firewall never sees.

Use Tailscale (or similar) for SSH and admin access

Tailscale is a WireGuard-based mesh VPN that, in practice, removes the need to expose SSH at all. You install the client on your laptop and the server, both authenticate against your identity provider, and they get a stable private IP on a network only your devices can reach. No port forwarding, no bastion host, no faffing with WireGuard configs by hand.

The pattern I use:

Install Tailscale on the new box: curl -fsSL https://tailscale.com/install.sh | sh
Bring it up: sudo tailscale up --ssh (the --ssh flag is optional but lets Tailscale itself broker SSH auth, which is nice).
Bind sshd to the Tailscale interface only, or just firewall 22 off the public internet entirely.
Public firewall: 443 open, 22 closed.

If Tailscale doesn't fit your environment, the same shape works with WireGuard directly, Headscale (FOSS Tailscale control plane), ZeroTier, or a traditional bastion host. The principle is the same: SSH is an admin-plane protocol, treat it like one.

If you genuinely need SSH on the public internet - fine, the rest of this guide hardens that path. Just make it a deliberate choice, not a default.

Defence in depth: use both firewalls

Cloud providers (AWS, GCP, Hetzner, Linode, DigitalOcean, etc.) all give you a network-level firewall - security groups, cloud firewalls, network ACLs, whatever the marketing calls them. Use it. Then also configure the host firewall (ufw, firewalld, nftables) on the box itself.

People sometimes ask why bother with both. A few reasons:

The cloud firewall protects you if something on the host is misconfigured (e.g. a service binds 0.0.0.0 when you thought it was on 127.0.0.1).
The host firewall protects you if the cloud firewall is misconfigured, or if a peer instance in the same VPC is compromised and starts scanning sideways.
They fail differently. A console mistake that opens 0.0.0.0/0 on the cloud side still hits a closed host firewall. A package upgrade that resets ufw rules still hits a closed cloud firewall.

It's the same logic as having both a building door and a flat door. Neither is sufficient, both is cheap.

SSH in to the box

ssh root@192.0.2.10

You'll usually be root on first boot with cloud images. That's fine for the next ten minutes; we're going to fix it shortly.

System updates

First thing, always. The base image is almost certainly behind on patches by the time you've booted it.

Debian based systems

apt update && apt upgrade -y

Fedora

sudo dnf upgrade --refresh -y

If a kernel update lands, reboot before continuing. Half-patched kernels are a category of weird bug you don't want to debug at 2am.

A decent editor

Optional, but life is short.

## fedora ##
dnf install -y neovim
# or
dnf install -y helix

## debian based ##
sudo apt install -y neovim
# or
add-apt-repository ppa:maveonair/helix-editor
apt update
apt install -y helix

Set the hostname and hosts file

A correctly set hostname matters more than people think. Mail (even outbound transactional mail), TLS certificate provisioning, log aggregation, and metrics tagging all key off it. Get it right once.

Set the hostname

Edit /etc/hostname directly, or:

hostnamectl set-hostname my-host-name

Edit `/etc/hosts`

With the public IP and the FQDN (fully qualified domain name).

For IPv4:

127.0.0.1       localhost.localdomain localhost
203.0.113.10    example-hostname.example.com example-hostname

And IPv6:

127.0.0.1                       localhost.localdomain localhost
203.0.113.10                    example-hostname.example.com example-hostname
2001:db8::a123:b456:c789:d012   example-hostname.example.com example-hostname

Add a limited user account

Working as root is fine for the install, dangerous as a habit. A typo in a long-running shell shouldn't be able to wipe the disk.

useradd -m -s /bin/bash example_user
passwd example_user
usermod --append --groups wheel,sudo example_user

wheel is the relevant group on Fedora; sudo on Debian/Ubuntu. Adding to both is harmless on either family.

Sudo without constant password prompts

Convenience, not security - make a deliberate choice. If this is a shared box or a production system with multiple operators, leave the password requirement on.

export VISUAL=nvim
# or export VISUAL=hx
visudo

Use visudo rather than editing /etc/sudoers directly. It validates syntax before saving, which matters because a broken sudoers file can lock you out of root entirely. If that happens on a cloud box with no console, you're rebuilding.

Add:

%wheel ALL = (ALL) NOPASSWD:ALL

Log out, log back in as the new user

exit

Then:

ssh example_user@example-hostname.example.com

Harden SSH access

This is the part that, in practice, stops the overwhelming majority of opportunistic attacks against a public-facing host.

Grant access to the new limited user

If you don't already have an SSH key, generate one. ed25519 is the right default in 2026 - smaller, faster, and not weakened by anything currently public.

On your local machine:

ssh-keygen -t ed25519 -C "user@domain.tld"

Then on the compute instance:

mkdir -p /home/example_user/.ssh

Upload your public key. From your local system:

Linux:

ssh-copy-id example_user@192.0.2.17

macOS: Install ssh-copy-id from Homebrew (brew install ssh-copy-id) and use it as above, or do it manually:

scp ~/.ssh/id_ed25519.pub example_user@203.0.113.10:/home/example_user/.ssh/authorized_keys

Windows (PowerShell):

scp C:\Users\MyUserName\.ssh\id_ed25519.pub example_user@192.0.2.17:~/.ssh/authorized_keys

Fix permissions - sshd will refuse to use a key file that's group- or world-readable:

chmod 700 /home/example_user/.ssh
chmod 600 /home/example_user/.ssh/authorized_keys

Disallow root login via SSH

Edit /etc/ssh/sshd_config:

# Authentication:
PermitRootLogin no

Even if you keep root password access locally, you do not want it reachable over the network. Every SSH brute-force bot tries root first.

Disable password authentication

Same /etc/ssh/sshd_config file:

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no
KbdInteractiveAuthentication no

Both lines matter - KbdInteractiveAuthentication can otherwise sneak password prompts back in via PAM on some distros.

Before you restart sshd, open a second terminal and confirm you can log in with your key. If something is wrong with authorized_keys permissions or the key itself, you want to find out while you still have a working session to fix it from.

Restrict to specific users (optional but useful)

Also in sshd_config:

AllowUsers example_user

Defence against the day someone creates a service account with a weak shell login they forget about.

Restart the SSH daemon

systemctl restart ssh
# could be sshd on older systems
# on non-systemd, use: service sshd restart

fail2ban

Bans IP addresses that make repeated failed authentication attempts. With key-only SSH this is mostly belt-and-braces, but it also keeps your auth.log readable by clamping the volume of brute-force noise, and it covers other services (web admin endpoints, mail) if you add jails for them.

Install

# debian/ubuntu
sudo apt install -y fail2ban

# fedora
sudo dnf install -y fail2ban

Configure

.local files override the default .conf files. Don't edit the .conf files directly - they get overwritten on package upgrade.

cd /etc/fail2ban
sudo cp fail2ban.conf fail2ban.local
sudo cp jail.conf jail.local

In jail.local, on CentOS or Fedora, change backend from auto to systemd:

backend = systemd

A reasonable default jail block at the top of jail.local:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
ignoreip = 127.0.0.1/8 ::1

[sshd]
enabled = true

ignoreip should also include your Tailscale CIDR (100.64.0.0/10) if you're using it, so an admin fat-fingering their passphrase a few times doesn't lock themselves out.

Enable

sudo systemctl enable --now fail2ban

Firewall

The host-level firewall. Pair this with the cloud provider's firewall - see the defence-in-depth note at the top.

sudo apt install -y ufw

Default policy

Set the default to deny inbound, allow outbound. Anything you want must then be explicitly allowed.

sudo ufw default deny incoming
sudo ufw default allow outgoing

Common rules

# Only open SSH publicly if you're not using Tailscale or similar
sudo ufw allow 22/tcp

# Web server
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

In the Tailscale model, you skip the SSH rule entirely on the public side and instead allow it on the Tailscale interface only:

sudo ufw allow in on tailscale0 to any port 22 proto tcp

Allow from a specific source

By IP:

sudo ufw allow from 198.51.100.7

By subnet:

sudo ufw allow from 198.51.100.0/24

By IP and port:

sudo ufw allow from 198.51.100.7 to any port 22 proto tcp

proto tcp can be removed or switched to proto udp depending on your needs, and allow can be swapped for deny as required.

Advanced rules

For anything beyond the above, edit the rules files directly. UFW processes /etc/ufw/before.rules before any of your CLI-added rules (it handles loopback, ping, and DHCP), and /etc/ufw/after.rules afterwards. IPv6 equivalents are before6.rules and after6.rules. Default behaviour and IPv6 toggling live in /etc/default/ufw.

Enable

sudo ufw enable

You'll be warned that this could disrupt existing SSH sessions. If you've correctly added an SSH allow rule (or you're SSH'ing over Tailscale), it won't.

Mirror this in the cloud firewall

Whatever you've allowed here, allow in the cloud provider's firewall too - and only that. If your host firewall allows 80, 443, and 22-on-tailscale, your cloud security group should allow 80 and 443 from 0.0.0.0/0 and nothing else publicly. SSH stays on the private network.

Intrusion detection

For a baseline IDS, OSSEC (or its more actively-maintained fork Wazuh) gives you file integrity monitoring, log analysis, and rootkit detection. The Linode docs have a reasonable getting-started guide, though check for a current version for your distro.

For most single-server setups, auditd plus fail2ban plus shipping logs somewhere off-box is enough. The point of an IDS is being able to notice an intrusion after the perimeter has been crossed - which means none of it matters if you're not actually reading the alerts.

Remove unused network-facing services

Anything listening that you don't need is just attack surface waiting for a CVE.

See what's actually listening

sudo ss -atpu

Where:

-a: all listening and non-listening
-t: TCP sockets
-p: show processes
-u: UDP sockets

For a cleaner view of just what's bound and waiting for connections:

sudo ss -tlnp

Anything you don't recognize, look up. Anything you don't need, mask the unit:

sudo systemctl disable --now <service>
sudo systemctl mask <service>

mask is stronger than disable - it prevents the service being started even as a dependency of something else.

Where to go from here

This gets you a sane baseline. Logical next steps, in roughly the order I'd take them:

Unattended security upgrades - unattended-upgrades on Debian/Ubuntu, dnf-automatic on Fedora. Auto-apply at least the security channel.
Off-host log shipping - journald remote, vector, loki, or whatever you're using. A log only on the compromised box is worth nothing post-incident.
Backups with restore tested - restic or borgbackup to S3-compatible or B2 storage. Untested backups are a story you tell yourself.
Time sync - chrony or systemd-timesyncd. Skewed clocks break TLS, logs, and auth in ways that are tedious to debug.
Configuration as code - once you've done this twice, write it down as Ansible, NixOS, or even a shell script. The next box should be five minutes, not an afternoon.

None of this is exotic. It's just the version of the basics that survives contact with a real internet.

Keep in touch →

The ideal: minimize your public attack surface

Use Tailscale (or similar) for SSH and admin access

Defence in depth: use both firewalls

SSH in to the box

System updates

Debian based systems

Fedora

A decent editor

Set the hostname and hosts file

Set the hostname

Edit /etc/hosts

Add a limited user account

Sudo without constant password prompts

Log out, log back in as the new user

Harden SSH access

Grant access to the new limited user

Disallow root login via SSH

Disable password authentication

Restrict to specific users (optional but useful)

Restart the SSH daemon

fail2ban

Install

Configure

Enable

Firewall

Default policy

Common rules

Allow from a specific source

Advanced rules

Enable

Mirror this in the cloud firewall

Intrusion detection

Remove unused network-facing services

See what's actually listening

Where to go from here

Edit `/etc/hosts`