, ,

Basic Security Hardening for a New Linux Server (Linux for Beginners, Part 17)

A fresh server gets brute-forced within minutes. The five changes that stop almost all automated attacks: patch, key-only SSH, a default-deny firewall, fail2ban, and least privilege.

Linux for Beginners · Part 17 of 24

Stand up a fresh server with a public address, leave password login switched on, and you do not have to wait long. Within minutes the authentication log fills with strangers trying to log in as root, then admin, then oracle, then every common name a bot has ever seen. This is not a targeted attack. It is the background noise of the internet, automated and constant, and it finds a new server in the time it takes to make coffee.

Hardening is the work of closing the easy doors before those bots find one open. You do not need a security team or a long checklist. A handful of changes stops the overwhelming majority of automated attacks, and they take about fifteen minutes on a new box.

New to servers? Do each step on a throwaway cloud VM you can rebuild, and keep a second terminal open so a mistake does not lock you out. Running production already? Confirm your firewall lets SSH through before you touch anything, then work down the list. This leans on Part 13 on SSH and the users and sudo ideas from earlier.

The five that matter

Almost all of your protection comes from five moves, in this order:

1. Apply updates and turn on automatic security patches. 2. Switch SSH to key only and forbid root login. 3. Put a firewall in front that allows only the ports you use. 4. Add fail2ban to ban repeat offenders. 5. Work as a normal user with sudo, not as root. Do these and the automated internet mostly stops being your problem.

Updates Firewall SSH keys fail2ban least privilege host Each layer removes a class of attack before it reaches the server.

Patch first, because old holes are how most breaches happen

The unglamorous truth of security is that most successful attacks use a flaw that was fixed months ago, on a machine nobody updated. Patching is the highest value thing you can do, so do it first and then make it automatic. Apply everything waiting, then turn on unattended security updates so the box keeps itself current without you.

# Debian and Ubuntu
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install unattended-upgrades
$ sudo dpkg-reconfigure -plow unattended-upgrades

# RHEL family
$ sudo dnf upgrade -y
$ sudo dnf install dnf-automatic
$ sudo systemctl enable --now dnf-automatic.timer

That timer at the end ties straight back to the scheduling part of this series: automatic updates are just a systemd timer running the updater on a cadence. Set it once and the most common way servers get owned, a missing patch, stops applying to you.

Lock down SSH, the front door

SSH is how you get in, which means it is also what the bots hammer. Two changes gut their odds. First, log in with a key instead of a password, because a key cannot be guessed the way a password can. Second, forbid the root account from logging in over SSH at all, so even a correct root password is useless remotely. The settings live in /etc/ssh/sshd_config.

Password login guessable, brute forced all day Key pair login private key stays on your laptop bots try thousands of passwords a minute nothing to guess, the server holds only the public half
Setting in sshd_configSet it toWhy
PermitRootLoginnoRoot can never log in remotely
PasswordAuthenticationnoOnly keys are accepted
PubkeyAuthenticationyesEnable key based login
# on your laptop, if you have no key yet
$ ssh-keygen -t ed25519
$ ssh-copy-id you@server        # push your public key up

# on the server, edit the three settings, then
$ sudo systemctl restart ssh    # or sshd on RHEL

Do these in order. Copy your key up and confirm you can log in with it in a fresh terminal before you turn passwords off. Flip PasswordAuthentication to no while your only working login is a password you just disabled, and you have locked yourself out of your own machine.

A firewall that allows only what you use

A firewall decides which ports the outside world can reach. The safe posture is to deny everything by default and allow only the handful of services you actually run, usually SSH and a web port or two. Debian and Ubuntu ship the simple ufw; the RHEL family ships firewalld. Both do the same job with different words.

Goalufw (Debian, Ubuntu)firewalld (RHEL)
Allow SSHufw allow OpenSSHfirewall-cmd --add-service=ssh --permanent
Allow webufw allow 80,443/tcpfirewall-cmd --add-service=http --permanent
Turn it onufw enablefirewall-cmd --reload
How people lock themselves out

The classic firewall accident: you run ufw enable before allowing SSH. The default policy drops incoming traffic, your live SSH session freezes, and the next connection is refused. You are now outside your own server, over the network at least.

Always run ufw allow OpenSSH first, then ufw enable. On a cloud box, keep the provider web console open as a rescue path, because it reaches the machine without going through SSH or the firewall. Test the new rules from a second terminal before you close the first.

Ban the brute force, and stop working as root

Even with keys only, the log noise continues, and a determined bot keeps knocking. fail2ban watches the auth log and temporarily bans any address that fails to log in too many times. Install it, enable it, and the constant background attacks drop to almost nothing.

$ sudo apt install fail2ban        # or dnf install fail2ban
$ sudo systemctl enable --now fail2ban
$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Currently banned: 37
`- Total banned:     412

The last piece is a habit, not a package. Do your daily work as a normal user and use sudo for the moments that need it, as covered in the users part of this series. A mistake typed as an ordinary user damages your files. The same mistake typed as root can damage the whole system. The gap between those two outcomes is the reason least privilege exists.

Failed SSH logins per day password open 4200 keys plus fail2ban 12 Same server, same week. Key only login plus fail2ban does the heavy lifting.

Close the doors you are not using

A service you are not using is a door you forgot to lock. Before you trust the firewall, look at what is actually listening on the machine. ss -tlnp lists every port with a program bound to it, so you can see your real exposure rather than what you assume it is.

$ sudo ss -tlnp
State   Local Address:Port   Process
LISTEN  0.0.0.0:22           sshd
LISTEN  0.0.0.0:80           nginx
LISTEN  127.0.0.1:3306       mysqld

The address before the port matters as much as the port itself. 0.0.0.0 means the service listens on every network and can be reached from outside. 127.0.0.1 means it listens only on the machine itself, which is exactly what you want for a database that only a local application talks to. A database bound to 0.0.0.0 on a public server is one of the most common serious mistakes, so bind internal services to localhost.

Then turn off what you do not need. Every running service is another thing to patch and another possible way in, so disable the ones a fresh install started that you will never use. Fewer services means a smaller surface to defend.

$ systemctl list-units --type=service --state=running
$ sudo systemctl disable --now avahi-daemon    # example, if unused

A real opinion: moving the SSH port is not security

A popular tip says to move SSH off port 22 to some high number and the attacks stop. The attacks against that one port do drop, which is why people believe it works, but this is not real security. A scanner finds your new port in seconds, and a listener on a high port is often less protected by default. Port changing is a curtain, not a lock. It can cut log noise, and that is the honest reason to do it, but never mistake a quieter log for a safer server.

Spend your effort where it counts. Key only login, a default deny firewall, fail2ban, and current patches defeat essentially all automated attacks whatever port you use. If you want one genuine upgrade beyond this list, add multi factor to SSH. That raises the bar for a real human attacker in a way that renumbering a port never will.

Harden a box now

On a throwaway VM, do the two changes that matter most and confirm you are still in before locking the door. Create a key, copy it up, prove it works, then check the SSH config:

ssh-keygen -t ed25519
ssh-copy-id you@testvm
ssh you@testvm            # must succeed with no password prompt
sudo sshd -t             # check config is valid before restart

Only after that clean key login works do you set PasswordAuthentication no and restart SSH. The sshd -t check catches a typo that would otherwise break the service on restart.

In the interview

You are handed a brand new internet facing Linux server. What are the first things you do to secure it?

Give the ordered list and say why. Apply all updates and enable automatic security patches. Set SSH to key only and disable root login. Put a default deny firewall in front and open only the ports in use. Add fail2ban. Work through sudo rather than as root. The interviewer wants to hear patching first, key based SSH, and a firewall that fails closed, plus the awareness to allow SSH before enabling the firewall so you do not lock yourself out.

What people get wrong

Is a firewall enough on its own? No. A firewall only controls which ports are reachable. If SSH is open, which it must be for you to work, the login itself still has to be hardened with keys and fail2ban. Layers work together; no single one is enough.

Do I still need updates if the server is behind a firewall? Yes, absolutely. Plenty of attacks arrive through the very services you allow, like a web app, and firewalls do nothing about a flaw in software you deliberately expose. Patching is not optional.

Should I disable SSH entirely to be safe? Only if you truly never need remote access, which is rare. The goal is not to remove SSH but to make it key only and watched. A locked door you can open is the point, not a wall.

What is the single highest value change? Patching, then key only SSH. If you only had time for two things, current software and no password logins would stop the vast majority of what actually happens to servers.

Does fail2ban replace a firewall? No. fail2ban reacts to bad behavior after it starts; the firewall blocks whole categories of traffic up front. They cover different gaps and belong together.

Should I leave SELinux or AppArmor on? Yes. Both confine what a service is allowed to do, so a compromised web server cannot roam the whole system. Beginners often disable SELinux at the first permission error, which throws away a real containment layer. Learn to adjust one policy rather than switch the whole thing off.

Is hardening a one time job? No, treat it as ongoing. New patches land every week, services get added over time, and yesterday safe defaults drift. Re-check what is listening, keep automatic updates on, and glance at the fail2ban bans now and then so the box stays as locked down as the day you set it up.

Is exposing SSH to the whole internet ever acceptable? With key only login and fail2ban it is common and workable for most servers. When you want more, put SSH behind a VPN or a bastion host so the machine is not reachable from the open internet at all, and only the hardened bastion is exposed. That shrinks the surface a real attacker can even touch.

Your server now shrugs off the automated internet: patched, key only, firewalled, watched, and run without standing as root. When something still goes wrong, and it will, you need a method rather than panic. That is the next part, a troubleshooting approach that finds the fault instead of guessing at it. Work the series in order from the Complete Guide.

References

Linux for Beginners · Part 17 of 24
« Previous: Part 16  |  Complete Guide  |  Next: Part 18

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading