How to initialize a server

This article is somewhat outdated.

First things first. What are you going to use your server for? Does it really need to serve internet-facing services?

For internet-facing servers, you'll want to choose a proper server-grade distribution. I've never had issues with Ubuntu Server (pick a suitable LTS version - at the time of writing that's 18.04 Bionic Beaver) or Debian 10. They're rock solid when configured well, and updated regularly (try to update it about twice a month, or automate it away).

So you've picked your distribution at your favorite hoster. Ubuntu is what we'll stick with for this tutorial.

Most of the time you'll be using SSH to interface with your servers.
To configure your SSH access, generate yourself a keypair on the node(s) that'll be accessing your server.
ssh-keygen -t ed25519
This will generate you an ed25519 keypair, which is as secure as RSA, but with a far smaller keylength. It's the current industry standard for secure servers.
It is highly recommended to use a password on this key, so that you'll have 2FA (something which you have - your keys, and something you know - your password). Make this password a strong one. You may also want to look into using solutions like Google Authenticator with your server.

So you've got yourself your keypair. Let's copy it over to the server. On the client, show the file with the following command.
cat ~/.ssh/id_ed25519.pub
Copy that output.
ssh user@server
You should be logged into your server now.
vim ~/.ssh/authorized_keys
Now paste your public key into that file, and exit using Esc :wq.
You should now exit the server shell, and try to login again. You'll be either logging in without password now (using your key) or be required to enter your key's password.

Now you've got a server with key-based SSH authentication. Let's make things final by disallowing password authentication altogether. Log back into your server.
sudo vim /etc/ssh/sshd_config
Now navigate to the line stating #PasswordAuthentication = yes
Omit the # in that line, and change "yes" to "no".
Now restart the sshd service.
sudo systemctl restart sshd
Exit from the server shell and log back in again.
If you can still log in, congratulations. Your server should now accept only keys. You could verify this from another client which doesn't have your current keys. It should get a rejection from the server, even if the password is correct.

Further security can be achieved from using a VPN server. I'd like to point you at Digital Ocean for this, it's the tutorial I use for setting up VPN servers all the time. This tutorial is made for Ubuntu 16.04, but it applies to Ubuntu 18.04 as well.
Once you've got that set up, you should have a running VPN server, a CA, and a client keypair.
Next up is configuring the firewall to deny SSH connections that don't come from its VPN network.
Be sure to check your VPN connection to the server first!!! And check your connectivity to 22/tcp from within the VPN as well (using nc -vz 10.8.0.1 22 or similar). You could lose your server access from improper iptables editing!!!
Now, in your server's settings, you may want to disable ufw. It's a framework for iptables that's still incomplete. It's unusable for advanced iptables configuration at this point in time, so I tend to disable it.
sudo systemctl disable ufw
Now, let's edit our iptables configuration.
Assuming that you went with 10.8.0.0/24 in your OpenVPN config, these are the appropriate iptables commands.
sudo iptables -A INPUT -s 10.8.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p udp -m udp --dport 1194 -j ACCEPT
sudo iptables -P INPUT DROP
Keep in mind that that last command can make you lose your server access if you don't do it right. Once again, be sure to check whether you're in 10.8.0.0/24 first!
You'll also want to export these rules if you use the persistence service iptables-persistent for iptables (which you definitely should).
sudo sh -c "iptables-save > /etc/iptables/rules.v4"

So now you've got a server running a secure sshd, hidden behind a VPN connection, which should eliminate bots cluttering your sshd logs.
At this point, all that's left really is learning more about iptables (which I'll talk about later), and periodic system updates. Congratulations! You've now got yourself a relatively secure server. Keep in mind that for further service installations you'll have to whitelist them in the firewall first. Be sure to learn about iptables.

Thanks for reading this post. If you've got any more questions, feel free to ping me on Facebook or Telegram. I'm most active on Telegram. Please do not ping me if there's an appropriate group for it though. See catb's smart questions article to learn why.
On Telegram, you can also check out Silicon Network where I'm active in most groups. You can ask questions in the appropriate groups there. Be sure to do your own research first though.