How to Set Up SSH Key Authentication in Linux: A Step-by-Step Deep Dive

Secure Shell (SSH) is a cornerstone of secure remote access in Linux environments. While password-based authentication is common, SSH key authentication offers superior security and convenience. By using cryptographic keys instead of passwords, you eliminate the risk of brute-force attacks and simplify logins with automation. In this guide, we’ll explore how to create SSH keys, perform an SSH key exchange, and configure the sshd_config file for key-based login. Whether you’re a system administrator or a Linux enthusiast, this deep dive will equip you with everything you need to master SSH key authentication.

What Is SSH Key Authentication?

SSH key authentication relies on a pair of cryptographic keys: a private key (kept secret on your local machine) and a public key (shared with the remote server). When you attempt to log in, the server verifies your private key against the public key, granting access without requiring a password. This method is not only more secure but also ideal for scripts and automated tasks.

In this article, we’ll break down the process into three main sections:

1. Generating SSH Keys in Linux
2. Performing an SSH Key Exchange
3. Configuring sshd_config for Key-Based Login

Let’s get started.

Section 1: Generating SSH Keys in Linux

Why Use SSH Keys?

Before diving into the how-to, let’s understand the why. Passwords can be guessed or stolen, especially if they’re weak or reused. SSH keys, on the other hand, are nearly impossible to crack due to their complexity and length (typically 2048 or 4096 bits). Plus, once set up, they allow passwordless logins, saving time and effort.

Step-by-Step Guide to Create SSH Keys

To generate SSH keys in Linux, you’ll use the ssh-keygen command. Here’s how:

Step 1: Open Your Terminal

Launch a terminal on your local Linux machine. This is where you’ll generate and manage your keys.

Step 2: Run the ssh-keygen Command

Type the following command and press Enter:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

• -t rsa: Specifies the key type (RSA is widely supported and secure).
• -b 4096: Sets the key length to 4096 bits for enhanced security (2048 is also common but less robust).
• -C “your_email@example.com”: Adds a comment (e.g., your email) to identify the key.

Step 3: Choose a File Location

You’ll be prompted to specify where to save the key pair. Press Enter to accept the default location (/home/$user/.ssh/id_rsa). The private key will be saved as id_rsa, and the public key as id_rsa.pub.

Step 4: Set a Passphrase (Optional, but highly recommended)

Next, you’ll be asked to enter a passphrase:

Enter passphrase (empty for no passphrase):

A passphrase adds an extra layer of security to your private key. If someone steals it, they can’t use it without the passphrase. For convenience (e.g., automation), you can leave it blank by pressing Enter twice. However, for sensitive systems, always use a strong passphrase.

Step 5: Verify Key Creation

Once completed, you’ll see output like this:

Your identification has been saved in /home/user/.ssh/id_rsa
Your public key has been saved in /home/user/.ssh/id_rsa.pub

Check the .ssh directory with:

ls -l ~/.ssh/

You should see id_rsa (private key) and id_rsa.pub (public key).

Section 2: Performing an SSH Key Exchange

What Is an SSH Key Exchange?

The key exchange involves copying your public key to the remote server so it recognizes your local machine. Once the server has your public key, it can authenticate you without a password.

Method 1: Using ssh-copy-id (Recommended)

The easiest way to perform an SSH key exchange is with the ssh-copy-id command. Here’s how:

Step 1: Ensure You Have Access

Make sure you can log in to the remote server with a password first:

ssh user@remote_server_ip

Replace user with your username and remote_server_ip with the server’s IP address.

Step 2: Copy the Public Key

Run this command from your local machine:

ssh-copy-id user@remote_server_ip

You’ll be prompted to enter your password. Once authenticated, the command appends your public key to the remote server’s ~/.ssh/authorized_keys file.

Step 3: Test the Connection

Log out and try logging in again:

ssh user@remote_server_ip

If successful, you’ll connect without entering a password. If you added a password to your key (as recommended) you will first need to unlock your key, before you will be logged in.

Method 2: Manual Key Exchange

If ssh-copy-id isn’t available, you can manually copy the key:

Step 1: Display Your Public Key

On your local machine, print your public key:

cat ~/.ssh/id_rsa.pub

Copy the output (it starts with ssh-rsa and ends with your comment).

Step 2: Log In to the Remote Server

Access the server via SSH with your password:

ssh user@remote_server_ip

Step 3: Add the Key to authorized_keys

(On the remote server, ensure the .ssh directory exists:)

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Edit or create the authorized_keys file:

nano ~/.ssh/authorized_keys

Paste your public key, save (Ctrl+O, Enter), and exit (Ctrl+X). Set the correct permissions:

chmod 600 ~/.ssh/authorized_keys

Step 4: Test Passwordless Login

Log out and reconnect from your local machine. It should work seamlessly. If you added a password to your key (as recommended) you will first need to unlock your key, before you will be logged in.

Section 3: Configuring sshd_config for Key-Based Login

Understanding sshd_config

The SSH daemon (sshd) configuration file, typically located at /etc/ssh/sshd_config, controls how the server handles authentication. To enforce or customize key-based login, you’ll need to edit this file.

Step-by-Step Configuration

Step 1: Back Up the File

Before editing, create a backup:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

Step 2: Open sshd_config

Use a text editor like nano or vim:

sudo nano /etc/ssh/sshd_config

Step 3: Enable Key-Based Authentication

Look for or add these lines:

PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

• PubkeyAuthentication yes: Enables public key authentication.
• AuthorizedKeysFile: Specifies where the server looks for public keys (default is usually fine).

Step 4: Disable Password Authentication (Optional)

For maximum security, disable password logins:

PasswordAuthentication no

Only do this after confirming key-based login works, or you could lock yourself out!

Step 5: Adjust Additional Settings (Optional)

• PermitRootLogin: Restrict root login to keys only:

PermitRootLogin prohibit-password

ChallengeResponseAuthentication: Disable if unused:

ChallengeResponseAuthentication no

Step 6: Save and Restart SSHD

Save the file and reload the SSH service:

sudo systemctl daemon-reload
sudo systemctl restart sshd

Step 7: Test the Configuration

From your local machine, test the connection, and if it fails, revert to the backup and troubleshoot.

Security Best Practices

• Use Strong Keys: Stick to 4096-bit RSA or Ed25519 keys.
• Regularly Rotate Keys: Replace keys periodically.
• Audit authorized_keys: Remove unused keys.

Conclusion: SSH Key Authentication in Linux

Setting up SSH key authentication in Linux is a powerful way to enhance security and streamline remote access. By generating a key pair with ssh-keygen, exchanging the public key with ssh-copy-id or manually, and fine-tuning sshd_config, you can create a robust, passwordless login system. This deep dive has walked you through each step with practical commands and troubleshooting tips.

Whether you’re securing a personal server or managing a fleet of machines, SSH keys are a must-have skill. Start implementing these steps today, and enjoy the peace of mind that comes with a more secure Linux environment.