What Are SSH PEM Keys?

SSH PEM (Privacy Enhanced Mail) keys are cryptographic files used for secure authentication. They eliminate password-based logins and provide stronger security through asymmetric encryption. SSH PEM keys consist of a private key kept secret on your local machine and a public key installed on remote servers.

Quick Answer

Generating Your SSH Key Pair

Step-by-Step Instructions

Step 1: Open a terminal on your local machine.

Step 2: Run the following command:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N ""

Step 3: Your keys are now generated in ~/.ssh/:

• id_rsa - Your private key (keep this secret)
• id_rsa.pub - Your public key (share with servers)

Step 4: Secure your private key with proper permissions:

chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub

Adding Your Public Key to a Server

Once you have your SSH key pair, add the public key to the target server:

ssh-copy-id -i ~/.ssh/id_rsa.pub user@server_ip

Or do it manually:

cat ~/.ssh/id_rsa.pub | ssh user@server_ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Connecting with Your SSH Key

Now you can connect to the server without entering a password:

ssh -i ~/.ssh/id_rsa user@server_ip

For the default SSH key location, you can simply use:

ssh user@server_ip

Troubleshooting

Common Issue 1: Permission denied (publickey)

Verify authorized_keys permissions are exactly 600:

chmod 600 ~/.ssh/authorized_keys

Common Issue 2: Can't find key file

Check whether the key exists in the expected location:

ls -la ~/.ssh/

Common Issue 3: Server rejects key

Ensure the public key is properly copied to the server and has correct permissions:

ssh user@server_ip "cat ~/.ssh/authorized_keys"

Common Issue 4: SSH timeout

Check that the firewall allows port 22 and the SSH service is running:

sudo systemctl status ssh

Best Practices for 2026

• Use RSA-4096 or Ed25519 keys - Stronger encryption than RSA-2048
• Always set a passphrase - Protects your private key if compromised
• Restrict key permissions - Keep private keys at 600, public keys at 644
• Keep private keys safe - Never share or upload them to version control
• Document key purposes - Maintain a list of which keys are used where
• Rotate keys annually - Generate new keys and update all servers
• Use SSH config file - Simplify connection management with aliases

SSH Config File Example

Create ~/.ssh/config for easier connections:

Host production
    HostName 203.0.113.50
    User deploy
    IdentityFile ~/.ssh/id_rsa
    Port 22

Host staging
    HostName 198.51.100.75
    User deploy
    IdentityFile ~/.ssh/id_rsa_staging
    Port 2222

Now you can simply use ssh production.

SSH PEM authentication remains one of the simplest ways to secure direct server access. Once your key pair is in place, daily connections become faster, more reliable, and easier to automate across environments.