Working with Git in Java: Part 2 - Using SSH with JGit

A developer working on a laptop with a Git repository open.

In Part 1, I introduced JGit and its basic concepts for working with Git in Java. We covered how to clone a repository, make some changes to it, stage those changes, commit them, and push them back to a remote repository using a PAT (Personal Access Token) for authentication over HTTPS. If you’re not familiar with JGit at all, then I recommend reading Part 1 first.

In this second and final part, we’ll be looking at how to use SSH with JGit to authenticate with a remote repository hosted on GitHub. If you’re keen to look at some code examples right away, I’ve created an accompanying repository containing a complete example of using JGit with SSH.

I’ll be starting by covering some of the core concepts of SSH and how GitHub uses SSH for both personal and repository-specific keys. So if you’re already familiar with SSH and how it works, especially with GitHub, feel free to skip to Using SSH with JGit. I’ll be covering what SSH key pairs are, when to use a passphrase, and where the SSH keys get stored on your machine as well as discussing the known_hosts and config files. Then we’ll look at how GitHub uses SSH for both personal and repository-specific keys, finishing with how to generate an SSH key pair for use with GitHub.

After that, we’ll dive into using SSH with JGit to authenticate with a remote repository on GitHub. We’ll cover two methods: using a key stored in the ~/.ssh directory associated with your GitHub account, and providing a key programmatically for a specific repository.

Let’s get started!

SSH Core concepts

SSH is a secure protocol used to connect to remote servers securely over an unsecured network using a public-private key pair (instead of a username and password).

SSH Key Pairs

SSH keys are a pair of cryptographic keys used to authenticate with a server over SSH and are made up of:

A public key - shared with the server you want to connect to.
A private key - kept secret on the client machine.

These keys are asymmetric, meaning that data encrypted with one key can only be decrypted with the other key.

An SSH key pair can be generated using different algorithms, such as RSA, ECDSA, and ED25519. Personally, I recommend using the ED25519 algorithm as it is more secure and is the algorithm recommended by GitHub.

Passphrase protected private keys

A passphrase is an optional feature of an SSH key pair. It’s essentially a password that you need to provide whenever you use the private key to authenticate with a server. This added layer of security means that even if someone gets hold of your private key, they won’t be able to use it without knowing the passphrase.

Depending on your use case, you may or may not want to use a passphrase. For example, if your private key is stored on a shared machine, then someone with access to that machine would need to know the passphrase to use the private key.

.ssh folder

The default location for SSH keys is the ~/.ssh directory on your machine. This is where the ssh-agent running on your machine looks for your keys when you try to connect to a server. It contains your private keys, public keys, and some configuration files.

These configuration files include:

known_hosts - a file that stores the public keys of servers you have connected to.
config - a configuration file that tells the ssh-agent what keys to use for which servers.

Let’s look at these files in more detail.

Protecting against bad actors with known_hosts

When a client initiates a connection to a server, both parties must verify each other’s identity. The server sends a challenge to the client, where the client uses its private key to decrypt and respond to the challenge. This challenge-response sequence confirms the client’s possession of the private key that corresponds to the public key stored on the server.

When connecting to a new server for the first time, you’ll usually be prompted to accept the server’s public key. This key is stored in the known_hosts file for future connections. If the SHA-256 hash of the server’s public key matches the one stored in the known_hosts file, the connection is allowed. This is because the known_hosts file serves as a record of trusted servers and their corresponding public key fingerprints, which are calculated using the SHA-256 algorithm.

It is always a good idea to verify the server’s public key before accepting it. In the case of GitHub, you can find the public keys and their corresponding fingerprints for their servers on their SSH key fingerprints page.

config file

The config file in the ~/.ssh directory is used to configure SSH connections. It allows you to set up different configurations for different hosts.

For example, for github.com, we can specify which private key to use when connecting:

Host github.com
  HostName github.com
  IdentityFile ~/.ssh/my-private-github-key

GitHub and SSH

GitHub supports using SSH to access repositories. To enable SSH access, you can add an SSH key to your GitHub account or set it up as a deploy key for access to a specific repository.

Personal SSH keys

When we talk about personal SSH keys, we mean an SSH key pair associated with a GitHub account. This key pair resides in the ~/.ssh directory on your machine. You add the public key to your GitHub account and use the corresponding private key to authenticate with GitHub.

Having a personal SSH key associated with a GitHub account is useful when interacting with multiple repositories through the command line or other Git clients. It is recommended to use a passphrase to protect your private key. This ensures that when you perform sensitive operations, such as pushing to a repository, you will need to provide the passphrase.

For more information on adding an SSH key to a GitHub account, see the GitHub documentation here.

Deploy Keys

When an application needs to interact with a repository (e.g. an application using JGit), it is not advisable to give it access to a personal SSH key associated with a GitHub account. This is because the application would then have access to all repositories associated with that account.

Instead, we can use a Deploy Key to grant access to a single repository. A Deploy Key is an SSH key stored in a repository’s settings that grants read-only or read-write access to that repository. When generating the key pair for a deploy key, it is recommended not to put it in the ~/.ssh folder, as it is intended for use by a specific application or service rather than for personal use. Additionally, it is generally unnecessary to use a passphrase for the private key associated with a deploy key. This is because the private key should already be managed securely, for example, through secret management software or utilising secure environment variables.

For more information on Deploy Keys, see the GitHub documentation here.

Generating an SSH key pair

When generating an SSH key pair, you can choose the type of key to create and whether to use a passphrase to secure the private key. We use the ssh-keygen command to generate SSH keys. In the following examples, I’ll show you how to generate an SSH key pair intended for personal use and for a deploy key.

As previously mentioned, I’ve chosen to use the ED25519 signature algorithm for my SSH keys. If you’re interested in learning more about ED25519 and why it’s a good choice for SSH keys, I recommend reading this blog post.

Generating an SSH key pair for personal use

To generate an SSH key pair for personal use (one that you’ll add to your GitHub account), you can use the following command:

ssh-keygen -t ed25519 -f ~/.ssh/my_ed25519_key

Then you’ll be prompted to enter a passphrase for the private key (you can leave it empty if you don’t want one, but it’s recommended to use one).

This will generate two files:

~/.ssh/my_ed25519_key - the private key.
~/.ssh/my_ed25519_key.pub - the public key.

The public key is what you’ll add to your GitHub account.

You can then configure the ~/.ssh/config file to use this key for GitHub:

Host github.com
  IdentityFile ~/.ssh/my_ed25519_key

Generating an SSH key pair for a deploy key

To generate an SSH key pair for a deploy key (one that you’ll use for a specific repository), I recommend creating a separate directory to store it. This is to separate such keys from personal keys stored in the ~/.ssh directory.

For example, you can create a directory called deploy-keys in your home directory:

mkdir ~/deploy-keys

and then generate the key pair using the following command:

ssh-keygen -t ed25519 -f ~/deploy-keys/my_specific_repo_ed25519_key

This will generate two files:

~/deploy-keys/my_specific_repo_ed25519_key - the private key.
~/deploy-keys/my_specific_repo_ed25519_key.pub - the public key.

You can then upload the public key as a deploy key to use with a specific repository on GitHub and provide the private key to the relevant infrastructure that needs to trust the host.

So far, we have covered some of the core concepts of SSH, including the use of key pairs and the importance of protecting private keys with passphrases. We also discussed how GitHub uses both personal and deploy keys for authentication. Additionally, we explored the process of generating SSH key pairs.

Now, let’s shift our focus back to JGit, the primary subject of this article, and delve into how to use SSH with JGit for secure interactions with remote repositories hosted on GitHub.

Using SSH with JGit

Adding an extra library for better SSH support

The core JGit library has limited SSH support provided by the JSch library. However, this library is no longer maintained and is lacking support for modern SSH features.

The good news is that JGit provides a JGit Apache SSHD Based SSH Support library that integrates with Apache MINA SSHD that provides us with better key support and modern ciphers. It’s actively maintained and supports modern SSH features such as ED25519 keys.

For example, in a Gradle project we’d include the following dependencies in build.gradle:

dependencies {
    implementation 'org.eclipse.jgit:org.eclipse.jgit:7.0.0.202409031743-r'
    implementation 'org.eclipse.jgit:org.eclipse.jgit.ssh.apache:7.0.0.202409031743-r'
}

A recap: How JGit works with remote repositories

In part 1 we covered how to clone a repository using JGit over HTTPS.

For example, this is how to clone a protected repository with an HTTPS URI using a PAT token for authentication:

CloneCommand cloneCommand = Git.cloneRepository()
    .setDirectory(Path.of("./my-repo").toFile())
    .setURI("https://github.com/my-username/my-repo.git")
    .setCredentialsProvider(
         new UsernamePasswordCredentialsProvider("OUR PAT TOKEN HERE", ""));

// try with resources to ensure the Git object is closed
try (Git git = cloneCommand.call()) {
    // We can now work with the Git object to interact with our cloned repository
}

To do the equivalent of this with SSH, we need to change the URI and depending on whether we’re using a key stored in the ~/.ssh directory or providing the key programmatically, will determine how we authenticate with the remote repository.

Cloning via SSH

SSH URLs are different from HTTPS URLs and will typically look like this:

david@host:/path

However, when working with GitHub, the SSH URI format is as follows:

git@github.com/my-username/my-repo.git

GitHub requires the git user to log in to the github.com server when using SSH. It will determine the GitHub user based on the public key associated with the private key used for authentication. GitHub identifies the public key and retrieves our user details from its database. As a result, the service will grant us access to the repository if we have the necessary permissions.

Think of the public key as your username and the private key as your password.

Now, let’s see how we can clone a repository using JGit over SSH using a local key or programmatically providing the private key and passphrase.

Clone using a key stored in the `~/.ssh` directory

As mentioned earlier, the default location for SSH keys is the ~/.ssh directory on your machine. Let’s see how we can configure JGit to clone a repository using the SSH protocol using a key stored in this directory.

Using a passphrase-less key

Provided we have a passphrase-less SSH key stored in the ~/.ssh directory on our machine that is associated with our GitHub account, this is how we can clone a protected repository:

CloneCommand cloneCommand = Git.cloneRepository()
    .setDirectory(Path.of("./my-repo").toFile())
    .setURI("git@github.com/my-username/my-repo.git");

try (Git git = cloneCommand.call()) {
    // We can now work with the Git object to interact with our cloned repository
}

JGit out of the box will scan the ~/.ssh directory for the appropriate key to use and any configuration set in the ~/.ssh/config and ~/.ssh/known_hosts files.

Using a passphrase

Remember from earlier that we can add a passphrase to our private key for added security. When we have a passphrase protected SSH key, we need to provide the passphrase in code to authenticate with the key.

Depending on the operation, a JGit command object can be configured with a CredentialsProvider object using the setCredentialsProvider method. We provide the passphrase for an SSH key as the password field in a UsernamePasswordCredentialsProvider object:

CloneCommand cloneCommand = Git.cloneRepository()
    .setDirectory(Path.of("./my-repo").toFile())
    .setURI("git@github.com/my-username/my-repo.git")
    .setCredentialsProvider(
         new UsernamePasswordCredentialsProvider("", "OUR PASSPHRASE HERE"));

try (Git git = cloneCommand.call()) {
    // We can now work with the Git object to interact with our cloned repository
}

Programmatically providing the private key and passphrase

So far, we’ve covered how to use keys stored in the ~/.ssh directory which works well when running JGit commands locally on your machine.

But what if instead, an application running JGit hosted in a Docker container needed to authenticate with a remote repository using SSH? We could add an ssh folder to the Docker image with the key, but a better way would be to provide the key programmatically to JGit via environment variables or some secret management system.

As we saw earlier, out of the box, JGit will look for keys in the ~/.ssh directory but this won’t work for us. Instead, we need a way to programmatically provide the private key (and the passphrase if you generated one) to JGit. This can be achieved by building a SshdSessionFactory and setting it globally for all JGit commands. Let’s see how we can do this.

Building a SshdSessionFactory object

The SshdSessionFactory object is responsible for creating SSH sessions for JGit commands. It is used to configure the client when connecting to a remote repository over SSH. To programmatically provide the private key and passphrase to JGit, we need to build a SshdSessionFactory object which can be done using the SshdSessionFactoryBuilder class.

To build a SshdSessionFactory, we need to provide the following information to the builder:

The preferred authentication method (e.g. publickey).
The private key we want to use and the passphrase for the key (if it has one). This is done by providing a DefaultKeysProvider object.
Provide the fingerprint of the public key on the remote server to verify the server’s identity. This is done by providing a ServerKeyDatabase object.

Note: Due to the way the SshdSessionFactory is built, we need to provide the builder with directories for the home and SSH directories. This might seem a bit odd since we are configuring the SshdSessionFactory to use an in-memory set of key pairs, but these directories are required by the SshdSessionFactory object. Setting them to null will result in a NullPointerException.

I’ve raised a GitHub issue to clarify why this is the case and if it can be improved which is being looked into by the JGit team.

For now, the only advice I can give is to set these directories to a temporary directory that you create for this purpose. See: Comment from tomaswolf

Providing the private key to the SshdSessionFactoryBuilder

The SshdSessionFactoryBuilder class includes a DefaultKeysProvider object that accepts a Function, enabling us to programmatically provide the private key and passphrase. This requires a KeyPair object, which can be created using the SecurityUtils.loadKeyPairIdentities method from the Apache sshd library.

I’ve written a helper method to load the private key into a KeyPair object from a string:

private Iterable<KeyPair> loadKeyPairs(@Nonnull String privateKeyContent, @Nullable String passphrase) {
    Iterable<KeyPair> keyPairs;
    try {
        keyPairs = SecurityUtils.loadKeyPairIdentities(null,
                null, new ByteArrayInputStream(privateKeyContent.getBytes()), (session, resourceKey, retryIndex) -> passphrase);
    } catch (Exception e) {
        throw new IllegalArgumentException("Failed to load ssh key pair", e);
    }
    return keyPairs;
}

Then we can use this method to provide the private key and passphrase to the SshdSessionFactoryBuilder:

Iterable<KeyPair> keyPairs = loadKeyPairs("OUR PRIVATE KEY", "OUR OPTIONAL PASSPHRASE");

SshdSessionFactory sshSessionFactory = new SshdSessionFactoryBuilder()
    .setPreferredAuthentications("publickey")
    .setDefaultKeysProvider(ignoredSshDirBecauseWeUseAnInMemorySetOfKeyPairs -> keyPairs)
    // ... other configuration
    .build(new JGitKeyCache());

Configuring the server key database (known_hosts)

Just as we need to verify the server’s identity when connecting to a server using SSH in the terminal, we need to do the same when creating our SshdSessionFactory object.

The ServerKeyDatabase interface provides two methods:

lookup - to look up the public key of a server based on the server’s address.
accept - whether to accept or reject the server’s public key based on the server’s address.

When building a SshdSessionFactory object, we need to provide an implementation of the ServerKeyDatabase interface to determine whether to accept or reject the server’s public key. As a reminder, we can find the public keys and their corresponding fingerprints for GitHub servers on their SSH key fingerprints page.

For example, to accept GitHub’s public key for the github.com server, we would need to set up the server key database like this on the SshdSessionFactoryBuilder:

SshdSessionFactory sshSessionFactory = new SshdSessionFactoryBuilder()
    .setServerKeyDatabase((ignoredHomeDir, ignoredSshDir) -> new ServerKeyDatabase() {
        @Override
        public List<PublicKey> lookup(String connectAddress, InetSocketAddress remoteAddress, Configuration config) {
            // Empty because we compare with a specific key in the accept method
            return Collections.emptyList();
        }

        @Override
        public boolean accept(String connectAddress, InetSocketAddress remoteAddress,
                              PublicKey serverKey, Configuration config, CredentialsProvider provider) {
            PublicKeyEntry gitHubPublicKeyEntry = PublicKeyEntry
                    .parsePublicKeyEntry("GitHub's public key here");

            PublicKey gitHubPublicKey;
            try {
                gitHubPublicKey = gitHubPublicKeyEntry.resolvePublicKey(null, null, 
                        PublicKeyEntryResolver.IGNORING);
            } catch (IOException | GeneralSecurityException e) {
                throw new RuntimeException(e);
            }

            return KeyUtils.compareKeys(serverKey, gitHubPublicKey);
        }
    })
    // ... other configuration
    .build(new JGitKeyCache());

Earlier, we generated an SSH key using the ED25519 signature algorithm for when we communicate with GitHub. However, it is important to note that the server’s public key may not always be in the same format as the key we provide. The specific key type used for a connection depends on the capabilities and preferences of the SSH client. So despite us using an ED25519 key, GitHub might provide a different type of key that we need to accept.

So depending on the SSH client’s capabilities, we may receive a different key type from the server such as a ecdsa-sha2-nistp256 key. So in the accept method, we need to compare the keys to see if they match.

Putting it all together

Here’s how we can build a SshdSessionFactory object providing the private key and passphrase programmatically, and configure the server key database to accept GitHub’s public key:

// These may be read from an environment variable or secret management system
String privateKeyContent = "OUR PRIVATE KEY HERE";
// leave empty or null if the key you're using is passphrase-less (can be null)
String passphrase = "OUR OPTIONAL PASSPHRASE HERE";

// Load the private key into a KeyPair object
Iterable<KeyPair> keyPairs = SecurityUtils.loadKeyPairIdentities(
        null, null, new ByteArrayInputStream(privateKeyContent.getBytes()), (session, resourceKey, retryIndex) -> passphrase
);

// Create a temporary directory to use for the home directory and SSH directory
// This is required by the SshdSessionFactory object despite us not using them
Path temporaryDirectory;
try {
    temporaryDirectory = Files.createTempDirectory("ssh-temp-dir");
} catch (IOException e) {
    throw new RuntimeException("Failed to create temporary directory", e);
}

// Build the SshdSessionFactory object
SshdSessionFactory sshSessionFactory = new SshdSessionFactoryBuilder()
    .setPreferredAuthentications("publickey")
    .setDefaultKeysProvider(ignoredSshDirBecauseWeUseAnInMemorySetOfKeyPairs -> keyPairs)
    .setHomeDirectory(tempDirectoryForSshSessionFactory.toFile())
    .setSshDirectory(tempDirectoryForSshSessionFactory.toFile())
    .setServerKeyDatabase((ignoredHomeDir, ignoredSshDir) -> new ServerKeyDatabase() {
        @Override
        public List<PublicKey> lookup(String connectAddress, InetSocketAddress remoteAddress, Configuration config) {
            return Collections.emptyList();
        }

        @Override
        public boolean accept(String connectAddress, InetSocketAddress remoteAddress, 
                              PublicKey serverKey, Configuration config, CredentialsProvider provider) {
            PublicKeyEntry gitHubPublicKeyEntry = PublicKeyEntry
                    .parsePublicKeyEntry("GitHub's public key here");

            PublicKey gitHubPublicKey;
            try {
                gitHubPublicKey = gitHubPublicKeyEntry.resolvePublicKey(null, null, 
                        PublicKeyEntryResolver.IGNORING);
            } catch (IOException | GeneralSecurityException e) {
                throw new RuntimeException(e);
            }

            return KeyUtils.compareKeys(serverKey, gitHubPublicKey);
        }
    })
    //The JGitKeyCache handles the caching of keys to avoid unnecessary disk I/O and improve performance
    .build(new JGitKeyCache());

Once the sshSessionFactory is built, it needs to be set globally for all JGit commands before we can do any JGit operations:

SshSessionFactory.setInstance(sshSessionFactory);

Then we can use JGit to clone a repository using SSH with the private key and passphrase provided programmatically:

CloneCommand cloneCommand = Git.cloneRepository()
        .setDirectory(Path.of("./my-repo").toFile())
        .setURI("git@github.com/my-username/my-repo.git");

try (Git git = cloneCommand.call()) {
    // We can now work with the Git object to interact with our cloned repository
}

And that’s it! We don’t need to worry about supplying the private key and passphrase in the code for every JGit command as it’s handled by the SshdSessionFactory object for any operation that requires SSH authentication.

Sample Java application using environment variables

For a complete example of using JGit with SSH, where the private key and passphrase are provided via environment variables, check out the sample Java application I wrote. It demonstrates how to clone a repository using SSH.

You can find the code for this application in the jgit-tutorial repository on Auto Trader’s GitHub.

Conclusion and closing thoughts

And that’s it for this tutorial series on working with Git in Java using JGit!

In this two-part series, we have explored the fundamentals of working with Git repositories using JGit in Java. This includes cloning a remote repository, making changes, committing those changes, and pushing them back to the repository using both HTTPS and SSH for authentication.

I hope you’ve found this tutorial series helpful and that you’ve learned something new about working with Git in Java.

I really enjoyed working with JGit and found it to be a valuable addition to my toolkit. In the future, I’d like to explore more of JGit’s capabilities and see how it can be used in more complex scenarios such as:

Working with branches and merging changes
Inspecting the history of a repository
Handling conflicts when merging changes

Please feel free to reach out if you have any questions or feedback.

Thanks for reading!

References and useful links

Enjoyed that? Read some other posts.