Extract certificates from Java Key Stores for use by CURL

I recently found myself working with a Tomcat-based web application that required its clients to present a certificate to authenticate themselves. Being Tomcat, the whole thing was put together using Java of course; if you wanted to make a call to the server, you had to include a reference to a Java Key Store (.jks) file. One of my coworkers made a good case for using CURL for automated testing — unfortunately, CURL doesn't understand the .jks format. Well, as it turns out, you can extract the key and certificate information from a Java Key Store for use by another client application, but the process is a bit involved, so I thought I'd document it here in case anybody else finds themselves in a similar situation.

Java Trust Stores and Key Stores

To motivate this example, I'll walk through the setup of a simple Java and JKS-based client and server that perform mutual SSL authentication — although I originally ran into this in the context of a Tomcat server, I think this is easier to see without all the extra Tomcat stuff. If you know what keystores are and already have something working, and you just want to see how to extract a certificate for use by CURL, feel free to skip down to the extraction part. Listing 1, below, is the simplest server application I can think of; it accepts a connection on port 1234 and echoes back every character it receives. It doesn't deal properly with multiple concurrent clients, or handle errors, or make itself configurable, or offer and security or authentication, but since the topic of this post is simply connectivity, it'll do well enough.

import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;

public class Server {
  public static void main(String[] args) throws IOException {
    ServerSocket listen = new ServerSocket(1234);

    Socket connection = null;
    while ((connection = listen.accept()) != null)  {
      InputStream in = connection.getInputStream();
      OutputStream out = connection.getOutputStream();

      int c;
      while ((c = in.read()) != -1) {
        out.write(c);
      }
			out.close();
			in.close();
			connection.close();
    }
  }
}

Listing 1: Very simple Java "server"

If you compile and run this, it will sit and wait for connections on port 1234. You can test it without a special client by just using the telnet utility as shown in example 1:

$ telnet localhost 1234
Trying ::1...
Connected to localhost.
Escape character is '^]'.
abc
abc
def
def
^]
telnet> quit
Connection closed.

Example 1: test using telnet

Now, of course, this connection is plaintext. We want to encrypt it; that's what SSL is for. Java has had built-in support for SSL since JDK 1.3 — of course, being SSL, it requires a bit of setup. The code changes, shown in listing 2, are relatively straightforward, requiring only a change to the server socket constructor:

import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;

import javax.net.ServerSocketFactory;
import javax.net.ssl.SSLServerSocketFactory;


public class SSLServer  {
  public static void main(String[] args) throws IOException {

    ServerSocketFactory fact = SSLServerSocketFactory.getDefault();
    ServerSocket listen = fact.createServerSocket(1234);


    Socket connection = null;
    while ((connection = listen.accept()) != null)  {
      InputStream in = connection.getInputStream();
      OutputStream out = connection.getOutputStream();

      int c;
      while ((c = in.read()) != -1) {
        out.write(c);
      }
			out.close();
			in.close();
			connection.close();
    }
  }
}

Listing 2: SSLServerSocket

Now, if you try to connect and test using telnet, you'll be kicked out as soon as you try to type a character, and the server will respond with an error message as shown in example 2:

$ java -classpath . SSLServer
Exception in thread "main" javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
	at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671)
	at sun.security.ssl.InputRecord.read(InputRecord.java:504)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
	at SSLServer.main(SSLServer.java:20)

Example 2: unrecognized SSL message

The server was expecting an SSL handshake, but didn't get one; it aborts the connection immediately. Since it throws an uncaught exception in this case, the whole server application terminates (I told you it didn't handle errors well). If you had openssl installed, you could use the s_client subprogram to try to connect: openssl s_client -connect localhost:1234. However, since I'll need it later, I'll show you how to develop a Java equivalent; the SSL client in listing 3.

import java.net.Socket;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import javax.net.ssl.SSLSocketFactory;
import javax.net.SocketFactory;

public class SSLClient  {
  public static void main(String[] args) throws IOException {
    SocketFactory fact = SSLSocketFactory.getDefault();
    Socket conn = fact.createSocket("localhost", 1234);
    OutputStream out = conn.getOutputStream();
    InputStream in = conn.getInputStream();
    out.write("abc".getBytes());
    int c;
    while ((c = in.read()) != -1) {
      System.out.print((char) c);
    }
    in.close();
    out.close();
    conn.close();
  }
}

Listing 3: SSL Server Client

However, whether you use openssl s_client or the Java SSL client of listing 3, you'll still get a failure on the server when you try to connect to it, as shown in example 3:

$ java -classpath . SSLServer
Exception in thread "main" javax.net.ssl.SSLHandshakeException: no cipher suites in common
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:266)
	at sun.security.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:894)
	at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:622)
	at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:167)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:878)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:814)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
	at SSLServer.main(SSLServer.java:20)

Example 3: No cipher suites in common

What the Java runtime is trying to tell you, in a very oblique way, is that the server is misconfigured; in particular, it's not presenting a certificate to the client. The server has to identify itself — this is how SSL prevents man in the middle attacks. You can set up a test server by create a self-signed certificate using Java's keytool as shown in example 4:

$ keytool -genkeypair -keystore server.jks -storepass password -alias server -keypass password
What is your first and last name?
[Unknown]:  localhost
What is the name of your organizational unit?
[Unknown]:  Blog
What is the name of your organization?
[Unknown]:  commandlinefanatic
What is the name of your City or Locality?
[Unknown]:  Dallas
What is the name of your State or Province?
[Unknown]:  TX
What is the two-letter country code for this unit?
[Unknown]:  US
Is CN=Joshua Davies, OU=Blog, O=commandlinefanatic, L=Dallas, ST=TX, C=US correct?
[no]:  yes

Example 4: creating a new, self-signed certificate

This creates a new file named "server.jks" that contains one self-signed certificate entry. You can verify this from the command line as shown in example 5.

$ keytool -list -keystore server.jks
Enter keystore password:  

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

server, Jul 28, 2015, PrivateKeyEntry, 
Certificate fingerprint (SHA1): 1A:E9:6D:DA:A0:21:AD:A3:0A:27:A1:02:F0:7B:30:4E:ED:EB:29:51

Example 5: list contents of keystore

If you're following along, the answers to the prompts don't matter, except for the answer to the first question, "what is your first and last name?" Since this will be used as a server certificate, you must answer with "localhost", or the DNS name of the server that this will be accessed through. Although the SSLClient won't care, this will be important later, when I start using CURL to access this server.

Now, to instruct the server to present this certificate on connection, you provide the path to the keystore and the password on the command line:

java -Djavax.net.ssl.keyStore=server.jks -Djavax.net.ssl.keyStorePassword=password SSLServer

This goes a little better when you run the client — at least, you'll get a different error message as shown in example 6:

$ java -classpath . SSLClient
Exception in thread "main" javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1439)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:878)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:814)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
	at java.io.OutputStream.write(OutputStream.java:75)
	at SSLClient.main(SSLClient.java:14)

Example 6: Untrusted server certiticate

What the client is saying here is that it found the certificate, but it didn't trust it, since it wasn't signed by a trusted certificate authority. That shouldn't come as a surprise, since the certificate was self-signed — clearly not a certificate authority that the client could have trusted. You can force it to trust this self-signed certificate by overriding its "trust store" when you run the client:

$ java -Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePassword=password SSLClient

And now, the client and the server will connect with each other.

Note that this is still "wrong", although it does work. Giving the client access to the server's full key store defeats the purpose of creating the keystore in the first place. Normally, you'd export just the certificate and distribute it. I'll show you how to do that later.

At this point, the client trusts the server because it recognizes the signer of the server's certificate. The server trusts the client because, by default, SSL servers trust any client that tries to connect to it. You can change this behavior by requiring the client to present a certificate as well:

ServerSocket listen = fact.createServerSocket(1234);
((SSLServerSocket)listen).setNeedClientAuth(true);

Socket connection = null;

Now, if you try to run the client against the server again, you'll get a failure as you'd expect:

$ java -Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePasswd=password SSLClient
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
	at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1959)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1077)
	at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1705)

Now, in order to get this to run, you'll need to generate another certificate and configure the client to present it and the server to trust it.

Technically, this would work if the client and the server both trusted and presented the same certificate (e.g. server.jks), but that makes it hard to see exactly what's going on — and definitely not something that would make sense in any sort of real-world scenario.

$ keytool -genkeypair -keystore client.jks -storepass password -alias client -keypass password
What is your first and last name?
  [Unknown]:  Joshua Davies
What is the name of your organizational unit?
  [Unknown]:  Blog
What is the name of your organization?
  [Unknown]:  commandlinefanatic
What is the name of your City or Locality?
  [Unknown]:  Dallas
What is the name of your State or Province?
  [Unknown]:  TX
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Joshua Davies, OU=Blog, O=commandlinefanatic, L=Dallas, ST=TX, C=US correct?
  [no]:  yes

$ java -Djavax.net.ssl.trustStore=client.jks -Djavax.net.ssl.trustStorePassword=password \
-Djavax.net.ssl.keyStore=server.jks -Djavax.net.ssl.keyStorePassword=password SSLServer

$ java -Djavax.net.ssl.keyStore=client.jks -Djavax.net.ssl.keyStorePassword=password \
-Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePassword=password SSLClient
abc

Notice how I've inverted the trust store and key store between the server and the client in this case.

Before I move on, I'll make one last minor change to the SSLServer in listing 4 — rather than having it echo back whatever it was presented, I'll have it act as a semi-valid HTTP server by responding to every request with a canned HTML page:

OutputStream out = connection.getOutputStream();


int c[] = new int[4];

while ((c[3] = in.read()) != -1)  {
  if (Arrays.equals(c, new int[] {'\r', '\n', '\r', '\n'})) {
    break;
  }
  for (int i = 0; i < 3; i++) {
    c[i] = c[i + 1];
  }
}

System.out.println("Sending response now");

out.write("HTTP/1.1 200 OK\r\n".getBytes());
out.write("Connection: Close\r\n".getBytes());
out.write("Content-Type: text/html\r\n".getBytes());
out.write("Content-Length: 47\r\n\r\n".getBytes());
out.write("<html><body><h1>Nothing here</h1></body></html>".getBytes());
out.close();

connection.close();

Listing 4: Return a valid HTTP response

Extracting a certificate for use by CURL

So here, with the server presenting a valid certificate and requiring one from its clients, is where I found myself initially — I have a keystore that I can use for Java-based clients, but I'd like to connect and test using CURL. If I try, I'll get an error because the server isn't presenting a certificate signed by a certificate authority trusted by CURL:

$ curl -XGET https://localhost:1234/index.html
curl: (60) SSL certificate problem: self signed certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.

I can't just point CURL to my .jks file; CURL doesn't know anything about .jks files. What I want to do is extract the self-signed certificate out of the .jks file in a format recognized by CURL for connection.

As it turns out, Sun anticipated this need and made it pretty straightforward using Java's keytool:

$ keytool -exportcert -rfc -keystore server.jks -storepass password -alias server > server.pem

The -rfc option outputs the certificate in the Privacy Enhanced Mail format (instead of the default Distinguished Encoding Rules format) that CURL can consume:

$ curl --cacert server.pem https://localhost:1234/index.html
curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

Now, although the client is now accepting the server's certificate, the server is challenging the client (CURL) to present one. Extracting this from the .jks file is just a bit harder. You can export the client certificate just as you did the server certificate and try to connect using it:

$ curl --cacert server.pem --cert client.pem https://localhost:1234/index.html
curl: (58) unable to set private key file: 'client.pem' type PEM

However, this isn't enough to complete a handshake; remember that the keytool creates a keypair — a public key and a private key. The public key is embedded in the certificate but, for obvious reasons, the private key has to be kept separate. keytool, unfortunately, doesn't have an "exportkey" option. However, it does support multiple keystore types; it isn't just limited to JKS (Java Key Store), although that's the default. What you can do instead, is you can convert the whole keystore from JKS format to PKCS #12 (sometimes called pfx):

$ keytool -importkeystore -srckeystore client.jks -destkeystore client.pfx -deststoretype PKCS12 -srcalias client -deststorepass password -destkeypass password
Enter source keystore password:

This makes a full copy of the client.jks keystore — the important thing here is that the copy is specified as being PKCS #12, a more general keystore format, and one that CURL knows how to take advantage of. However, by default, the file is encrypted, and not in a way that CURL can decrypt, so you'll need to re-export the keystore unencrypted. You can't use keytool to do this, but the openssl command line has an option to permit this:

$ openssl pkcs12 -in client.pfx -out client.p12 -nodes
Enter Import Password:
MAC verified OK

And now you can connect to the server, using the required credentials:

$ curl --cacert server.pem --cert client.p12 https://localhost:1234/index.html
<html><body><h1>Nothing here</h1></body></html>

Add a comment:

Completely off-topic or spam comments will be removed at the discretion of the moderator.

You may preserve formatting (e.g. a code sample) by indenting with four spaces preceding the formatted line(s)

Name: Name is required
Email (will not be displayed publicly):
Comment:
Comment is required
Jagan, 2015-07-30
Nice post!
Sunny, 2016-01-21
well explained nice article.
Furquan Ahmed, 2016-04-04
Just what I was looking for. Thank you very much. Keep up the great work.
Shrey, 2016-09-14
Nice article. The java server and java client worked but when I follow the steps to execute it with curl I get the following error:

* Trying ::1...
* connect to ::1 port 8082 failed: Connection refused
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8082 (#0)
* WARNING: SSL: Certificate type not set, assuming PKCS#12 format.
* SSL: Couldn't make sense of the data in the certificate "client.p12" and its private key.
* Closing connection 0

Any suggestion would be appreciated.
Josh, 2016-10-03
Hard to say without seeing the keystore, but most likely the conversion from JKS to PKCS #12 failed. Did you get any error output when you ran the "$ openssl pkcs12 -in client.pfx -out client.p12 -nodes" step?
Venkat, 2017-02-08
Very nice post.. examples are so simple to understand and to the point.
amcc, 2017-03-10
Hi Joshua,
This is a very good post
I have a question, I have a struststore.jks and a keystore.jks that where provided to me as well as the https url of the web service. I would like to access the web service using curl. I tried to follow your steps to export the certificate and/or import the keystore and I do not know how to use the -alias or the -srcalias options.
i get the following error: (this is on linux)

keytool error: java.lang.Exception: Alias <client> does not exist
I would appreciate your help.

thanks,
amcc
Josh, 2017-03-10
Every entry in a java key store has an alias (the default is "mykey"). You can see a list of the aliases in any keystore with:
    $ keytool -list -keystore <path/to/keystore>
Most likely, in your case, there will only be one entry. If not, go ahead and just paste the contents of the output here and I should be able to tell you which one you want to extract.
amcc, 2017-03-10
Hi Josh, this is the output of my $ keytool -list -keystore ./keystore.jks

nprod, Apr 9, 2015, PrivateKeyEntry,
Certificate fingerprint (SHA1): A1:29:01:0C:8B:6A:AF:EC:13:31:14:4C:C3:55:EB:5A:26:CA:1D:1C
a-issuing-ca (aero-root-ca), Apr 13, 2015, trustedCertEntry,
Certificate fingerprint (SHA1): 69:FC:D2:83:F2:13:72:F4:04:23:81:9D:AD:91:EE:F8:42:F0:08:9C
a-root-ca, Apr 13, 2015, trustedCertEntry,
Certificate fingerprint (SHA1): 0D:8B:73:A0:8B:72:13:03:AB:EA:82:B5:69:5E:B5:9C:84:A3:D4:9B

please let me know which column or entry is the alias.
Thanks for your help.
amcc
My Book

I'm the author of the book "Implementing SSL/TLS Using Cryptography and PKI". Like the title says, this is a from-the-ground-up examination of the SSL protocol that provides security, integrity and privacy to most application-level internet protocols, most notably HTTP. I include the source code to a complete working SSL implementation, including the most popular cryptographic algorithms (DES, 3DES, RC4, AES, RSA, DSA, Diffie-Hellman, HMAC, MD5, SHA-1, SHA-256, and ECC), and show how they all fit together to provide transport-layer security.

My Picture

Joshua Davies

Past Posts