I have a spring boot application wherein a Dockerfile executes a "keytool -importkeystore …" command to import a p12 keystore. The reason the keytool command is executed in the entrypoint.sh is to give the ability to parameterize the setting of the certificate itself so it doesn’t have to be bundled with the docker image of the app.
Here is the Dockerfile
FROM openjdk:8-jdk-alpine as builder
WORKDIR application
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=layertools -jar application.jar extract
FROM openjdk:8-jdk-alpine
WORKDIR application
COPY --from=builder application/dependencies/ ./
COPY --from=builder application/spring-boot-loader/ ./
COPY --from=builder application/snapshot-dependencies/ ./
COPY --from=builder application/application/ ./
COPY entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/entrypoint.sh
# Install CURL
RUN apk --no-cache add curl
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
Here is the entrypoint.sh that is executed when docker launches container
#!/bin/sh
#echo "executing keytool import store"
echo "statement is ${STATEMENT}"
keytool ${STATEMENT}
echo "Executing springboot load for app"
#Execute using linux 'exec' Java launcher
exec java org.springframework.boot.loader.JarLauncher "$@"
Using docker-compose referencing a custom .env, the variable STATEMENT referenced in .sh file above, paired with keytool command gives:
-v -importkeystore -srckeystore /etc/certs/ssl/appc.com.p12 -srcstorepass Appcpassword1234 -destkeystore /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/cacerts -srcstoretype pkcs12 -deststoretype JKS -deststorepass changeit
Please note, relative to printed command above:
- p12 keystore file = appc.com.p12
- keystore password = Appcpassword1234
In same .env file, I am using spring boot app properties to set the SSL variables and the process does respond to them:
SERVER_SSL_KEYALIAS=appc-alias
SERVER_SSL_KEYSTORE=file:/etc/certs/ssl/appc.com.p12 #tried different formats as well like removing the "file:"
SERVER_SSL_KEYPASSWORD=Appcpassword1234
SERVER_SSL_KEYSTORE_PASSWORD=Appcpassword1234
When running the container in my windows-system via docker desktop, all is well and the application starts up just fine. When running in centOs with docker engine, the app consistently fails with below error stack trace. I am in control of creating all passwords (except perhaps for the cacerts which I think is changeit by default). However, it seems to want a different one than what I created and set on the p12. I’m utterly confused.
Could the password I set be encoded in one way, and then need to be supplied in the keytool command in a different way ??
Caused by: org.apache.catalina.LifecycleException: Protocol handler start failed
at org.apache.catalina.connector.Connector.startInternal(Connector.java:1058) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.catalina.core.StandardService.addConnector(StandardService.java:227) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
... 26 common frames omitted
Caused by: java.lang.IllegalArgumentException: keystore password was incorrect
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:99) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:71) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:216) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1141) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1227) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:592) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
at org.apache.catalina.connector.Connector.startInternal(Connector.java:1055) ~[tomcat-embed-core-9.0.35.jar:9.0.35]
... 28 common frames omitted
Caused by: java.io.IOException: keystore password was incorrect
at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2059) ~[na:1.8.0_212]
at java.security.KeyStore.load(KeyStore.java:1445) ~[na:1.8.0_212]
- I attempted to use different regenerated P12 with more complex alphanumeric and/or special character passwords – no luck, although I found out that the password needed to be longer than 11 characters.
- After several hours of googling, I tried setting different flags in keytool command (-keypass, -storepass). Adding them results in this command:
-v -importkeystore -srckeystore /etc/certs/ssl/appc.com.p12 -srcstorepass Appcpassword1234 -keypass Appcpassword1234 -destkeystore /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/cacerts -srcstoretype PKCS12 -storepass changeit -J-showversion
The flag -J-showversion simply prints version details, which, in this case is:
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (IcedTea 3.12.0) (Alpine 8.212.04-r0)
OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)
- In response to warning [1] I see printed, when I add the flag -destoretype=PKCS12 to command, I get the following error so I leave it as JKS or omit the flag altogether
keytool error: java.io.IOException: DerInputStream.getLength(): lengthTag=109, too big.
java.io.IOException: DerInputStream.getLength(): lengthTag=109, too big.
at sun.security.util.DerInputStream.getLength(DerInputStream.java:599)
at sun.security.util.DerValue.init(DerValue.java:391)
at sun.security.util.DerValue.<init>(DerValue.java:332)
-
I’ve also created P12 using OPENSSL (instead of keytool) per this thread: Java keytool : Importing PKCS12 to jks , getting error keystore password was incorrect. The result is the same.
-
I have NOT tried to use the Oracle JDK 8 as suggested here: "java.io.IOException: keystore password was incorrect" on KeyStore load. If there is an equivalent version of OpenJDK that I could try, I’d prefer that rather than going with Oracle’s (license/open-source needs)
Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/cacerts -destkeystore /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/cacerts -deststoretype pkcs12".
2
Answers
I got it to work but this may be more of a story about spring boot properties being converted into environment variables than intricacies of 'keytool', openssl, and TLS certs in general.
I still can't quite explain this but the combination of these 2 changes in the .env file referenced in my docker-compose did the trick - it allowed all services to start up without the "keytool error" and with TLS encryption:
What I still don't understand and could use some insights:
I realize that this is probably after the fact, but I came here because I was experiencing a very similar issue and this made me double check your environment variables as I am using the same one for password. I believe the issue is indeed as you suspected with your conversion from properties to environment variable names. I believe that "SERVER_SSL_KEYSTORE_PASSWORD" needs to be changed to "SERVER_SSL_KEYSTOREPASSWORD" according to the rules listed at https://docs.spring.io/spring-boot/docs/2.7.x/reference/html/features.html#features.external-config.typesafe-configuration-properties.relaxed-binding.environment-variables