How to Rearrange a Certificate Chain using OpenSSL

On one of my recent Exchange migration projects I ran into an issue after installing a certificate on a Network Load Balancing device and it took some Scooby Dooing to get it to install properly, so I thought I’d share how we resolved it.  The issue was that the NLB device was not installing the Certificate chain in the correct order and it was causing issues with any device that would not reorder the chain correctly, mostly Android devices.  The symptom was that any Android devices that had “Accept All SSL Certificates” unchecked were getting certificate errors.  Since my goal during a migration is to have little to no impact to the end-users, this was a problem for me.

Little background…

If you’re having this issue and don’t care about the details of my problem, feel free to skip this part. 

A certificate that you install for Exchange is usually part of a certificate chain:  private key, server certificate, intermediates, and a root.  You might not notice it because when you purchase a certificate they are all stored in a single .CER file, and when you install the .CER onto the device that generated the CSR, it installs all the certificates for you.  Some devices are not intelligent enough to install the certificate chain in the correct order (Some NLB devices for example) and this causes problems for devices that don’t rearrange the certificate chain in the correct order (Androids for example).

In my particular issue the SSL Certificate was purchased from DigiCert and installed on an Exchange 2010 server, then exported to a .PFX file, and then installed onto a NLB device.  The certificates in the .PFX file were not in the correct order and the NLB device did not arrange them in the correct order when the certificates were installed.  I know you’re wondering, “Jerrid, how did you know they were in the wrong order?”  Well, I’m glad you’ve asked…  We got help desk calls during our pilot and we cheated.  We cheated with an online tool provided by DigiCert (http://digicert.com/help).  This tool will quickly tell you if your certificate is installed in the correct order.  When we tested the OWA URL for this particular customer, it was broke.  When I say broke, it was only broke for those devices that don’t rearrange a certificate chain in the correct order if the certificate chain is not right on the NLB devices.  Other devices and IE didn’t have an issue.

How did we fix it?  Well, we had to get the chain into a .PFX format and in the correct order before we installed it on the NLB device.  The correct order is:  Private Key à Server Certificate à Intermediates à Root Certificate.  To do this, we used OpenSSL.

How to fix it!

From this point forward it is assumed that you have a valid certificate on a Windows 2008 box or you have a .PFX file that contains all required certificates.  If you don’t, then I would suggest calling support from the company you purchased your certificate from to get to this point.

High Level Steps:

1. Get all Certificates in a .PFX file format
2. Convert .PFX file to a .PEM file format
3. Move stuff around until it looks right
4. Convert new .PEM file back to .PFX format
5. Install .PFX file on NLB device
6. Test
7. Celebrate!

Get all Certificates in a .PFX format

If you don’t already have a .PFX file with all the certificates, this step assumes you have all the certificates installed on a Windows box.

1. Log onto the server that has the certificates installed.
2. Click Start à Run and type in MMC.
3. In the blank MMC, click File à Add/Remove Snap-ins..
4. Select Certificates à Add >
5. I’M PUTTING THIS IN CAPS BECAUSE IT’S IMPORTANT!!  Make sure you select Computer Account.  Click Next, select Local, then click Finish, then OK.  This loads the Certificate Snap-in for the local computer.
6. Under Certificates, expand Personal and click Certificates.  In the detail pain, you should see your certificate and it should have a little key icon on it.  If it doesn’t, your certificate is not installed correctly and you should contact your public certificate provider to get it installed correctly.  Right click it, and select All Tasks à Export…
7. For the Export Wizard
 
 
 
1. Select Yes, export  the private key.
1. If you don’t have this option, then whoever you purchased the key from does not allow you to install the certificate on multiple servers, or you didn’t allow the key to be exportable when you installed the certificate
2. In the Export File Format, select Include all certificates in the certificate path if possible and Export all extended  properties.
3. Secure the .PFX file with a password.  Don’t skimp on this.  Use something difficult that you’ll remember.  If someone gets this file and guesses your password, your certificate security is toast because they’ll have access to your private key and can impersonate you.
4. Enter a file name.  For this example we are going to save the cert in C:\Cert\temp.pfx

Convert .PFX to .PEM format

Well done folks, we now have all the certificates needed in a .PFX file.  Now we need to arrange it correctly and this is where OpenSSL comes into play.  Get it from here (http://slproweb.com/products/Win32OpenSSL.html) and install it.  For this example I downloaded Win64 OpenSSL v1.0.1c Light.  You will get a warning about Visual C++ not being installed.  You can ignore this and continue.  I also chose to put the DLL files in the Bin directory instead of the System32 directory.

After the install, open a command prompt and change to the bin directory if you chose that option so that your command prompt focus is on the directory where openssl.exe is installed.

Next, we need to run the following command:

openssl pkcs12 -in c:\cert\temp.pfx -out c:\cert\temp.pem –nodes

 

Provide the password you specified when you exported the certificate, and now you should have a nice unencrypted PEM file.  WARNING!!!  Your key is now in a text file unencrypted.  Protect it!  Protect it like it’s a ring forged in the pits of Mount Doom and you’re a hobbit in the Shire.  “Keep it secret…Keep it safe…”  Middle-Earth depends on it.

NOTE:  If you get this error message:  WARNING: can't open config file: /usr/local/ssl/openssl.cnf.  Then you need to run the command prompt as an Administrator.

Move Stuff Around Until it Looks Right

Now it’s time to rearrange some stuff.  Open your new .PEM file with Wordpad.  Notepad is no good here, so stick with Wordpad.

Once you have it open, don’t freak out.   You don’t have to be Neo to figure this out, but you do need to know the correct certificate order.  The first two are easy, the key should be first and the Server Certificate should be second.  Generally the third certificate will be an intermediate and the last will be a root.  If you look at each section, you’ll see a -------Begin Certificate------ and -------End Certificate------ section preceded by a header.  In the header you’ll see what certificate is what.  For example, for the key you’ll see this:

 

You’ll use these headers to identify the certificates and put them in the correct order.  DigiCert was a little tricky because what they call their root certificate is not the root in the certificate chain because their root is signed by Entrust’s root.  This caused the order to be:  Key à Server Cert à Digicert High Assurance CA à Digicert High Assurance EV Root CA à Entrust Root.  Again, if you’re unsure of the order, contact support at the company you purchased your certificate from.

Once you know the correct order, simply cut and paste the header and certificate (crap between and including the ----Begin Certificate----- and ------End Certificate-----) in the correct order. 

Convert new .PEM back to .PFX file

openssl pkcs12 -export -in c:\cert\temp.pem -out c:\cert\newCert.p12 -name "Exchange Certificate"

 

Ok, we’re almost there!  We now have a certificate with the correct order in a .P12 format, just rename it to .PFX to get our .PFX file.  I would suggest deleting the .PEM file to protect your key.

Install, Test, Celebrate

Now install the .PFX file and test.  To test, you can use http://digicert.com/help to make sure the chain is complete and you can test whatever device you were having issues with.

At this point, if the testing works out then it’s time to celebrate.  I would like to put a Monty Python quote here about great rejoicing, but I’ve already used a Matrix and Lord of the Rings quotes, so I’ll just end with, “That’ll do pig….  That’ll do…”