Difference between revisions of "FAQ SSL/TLS Certificate Authority Root Stores"

From Overbyte
Jump to navigation Jump to search
(Creating FAQ SSL/TLS Certificate Authority Root Stores)
 
 
(20 intermediate revisions by the same user not shown)
Line 32: Line 32:
 
But it's not easy to create root bundles from CCADB and another developer
 
But it's not easy to create root bundles from CCADB and another developer
 
got frustrated with updating roots, and created a Trust Stores Observatory
 
got frustrated with updating roots, and created a Trust Stores Observatory
Git repository:
+
Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 680 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM and PKCS7/12 bundle files, one each for the different operating systems.
 
 
https://github.com/nabla-c0d3/trust_stores_observatory
 
 
 
which contains over 500 root certificates and lists of which trust store
 
contain which roots by different operating systems. But even this does
 
not contain certificates in a form easily used by OpenSSL, so Magenta
 
Systems Ltd has written a small tool that converts the YAML files from
 
TSO into PEM bundle files, one each for the different operating systems.
 
 
 
  
 
== New PEM Bundle CA Trusted Store Files ==  
 
== New PEM Bundle CA Trusted Store Files ==  
  
 
There are six different PEM CA bundle files, built from the Trust Stores
 
There are six different PEM CA bundle files, built from the Trust Stores
Observatory Git repository:
+
Observatory Git repository in May 2025:
  
  apple.pem - 174 Certificates
+
  apple.pem - 153 Certificates
  google_aosp.pem - 137 Certificates
+
  google_aosp.pem - 145 Certificates
  microsoft_windows.pem - 289 Certificates
+
  microsoft_windows.pem - 310 Certificates
  mozilla_nss.pem - 137 Certificates
+
  mozilla_nss.pem - 143 Certificates
  openjdk.pem - 88 Certificates
+
  openjdk.pem - 89 Certificates
  oracle_java.pem - 88 Certificates
+
  oracle_java.pem - 93 Certificates
  
Each certificate is prefixed by it's description, issuer fields, expiry,
+
Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store.  Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers.
public key type and SHA256 hash, so the bundles are self documenting
+
 
rather than being just cryptic base64 blocks. These PEM bundles may be
+
The zip file contains three versions of each bundle, the name above, one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters, and a third PKCS7/12 version with extension P12 which is smaller than PEMs.  There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file.  There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update.  
loaded into an OpenSSL context as a root store.
 
  
 
These bundles may be downloaded at:
 
These bundles may be downloaded at:
Line 66: Line 56:
 
Magenta Systems Ltd will periodically update these bundles, as needed.
 
Magenta Systems Ltd will periodically update these bundles, as needed.
  
 +
== ICS CA Trusted Stores ==
  
== ICS CA Trusted Stores ==
+
ICS includes three CA Trusted Stores, as resource files that can be linked in applications,
 +
or PEM bundle files.  The OverbyteIcsDef.inc file should include only one of these defines
 +
depending on which bundle you want built into applications:
  
ICS includes three CA CA Trusted Store, two as PEM bundle files, one in
+
{$DEFINE OpenSSL_CA_Bundle_Small}
a source unit, and access to the Window Certificate Store directly:
+
{$DEFINE OpenSSL_CA_Bundle_Medium}
 +
{$DEFINE OpenSSL_CA_Bundle_Large}
  
 
1 - RootCaCertsBundle.pem is a large file that was originally created
 
1 - RootCaCertsBundle.pem is a large file that was originally created
15 years ago by exporting the Windows certificate store using the
+
20 years ago by exporting the Windows certificate store using the
OverbyteIcsPemTool sample. But Windows 10 no longer has a complete local
+
OverbyteIcsPemTool sample. But Windows no longer has a complete local
 
certificate store and instead downloads new certificates as needed by
 
certificate store and instead downloads new certificates as needed by
Windows browsers.  So with ICS V8.63, it is now the same as the new
+
Windows browsers.  So with ICS V8.63 and later, it is now the same as the new
microsoft_windows.pem bundle mentioned above.  It currently contains 289
+
microsoft_windows.pem bundle mentioned above.  It currently contains 310
certificates and is 595 Kbytes in size and may be found in the
+
certificates and is 621 Kbytes in size for the PEM version.
Samples/Delphi/SslInternet/ directory.
 
  
 
2 - TrustedCABundle.pem is a smaller file, with certificate for major
 
2 - TrustedCABundle.pem is a smaller file, with certificate for major
 
commercial issuers manually updated as newer sites are found to have
 
commercial issuers manually updated as newer sites are found to have
 
missing root certificates.  But this file is more dynamic than
 
missing root certificates.  But this file is more dynamic than
RootCaCertsBundle.pem.  It currently contains 51 certificates and is
+
RootCaCertsBundle.pem.  It currently contains 117 certificates and is
88 Kbytes in size and may be found in the Samples/Delphi/SslInternet/
+
215 Kbytes in size for the PEM version.
directory.
 
  
3 - To avoid distributing bundle files and as a fail safe if a file can
+
3 - sslRootCACertsBundle.pem is the smallest bundle with just certificates
not be found, ICS includes 34 built-in hard coded certificates in
+
for some major commercial issuers manually updated as newer sites are found to have
OverbyteIcsSslX509Utils.pas which can be returned as a string by the
+
missing root certificates. It currently contains 83certificates and is
function sslRootCACertsBundle. Again this unit may be dynamic with new
+
178 Kbytes in size for the PEM versionOriginally, this bundle was created
certificates added as neededNote only the TSslHttpRest, TIcsIpStrmLog,
+
as constants in code, but is now a resource similar to the other bundles.  
TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components use the built-in
 
bundle by default, other components need to add it manually to avoid the
 
extra program code involved.
 
  
 
4 - ICS also includes a component TMsCertChainEngine in the unit
 
4 - ICS also includes a component TMsCertChainEngine in the unit
Line 110: Line 99:
 
contacting each issuer and can slow down chain validation by a few seconds
 
contacting each issuer and can slow down chain validation by a few seconds
 
or even longer.
 
or even longer.
 
Either of the certificate bundle files may be loaded into an SslContext
 
by using the SslCAFile property.  The built in bundle may be specified
 
before the SslContext is initialised using SslCALines.Text property, or
 
the LoadCAFromString method after initialisation.
 
  
 
If checking a certificate chain, OpenSSL will issue the error message
 
If checking a certificate chain, OpenSSL will issue the error message
 
'unable to get local issuer certificate' if a trusted certificate is not
 
'unable to get local issuer certificate' if a trusted certificate is not
 
found in the store.
 
found in the store.
 +
 +
With ICS version V9.1 and later, the bundles files were moved to a new common directory
 +
'C:\ProgramData\ICS-OpenSSL\ICS-Certs\' and are available as three resource files
 +
RootCaCertsBundle.RES, TrustedCaBundle.RES and sslRootCACertsBundle.RES.
 +
 +
A common IcsSslRootCAStore component is created at start-up,
 +
optionally loading OpenSSL and the root CA bundle linked as a resource. 
 +
 +
The contents of the three certificates bundles are listed at [[FAQ_ICS_SSL/TLS_CA_Trusted_Store_Contents | FAQ - ICS SSL/TLS CA Trusted Store Contents]]

Latest revision as of 17:27, 3 July 2025

CA Trusted Store Background

The whole SSL/TLS certificate verification process depends upon finding a trusted root certificate that signed the next certificate up the chain, and so on. But who chooses those trusted roots?

The answer is the author of the application that accepts an SSL/TLS certificate, or maybe the operating system the application is running on, or the SSL library used to build the application.

The bad news is that root certificates come and go, old roots stop being accepted for various reasons, and new roots are introduced for new security standards or new businesses.

Historically most roots were RSA Sha1 digest signed and most still are, despite Sha1 being banned for new certificates. Increasingly Sha256 and ECC root certificates are being introduced and required by new intermediate certificates. Typically, active certificate authorities have at least four roots, with 2,048 and 4,096 bit RSA keys and 256 and 394 bit ECDSA keys (which are much smaller than RSA).


Sources of CA Trusted Stores

SSL/TLS trusted root certificate bundles and always changing, annually perhaps for major changes, although Microsoft officially Windows roots every two months. The Common CA Database (CCADB) https://www.ccadb.org/ is a repository of information about Certificate Authorities (CAs), and is used by a number of different root store operators to manage their root stores.

But it's not easy to create root bundles from CCADB and another developer got frustrated with updating roots, and created a Trust Stores Observatory Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 680 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM and PKCS7/12 bundle files, one each for the different operating systems.

New PEM Bundle CA Trusted Store Files

There are six different PEM CA bundle files, built from the Trust Stores Observatory Git repository in May 2025:

apple.pem - 153 Certificates
google_aosp.pem - 145 Certificates
microsoft_windows.pem - 310 Certificates
mozilla_nss.pem - 143 Certificates
openjdk.pem - 89 Certificates
oracle_java.pem - 93 Certificates

Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store. Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers.

The zip file contains three versions of each bundle, the name above, one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters, and a third PKCS7/12 version with extension P12 which is smaller than PEMs. There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file. There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update.

These bundles may be downloaded at:

https://www.magsys.co.uk/download/software/ca-root-bundles.zip

Magenta Systems Ltd will periodically update these bundles, as needed.

ICS CA Trusted Stores

ICS includes three CA Trusted Stores, as resource files that can be linked in applications, or PEM bundle files. The OverbyteIcsDef.inc file should include only one of these defines depending on which bundle you want built into applications:

{$DEFINE OpenSSL_CA_Bundle_Small}
{$DEFINE OpenSSL_CA_Bundle_Medium}
{$DEFINE OpenSSL_CA_Bundle_Large}

1 - RootCaCertsBundle.pem is a large file that was originally created 20 years ago by exporting the Windows certificate store using the OverbyteIcsPemTool sample. But Windows no longer has a complete local certificate store and instead downloads new certificates as needed by Windows browsers. So with ICS V8.63 and later, it is now the same as the new microsoft_windows.pem bundle mentioned above. It currently contains 310 certificates and is 621 Kbytes in size for the PEM version.

2 - TrustedCABundle.pem is a smaller file, with certificate for major commercial issuers manually updated as newer sites are found to have missing root certificates. But this file is more dynamic than RootCaCertsBundle.pem. It currently contains 117 certificates and is 215 Kbytes in size for the PEM version.

3 - sslRootCACertsBundle.pem is the smallest bundle with just certificates for some major commercial issuers manually updated as newer sites are found to have missing root certificates. It currently contains 83certificates and is 178 Kbytes in size for the PEM version. Originally, this bundle was created as constants in code, but is now a resource similar to the other bundles.

4 - ICS also includes a component TMsCertChainEngine in the unit OverbyteIcsMsSslUtils.pas which allows applications to avoid using bundle files and instead access the Windows Certificate Store directly to validate certificates. There is a very slight overhead as the store is opened and Windows may need to download missing root certificates. Only the TSslHttpRest, TIcsIpStrmLog, TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components includes TMsCertChainEngine by default, with the CertVerMethod property selecting CertVerNone, CertVerBundle or CertVerWinStore. TMsCertChainEngine does include one extra optional feature to check if any certificates in the chain have been revoked by their issuer, perhaps for fraudulent use, beware revoke checks involve contacting each issuer and can slow down chain validation by a few seconds or even longer.

If checking a certificate chain, OpenSSL will issue the error message 'unable to get local issuer certificate' if a trusted certificate is not found in the store.

With ICS version V9.1 and later, the bundles files were moved to a new common directory 'C:\ProgramData\ICS-OpenSSL\ICS-Certs\' and are available as three resource files RootCaCertsBundle.RES, TrustedCaBundle.RES and sslRootCACertsBundle.RES.

A common IcsSslRootCAStore component is created at start-up, optionally loading OpenSSL and the root CA bundle linked as a resource.

The contents of the three certificates bundles are listed at FAQ - ICS SSL/TLS CA Trusted Store Contents