Difference between revisions of "FAQ SSL/TLS Certificate Authority Root Stores"
(15 intermediate revisions by the same user not shown) | |||
Line 32: | Line 32: | ||
But it's not easy to create root bundles from CCADB and another developer | But it's not easy to create root bundles from CCADB and another developer | ||
got frustrated with updating roots, and created a Trust Stores Observatory | got frustrated with updating roots, and created a Trust Stores Observatory | ||
− | Git repository: | + | Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 680 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM and PKCS7/12 bundle files, one each for the different operating systems. |
− | |||
− | https://github.com/nabla-c0d3/trust_stores_observatory | ||
− | |||
− | which contains over | ||
− | contain which roots by different operating systems. | ||
− | not contain certificates in a form easily used by OpenSSL, so Magenta | ||
− | Systems Ltd has written a small tool that converts the YAML files from | ||
− | TSO into PEM bundle files, one each for the different operating systems. | ||
− | |||
== New PEM Bundle CA Trusted Store Files == | == New PEM Bundle CA Trusted Store Files == | ||
There are six different PEM CA bundle files, built from the Trust Stores | There are six different PEM CA bundle files, built from the Trust Stores | ||
− | Observatory Git repository: | + | Observatory Git repository in February 2024: |
− | apple.pem - | + | apple.pem - 169 Certificates |
− | google_aosp.pem - | + | google_aosp.pem - 134 Certificates |
− | microsoft_windows.pem - | + | microsoft_windows.pem - 282 Certificates |
− | mozilla_nss.pem - | + | mozilla_nss.pem - 147 Certificates |
− | openjdk.pem - | + | openjdk.pem - 89 Certificates |
− | oracle_java.pem - | + | oracle_java.pem - 93 Certificates |
− | Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store. Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the | + | Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store. Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers. |
− | |||
− | |||
+ | The zip file contains three versions of each bundle, the name above, one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters, and a third PKCS7/12 version with extension P12 which is smaller than PEMs. There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file. There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update. | ||
These bundles may be downloaded at: | These bundles may be downloaded at: | ||
Line 68: | Line 58: | ||
== ICS CA Trusted Stores == | == ICS CA Trusted Stores == | ||
− | ICS includes three | + | ICS includes three CA Trusted Store, two as PEM bundle files, one in |
a source unit, and access to the Window Certificate Store directly: | a source unit, and access to the Window Certificate Store directly: | ||
Line 75: | Line 65: | ||
OverbyteIcsPemTool sample. But Windows 10 no longer has a complete local | OverbyteIcsPemTool sample. But Windows 10 no longer has a complete local | ||
certificate store and instead downloads new certificates as needed by | certificate store and instead downloads new certificates as needed by | ||
− | Windows browsers. So with ICS V8.63, it is now the same as the new | + | Windows browsers. So with ICS V8.63 and later, it is now the same as the new |
− | microsoft_windows.pem bundle mentioned above. It currently contains | + | microsoft_windows.pem bundle mentioned above. It currently contains 282 |
− | certificates and is | + | certificates and is 570 Kbytes in size for the PEM version. |
− | |||
2 - TrustedCABundle.pem is a smaller file, with certificate for major | 2 - TrustedCABundle.pem is a smaller file, with certificate for major | ||
commercial issuers manually updated as newer sites are found to have | commercial issuers manually updated as newer sites are found to have | ||
missing root certificates. But this file is more dynamic than | missing root certificates. But this file is more dynamic than | ||
− | RootCaCertsBundle.pem. It currently contains | + | RootCaCertsBundle.pem. It currently contains 109 certificates and is |
− | + | 201 Kbytes in size for the PEM version. | |
− | |||
3 - To avoid distributing bundle files and as a fail safe if a file can | 3 - To avoid distributing bundle files and as a fail safe if a file can | ||
− | not be found, ICS | + | not be found, ICS V8.32 to V9.0 include built-in hard coded certificates in |
OverbyteIcsSslX509Utils.pas which can be returned as a string by the | OverbyteIcsSslX509Utils.pas which can be returned as a string by the | ||
− | function sslRootCACertsBundle. | + | function sslRootCACertsBundle. Note only the TSslHttpRest, TIcsIpStrmLog, |
− | |||
TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components use the built-in | TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components use the built-in | ||
bundle by default, other components need to add it manually to avoid the | bundle by default, other components need to add it manually to avoid the | ||
− | extra program code involved. | + | extra program code involved. With V9.1, all the bundles can be optionally liniked |
+ | into applications as resource files and loaded automatically on start-up, | ||
+ | so this bundle is no longer needed. | ||
4 - ICS also includes a component TMsCertChainEngine in the unit | 4 - ICS also includes a component TMsCertChainEngine in the unit | ||
Line 110: | Line 99: | ||
or even longer. | or even longer. | ||
− | + | With ICS versions V9.0 and earlier, the certificate bundle files may be found in the | |
+ | Samples/Delphi/SslInternet/ directory and may be loaded into an SslContext | ||
by using the SslCAFile property. The built in bundle may be specified | by using the SslCAFile property. The built in bundle may be specified | ||
before the SslContext is initialised using SslCALines.Text property, or | before the SslContext is initialised using SslCALines.Text property, or | ||
Line 119: | Line 109: | ||
found in the store. | found in the store. | ||
− | The contents the three certificates bundles are listed at [[FAQ_ICS_SSL/TLS_CA_Trusted_Store_Contents | FAQ - ICS SSL/TLS CA Trusted Store Contents]] | + | With ICS version V9.1 and later, the bundles files have moved to a new common directory |
+ | 'C:\ProgramData\ICS-OpenSSL\ICS-Certs\' and are available as three resource files | ||
+ | RootCaCertsBundle.RES, TrustedCaBundle.RES and sslRootCACertsBundle.RES, one of | ||
+ | which may be linked into applications using {.$DEFINE OpenSSL_CA_Bundle_Large}, | ||
+ | {.$DEFINE OpenSSL_CA_Bundle_Medium} or {$DEFINE OpenSSL_CA_Bundle_Small} in the | ||
+ | OverbyteIcsDefs.inc file. A common IcsSslRootCAStore component is created at start-up, | ||
+ | optionally loading OpenSSL and the root CA bundle linked as a resource. | ||
+ | |||
+ | The contents of the three certificates bundles are listed at [[FAQ_ICS_SSL/TLS_CA_Trusted_Store_Contents | FAQ - ICS SSL/TLS CA Trusted Store Contents]] |
Latest revision as of 19:22, 16 February 2024
Contents
CA Trusted Store Background
The whole SSL/TLS certificate verification process depends upon finding a trusted root certificate that signed the next certificate up the chain, and so on. But who chooses those trusted roots?
The answer is the author of the application that accepts an SSL/TLS certificate, or maybe the operating system the application is running on, or the SSL library used to build the application.
The bad news is that root certificates come and go, old roots stop being accepted for various reasons, and new roots are introduced for new security standards or new businesses.
Historically most roots were RSA Sha1 digest signed and most still are, despite Sha1 being banned for new certificates. Increasingly Sha256 and ECC root certificates are being introduced and required by new intermediate certificates. Typically, active certificate authorities have at least four roots, with 2,048 and 4,096 bit RSA keys and 256 and 394 bit ECDSA keys (which are much smaller than RSA).
Sources of CA Trusted Stores
SSL/TLS trusted root certificate bundles and always changing, annually perhaps for major changes, although Microsoft officially Windows roots every two months. The Common CA Database (CCADB) https://www.ccadb.org/ is a repository of information about Certificate Authorities (CAs), and is used by a number of different root store operators to manage their root stores.
But it's not easy to create root bundles from CCADB and another developer got frustrated with updating roots, and created a Trust Stores Observatory Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 680 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM and PKCS7/12 bundle files, one each for the different operating systems.
New PEM Bundle CA Trusted Store Files
There are six different PEM CA bundle files, built from the Trust Stores Observatory Git repository in February 2024:
apple.pem - 169 Certificates google_aosp.pem - 134 Certificates microsoft_windows.pem - 282 Certificates mozilla_nss.pem - 147 Certificates openjdk.pem - 89 Certificates oracle_java.pem - 93 Certificates
Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store. Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers.
The zip file contains three versions of each bundle, the name above, one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters, and a third PKCS7/12 version with extension P12 which is smaller than PEMs. There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file. There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update.
These bundles may be downloaded at:
https://www.magsys.co.uk/download/software/ca-root-bundles.zip
Magenta Systems Ltd will periodically update these bundles, as needed.
ICS CA Trusted Stores
ICS includes three CA Trusted Store, two as PEM bundle files, one in a source unit, and access to the Window Certificate Store directly:
1 - RootCaCertsBundle.pem is a large file that was originally created 15 years ago by exporting the Windows certificate store using the OverbyteIcsPemTool sample. But Windows 10 no longer has a complete local certificate store and instead downloads new certificates as needed by Windows browsers. So with ICS V8.63 and later, it is now the same as the new microsoft_windows.pem bundle mentioned above. It currently contains 282 certificates and is 570 Kbytes in size for the PEM version.
2 - TrustedCABundle.pem is a smaller file, with certificate for major commercial issuers manually updated as newer sites are found to have missing root certificates. But this file is more dynamic than RootCaCertsBundle.pem. It currently contains 109 certificates and is 201 Kbytes in size for the PEM version.
3 - To avoid distributing bundle files and as a fail safe if a file can not be found, ICS V8.32 to V9.0 include built-in hard coded certificates in OverbyteIcsSslX509Utils.pas which can be returned as a string by the function sslRootCACertsBundle. Note only the TSslHttpRest, TIcsIpStrmLog, TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components use the built-in bundle by default, other components need to add it manually to avoid the extra program code involved. With V9.1, all the bundles can be optionally liniked into applications as resource files and loaded automatically on start-up, so this bundle is no longer needed.
4 - ICS also includes a component TMsCertChainEngine in the unit OverbyteIcsMsSslUtils.pas which allows applications to avoid using bundle files and instead access the Windows Certificate Store directly to validate certificates. There is a very slight overhead as the store is opened and Windows may need to download missing root certificates. Only the TSslHttpRest, TIcsIpStrmLog, TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components includes TMsCertChainEngine by default, with the CertVerMethod property selecting CertVerNone, CertVerBundle or CertVerWinStore. TMsCertChainEngine does include one extra optional feature to check if any certificates in the chain have been revoked by their issuer, perhaps for fraudulent use, beware revoke checks involve contacting each issuer and can slow down chain validation by a few seconds or even longer.
With ICS versions V9.0 and earlier, the certificate bundle files may be found in the Samples/Delphi/SslInternet/ directory and may be loaded into an SslContext by using the SslCAFile property. The built in bundle may be specified before the SslContext is initialised using SslCALines.Text property, or the LoadCAFromString method after initialisation.
If checking a certificate chain, OpenSSL will issue the error message 'unable to get local issuer certificate' if a trusted certificate is not found in the store.
With ICS version V9.1 and later, the bundles files have moved to a new common directory 'C:\ProgramData\ICS-OpenSSL\ICS-Certs\' and are available as three resource files RootCaCertsBundle.RES, TrustedCaBundle.RES and sslRootCACertsBundle.RES, one of which may be linked into applications using {.$DEFINE OpenSSL_CA_Bundle_Large}, {.$DEFINE OpenSSL_CA_Bundle_Medium} or {$DEFINE OpenSSL_CA_Bundle_Small} in the OverbyteIcsDefs.inc file. A common IcsSslRootCAStore component is created at start-up, optionally loading OpenSSL and the root CA bundle linked as a resource.
The contents of the three certificates bundles are listed at FAQ - ICS SSL/TLS CA Trusted Store Contents