ICS V8.65

Changes in ICS V8.65 include:

1. The ReadMe8.txt file has a new 'Getting Started with ICS' section listing the types of projects ICS may be used for, and suggesting the correct components to use, and their related sample applications for testing. This is recommended reading for anyone doing new ICS development since it discusses all the new high level components like TSslHttpRest added in the last few years which can reduce development effort considerably. It may also be viewed at Getting Started
2. Added new TIcsRestEmail component that provides basic support for Google and Microsoft Outlook email REST APIs including OAuth2 login and refresh to get an access token for SMTP and POP3 XOAuth2 and 0AuthBearer authentication. TIcsRestEmail has methods to send and read email, to list IDs in a mailbox, read headers and message bodies by ID, send emails and delete emails. The OverbyteIcsHttpRestTst1 sample shows how to prepare email using the SMTP component (Google) or Json (Microsoft) to be sent using the REST API. All responses are Json which the application needs to untangle.
3. The SMPT, POP3 and MailQueue samples all now support XOAuth2 and 0AuthBearer authentication using the TIcsRestEmail component. The low level component call an event to get the authentication access token, which is provided by IcsRestEmail, together with a refresh token which is saved instead of a password. Without a refresh token, the component will open a browser window to login to your Google or Microsoft account. The refresh token usually remains valid for many months.
4. To access email using REST APIs or OAuth2/SMTP/POP3 an 'application account' needs to be created though the Google or Microsoft provider console. This is generally done once by the developer and an application API ID and secret are then distributed with the application (usually hidden). These are then used by OAuth2 when logging in with an end user account. Note end users don't need to access the provider console or know it exists.
5. Added a new TIcsTwitter component and sample, requires a developer account from Twitter. Includes login to Twitter, send tweet, search tweets and get specific tweets, all responses are Json which the application needs to untangle.
6. Improved TRestParams allowing them to save more Delphi types correctly without conversion to strings, and to save parameters in new formats. AddItem has new overloaded methods to add Integer, Double and Boolean values, AddItemDT for TDateTime, AddItemSO for ISuperObject and AddItemAR for TStrings as Json array. Numerics and bool are saved as Json without quotes which is required for strict Json parsers. There is a new RfcStrict method that forces RFC3986 strict URL percent encoding, four unreserved chars (- . _ -) are not percent encoded. A new RemoveItem method to delete a parameter and new PContent type of PContCommaList which saves parameters as a name="value", comma separated list for Twitter.
7. There are various OAuth2 improvements to make it easier to implement. Added several TOAuthUri records designed to set-up common OAuth2 account settings for providers like Google, Twitter, Microsoft and Sipgate, by using the LoadAuthUri method. Note to avoid dragging all URIs into all applications, they need to be referenced specifically in applications. The REST sample builds an array to allow them to be selected from a list. TRestOAuth has new AccName and ConsoleUrl properties for display.
8. TSimpleWebSrv continues to get less simple, it has aWebSrvIP2 property for a second address so it can listen on IPv4 and IPv6 at the same time, with and without SSL if necessary. Setting WebSrvIP to localhost sets both 127.0.0.1 and [::1] so the browser OAuth2 redirect can choose IPv4 or IPv6.
9. Made some improvements to SuperObject, used for Json creation and parsing. When parsing Json there are new functions that return a sensible error message about parse errors and the location. There is a new object type DateTime or DT which reads or writes TDateTime to avoid the application needing to do the ISO string conversion. Json can now be parsed to a depth of 64 levels.
10. Rewrote and improved the way ICS reads SSL/TLS certificates and bundles, simplifying code that has got partly duplicated over the years as new methods were added, and improving error handling so the infamous stack error should no longer appear, instead more useful messages. All certificate files are now written with the UTF8 character set for the added comments that may include non-ASCII characters, and all comments are read and ignored by ICS rather than passing them to OpenSSL that may get confused by non-ASCII character comments which it is supposed to ignore. This also seems to fix a problem if the latest version of Windows 10 is configured for experimental UTF8 APIs which was where the 8-bit issues appeared, never a problem until then or if UTF8 APIs was disabled again. There are new methods for reading certificate bundles that return a list of certificate errors rather than an exception on the first error.
11. All the ICS root bundles are now created cleanly by an application, rather than mostly manually by copy and editing, to reduce errors. This fixed four corrupted root certificates in the older bundles, see FAQ - SSL/TLS Certificate Authority Root Stores
12. Previously the Jose unit offered functions primarily for client JWS/JWT REST applications, it now includes extra functions for REST servers to check and verify the JWS/JWT sent by clients. The Jose sample application has new tests for JWS/JWT, and to parse and display Json.
13. Added a new Multi Host FTP Server sample using IcsHosts. Really designed to be a Windows service application. It supports multiple SSL hosts with multiple listeners, can order it's own SSL certificates and will create self signed certificates for any missing, and will email status information and errors to an administrator.
14. Fixed a long term external SSL session cache issue in some components and samples that meant if an SSL handshake fails due to a bad certificate or chain, it is necessary to remove the SSL session from cache so an immediate retry does not succeed by skipping the certificate checks. This is only a short term issue, because the cache is usually cleared after a few minutes. This will effect any client applications using the external SSL session cache including HTTPS.
15. Increased the TCP send and receive buffer size to 64K in all components and samples, and generally don't allow it to be set lower. The TCP protocol has a window size field to control flow of data, the amount of data thatcan be sent without an acknowledgement that it has been receiv ed safely and does not need to be repeated. Back in dial-up days, the window size was small since corruption and resending was common. Early applications would increase the window size up to the original TCP maximum of 64K to improve performance with registry changes or using a TCP option that ICS exposes as SocketRcvBufSize and SocketSndBufSize properties in TWSocket. In ICS, only the FTP client and server components ever used this feature since they did large file transfers. Unfortunately the default buffer size never kept up with faster internet speeds which meant some components transferred data slowly.
16. Made some improvements ordering SSL/TLS certificates. Improved validation for remote certificate directories, better logging when waiting for challenges to complete, remove old challenges when starting new Acme order, save local server IP address with domains, and check using correct address, may be different for different order, Close local web server before collecting certificate to stop hacking. Made Windows Server DNS updating using WMI more robust so wild card Acme orders work reliably.
17. Made some internal changes loading OpenSSL, to avoid the two DLLs being loaded from different directories and to give better exceptions if they are missing. GSSLEAY_DLL_IgnoreOld defaults true since OpenSSL 1.0.2 is out of support. GSSLEAY_DLL_IgnoreNew now means use 1.1.1 not 3.0.0.
18. Renamed the unit Ics.Posix.Messages.pas to Ics.Posix.PXMessages.pas to avoid conflicts with projects loading it instead of WinApi.Messages, specifically those using THtmlViewer like our sample FrameBrowserIcs.dpr, maybe others. Made a lot of minor changes so that ICS now builds for Posix for MacOS and MacOS64, and for Linux64, but the 64-bit platforms are not yet supported pending integration with new message handling APIs.