ICS Getting Started

Introduction

ICS has a large number of sample application whose primary purpose is to test all the components and to learn about using those components and how to use them in your own applications. There are often Multiple samples for a single protocol with different purposes, so this section should help get you started choosing the components and samples for your internet project.

ICS often offers low and high level versions of components, the former allow your application to send the various commands used by the protocol but you need to send those commands in the correct order often dependent upon the result from earlier commands, so you need to understand the protocol, but have control over the commands. The high level components are quicker and easier to implement because they hide most of the protocol and offer complex methods instead such as download a file, they often include extra functionality.

Historically, most ICS components are available on non-SSL and SSL versions, these notes assume you are using SSL/TLS components which are often essential today. Note most low level component need SSL/TLS adding using an SslContext and need SSL certificate chain checking added to applications, while the higher level components mostly already include the SslContext and chain checking and hide much of the SSL/TLS complexity making them faster to implement and easier to maintain as SSL changes. Note that any ICS applications using SSL/TLS need to redistribute two OpenSSL DLLs.

World Wide Web, HTTP Client

There are three types of HTTP component, with many extra components used to extend their capabilities.

TSslHttpCli in unit OverbyteIcsHttpProt is the low level HTTP protocol client that is tested using OverbyteIcsHttpsTst.dpr. It has buttons for GET and HEAD commands and allows numerous SSL parameters to be specified. POST requests are tested with OverbyteIcsHttpPost.dpr and OverbyteIcsHttpPg.dpr. Other units containing components assisting HTTP include OverbyteIcsHttpCCodZLib, OverbyteIcsHttpContCod, OverbyteIcsCookies, OverbyteIcsMimeUtils, OverbyteIcsFormDataDecoder, OverbyteIcsCharsetUtils, OverbyteIcsMsSslUtils, MIME with OverbyteIcsMimeDemo.dpr, SSL certificate chains with OverbyteIcsMsVerify.dpr. Note TSslHttpCli requires an SslContext for SSL configuration. Note HTTP clients do not need SSL/TLS certificates, but generally should check the certificate chains received from HTTPS servers to ensure they are talking to the correct servers.

TSslHttpRest in unit OverbyteIcsSslHttpRest is the high level HTTP protocol client that has additional methods and properties for making GET, POST, PUT and HEAD REST (REpresentional State Transfer) client requests, but can still do everything TSslHttpCli does. It includes a TRestParams class to build and encode GET/PUT/POST parameter strings. It also includes SSL configuration and certificate validation with a root bundle, SSL session caching, content compression, content code page decoding, persistent cookies, Json handling, logging and client SSL certificate support. There is also TRestOAuth for OAuth1/2 authentication and some REST examples TDnsQueryHttps, TIcsSms, TIcsTwitter and TIcsRestEmail. All tested using OverbyteIcsHttpRestTst.dpr.

TIcsHttpMulti in unit OverbyteIcsHttpMulti is another high level HTTP client that allows downloading of multiple files from an HTTP server using full URLs, or listed by parsing links from a web page, using a single function call. It also includes SSL configuration and certificate validation with a root bundle. Tested using OverbyteIcsXferTst.dpr.

ICS has a visual web browser sample FrameBrowserIcs.dpr which needs the HtmlViewer component to be installed, which will view simple web pages that don't need Javascript, it logs both HTTP and HTML protocol and can be very useful for debugging.

There are several simple HTTP samples to illustrate cookies, POST, threads, etc, but most of these are now included in OverbyteIcsHttpRestTst.dpr. See OverbyteIcsConHttp.dpr, OverbyteIcsHttpAsy.dpr, OverbyteIcsHttpDmo.dpr, OverbyteIcsHttpGet.dpr, OverbyteIcsHttpAsp.dpr and OverbyteIcsHttpThrd.dpr. Another sample OverbyteIcsJoseTst.dpr can be used to test Json Object Signing (Jose) functions often used for REST requests, URL encoding and decoding and display of Json and XML data.

World Wide Web, HTTP Server

There are three different HTTP web servers, which are based on TSslWSocketServer.

TSslHttpServer in unit OverbyteIcsHttpSrv is the main web server, tested with OverbyteIcsSslWebServ.dpr, while TSslHttpAppSrv in unit OverbyteIcsHttpAppServer adds session support and page handlers for creating dynamic page web applications. These servers only listen on one IP address and port, but you use multiple components for multiple listeners sharing the same events. Note TSslHttpServer and OverbyteIcsHttpAppServer usually require an SslContext for SSL configuration. The samples are full web servers with a lot of SSL configuration options for an SSL/TLS certificate, note HTTPS servers require an SSL certificate and will not start without one. Both samples include a number of dynamic web pages to illustrate basic web server facilities, including a contact form that sends email.

There is a third more advanced HTTP sample OverbyteIcsSslMultiWebServ.dpr which configures TSslHttpAppSrv differently using collections of IcsHosts properties. This allows the web server to listen on multiple IP addresses and ports at the same time, and to support multiple hosts on the same IP address serving different page content (as do most web servers). IcsHosts allow different SSL/TLS certificates to be specified for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire.

OverbyteIcsSslMultiWebServ is different to most ICS samples in having a minimal GUI and being entirely configured using an INI file, it is really designed to be built as a Windows service application to run unattended in background. It includes a separate web log for each host, and will send emails when it starts and stops. The sample is based on a commercial web server.

The third web server is TSimpleWebSrv in unit OverbyteIcsSslHttpRest which is a lightweight server with minimal functionality designed for embedding in applications needing OAuth2 or SSL/TLS certificate ordering that require access to web server to check a host exists. It has a single event that presents a request and returns a response. It supports SSL with IcsHosts.

File Transfer Protocol, FTP Client

There are two types of FTP components for file transfers.

TSslFtpClient in unit OverbyteIcsFtpCli is the low level FTP client that is tested with OverbyteIcsSslFtpTst.dpr. It has about 50 buttons the test the various FTP commands in various ways, and allows numerous SSL parameters to be specified. Note TSslFtpClient requires an SslContext for SSL configuration. Other FTP samples include OverbyteIcsBasFtp.dpr, OverbyteIcsConFtp.dpr, OverbyteIcsFtpAsy.dpr and OverbyteIcsFtpMulti.dpr.

TIcsFtpMulti in unit OverbyteIcsFtpMulti is a high level FTP client that indexes, uploads or downloads single or multiple files automatically, without needing to understand most FTP commands. One function indexes files and directories on an FTP server building a list compatible with the TIcsFileCopy component that indexes Windows directories, allowing local and remote directories to be compared and files FTP uploaded or downloaded so they match. It also includes SSL configuration and certificate validation with a root bundle, SSL session caching and logging.

File Transfer Protocol, FTP Server

The FTP server is based on TSslWSocketServer.

TSslFtpServer in unit OverbyteIcsFtpSrv is the FTP server, tested using OverbyteIcsSslFtpServ.drp. The FTP server only listens on one IP address and port, but you use multiple components for multiple listeners sharing the same events. Note TSslFtpServer usually requires an SslContext for SSL configuration. The sample is a full FTP server for file uploads and downloads, with a lot of SSL configuration options for the SSL/TLS certificate and will not start without one.

There is a more advanced FTP server sample OverbyteIcsSslMultiFtpServ.dpr which configures TSslFtpServer differently using collections of IcsHosts properties. This allows the FTP server to listen on multiple IP addresses and ports at the same time, and to support multiple hosts on the same IP address. IcsHosts allow different SSL/TLS certificates to be specified for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire.

OverbyteIcsSslMultiFtpServ is different to most ICS samples in having a minimal GUI and being entirely configured using an INI file, it is really designed to be built as a Windows service application to run unattended in background. The sample is based on a commercial FTP server.

Sending Email, SMTP Client

There are three types of components for sending email using the SMTP protocol or HTTP REST protocol.

TSslSmtpCli in unit OverbyteIcsSmtpProt is the low level SMTP client that is tested with OverbyteIcsSslMailSnd1.dpr. It has about 16 buttons to test various SMTP commands and allow an email to be sent with attachments. Note TSslSmtpCli requires an SslContext for SSL configuration. Other test samples include OverbyteIcsConSmtp.dpr, OverbyteIcsMailHtml.dpr and OverbyteIcsMailSndAsync.dpr.

TIcsMailQueue in unit OverbyteIcsMailQueue is the high level SMTP client, tested by OverbyteIcsMailQuTst.dpr. It supports extended retries over many hours or days, and supports multiple SMTP relay servers or looks up MX servers, while alleviating the need for the application to handle retries. It spools emails as EML files, and can send them as well. It includes SSL configuration and certificate validation with a root bundle and logging.

TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of sending email using HTTP REST requests to Google and Microsoft, instead of using SMTP. Tested using OverbyteIcsHttpRestTst.dpr. This component also adds XOAuth2 authentication to the other SMTP components.

Receiving Email, POP3 Client

There are two types of components for receiving email using the POP3 protocol or HTTP REST protocol.

TSslPop3Cli in unit OverbyteIcsPop3Prot is the low level POP3 client that is tested with OverbyteIcsSslMailSnd1.dpr. It has about 22 buttons to test various POP3 commands and allow emails to be retrieved from a mailbox. The unit OverbyteIcsMimeDec contains functions for decoding MIME encoded emails, tested with OverbyteIcsMimeDemo.dpr. AnOther test sample is OverbyteIcsConPop3.dpr. Note TSslPop3Cli requires an SslContext for SSL configuration.

TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of receiving email using HTTP REST requests to Google and Microsoft, instead of using POP3. Tested using OverbyteIcsHttpRestTst.dpr. This component also adds XOAuth2 authentication to the POP3 component.

Forwarding Email, SMTP Server

TSslSmtpServer in unit OverbyteIcsSmtpSrv is an SMTP server that accepts emails from a client, making some checks and adding headers, which is tested by OverbyteIcsSslSmtpServ.drp which writes emails to an EML spool file. Note neither component or sample support POP3 access, nor do they do anything with the EML file. The TIcsMailQueue component could be used to forward EML files. Note TSslSmtpServer requires an SslContext for SSL configuration and SSL/TLS certificate, it does not yet support IcsHosts.

Simple TCP Socket Client

TSslWSocket in unit OverbyteIcsWSocket is the root of most other ICS components opening a socket to either connect to a remote server, or to listen for connections from a remote server. The component always opens a socket by IP address, but will look-up that IP address from a host name if required, or provide a reverse look-up of host or domain name from an IP address. TSslWSocket sends or receives a stream of 8-bit binary characters, but does have methods to send and receive lines by checking or sending a CRLF line ending, which is the Telnet protocol, used for the headers all most other high level protocols like HTTP, FTP, SMTP, etc. TSslWSocket can use TCP or UDP transmission, most protocols use TCP, except DNS and SNMP. TSslWSocket can be tested using OverbyteIcsSimpleSslCli.dpr, OverbyteIcsCliDemo.dpr, OverbyteIcsClient5.dpr, OverbyteIcsClient7.dpr, OverbyteIcsUdpLstn.dpr, OverbyteIcsUdpSend.dpr and many others. Note TSslWSocket requires an SslContext for SSL configuration.

TIcsIpStrmLog in unit OverbyteIcsIpStreamLog is a higher level version of TSslWSocket, originally designed for IP stream logging with minimal events and extra coding, including an SslContext and full SSL/TLS certificate chain checking, with better line handling, multiple connection attempts and retries on failure or loss of connection. TIcsIpStrmLog can be configured a client or server, TCP or UDP, and is tested by OverbyteIcsIpStmLogTst.dpr which can run as client and server at the same time, sending data to itself.

Simple TCP Socket Server

TSslWSocketServer in unit OverbyteIcsWSocketS is the main socket server accepting a few thousand remote clients using multiple IP addresses and ports, and separately allowing data to be sent and received from those remote clients, all in a single thread. Applications need to derive a client from TSslWSocketClient into which the required functionality is added. TSslWSocketServer supports using collections of IcsHosts properties. This allows the server to listen on multiple IP addresses and ports at the same time with different SSL/TLS certificates for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire.

TSslWSocketServer is mostly tested using the ICS HTTP and FTP servers, but there are other samples, OverbyteIcsSslSmtpServ, OverbyteIcsTcpSrv.dpr, OverbyteIcsTcpSrvIPv6.dpr, etc.

TIcsIpStrmLog mentioned just above uses TSslWSocketServer for simpler server applications with a small number of remote clients.

There is also a threaded version TSslWSocketThrdServer in unit OverbyteIcsWSocketTS where each client is created with a separate thread to avoid blocking on high load servers. Beware this server does not yet support IcsHosts and multiple IP addresses, nor is there a web server using it. It is tested using OverbyteIcsThrdSrvV3.dpr.

Forward or Reverse Proxy Server

TIcsProxy and TIcsHttpProxy in unit OverbyteIcsProxy are designed for forward or reverse socket proxying and are tested by OverbyteIcsProxySslServer.dpr. Despite the component names, these components support SSL using IcsHosts with all the usual functions. TIcsProxy is protocol agnostic and may be used to proxy any TCP protocol, the sample includes SMTP, POP3, NNTP and telnet. TIcsHttpProxy is a full forward and reverse HTTP/HTTPS proxy with header and body parsing and processing host names and URLs to match the source and destination. Note the sample has a minimal GUI and is configuring using an INI file.

Websockets Server

TWebSocketSocket in unit OverbyteIcsWebSockets is a basic websockets server based on TWSocketServer and websockets implementation ported from phpws project, tested by OverbyteIcsWebSocketSrv.

Telnet Client

TTnCnx in unit OverbyteIcsTnCnx implements the TCP/IP telnet protocol including some options negotiations, tested by OverbyteIcsTnDemo.dpr. There are samples that support telnet server, ANSI terminal emulation, and chat components.

Network News Reader, NNTP Client

TNntpCli in unit OverbyteIcsNntpCli is a NNTP client, tested by OverbyteIcsNewsReader.dpr with 28 buttons for the various commands, SSL is not supported yet.

Create, Order or Review SSL/TLS Certificates

ICS contains many functions for processing SSL/TLS X509 certificates and private keys. TX509Base in unit OverbyteIcsWSocket may contain and server or client certificate, private key and one of more intermediate certificates, and has properties to display most of the certificate elements, all tested by OverbyteIcsPemtool.dpr. TX509List contains multiple certificates, typically a root store. The sample allow root stores and single certificates to be viewed, roots extracted from the Windows Certificate Store, and for SSL/TLS certificates, private keys and certificate requests to be created and saved in multiple file formats and bundles using TSslCertTools in unit OverbyteIcsSslX509Utils.

TSslX509Certs in unit OverbyteIcsSslX509Certs, tested by OverbyteIcsX509CertsTst.dpr automatically downloads SSL/TLS X509 certificates from various issuers, including free certificates from Let's Encrypt, and commercial certificates from CertCentre AG. Supports ACME V2 protocol, and REST protocols for specific vendors. Domain and DNS validated certificates should generally be issued without intervention, other commercial certificates may take days to be approved. This unit may be added to ICS server applications using IcsHosts while the sample may separately used to order certificates, including DNS validated wildcard certificates from Let'S Encrypt. All orders are kept in a database to allow automatic or manual re-ordering before expiry.

Lookup Domain Names, DNS

Simple DNS host look-ups are done using the DnsLookup method in TSslWSocket and also ReverseDnsLookup, both fire an event with potentially multiple results, tested by OverbyteIcsDnsLook.dpr and OverbyteIcsBatchDnsLookup.dpr. These functions use Windows APIs that use the DNS servers configured for the PC.

TDnsQuery in unit OverbyteIcsDnsQuery allows more complex DNS requests to be made to any DNS server to get any DNS records, tested using OverbyteIcsNsLookup.dpr and OverByteIcsDnsResolver.dpr.

Unit OverbyteIcsWmi contains a number of functions for accessing a Windows DNS Server (Windows Server 2012 and later) to list DNS zones and zone records, and to add zone records, tested by OverbyteIcsWmiTst.dpr. The functions are also used by OverbyteIcsX509CertsTst.dpr to add DNS records for the ACME DNS challenge.

Network Diagnostic Tools

TPing in unit OverbyteIcsPing is used to ping any host to see if it's available on the internet, note some hosts may deliberately not reply, tested by OverbyteIcsPingTst.dpr which includes trace route.

TSnmpCli in unit OverbyteIcsSnmpCli does SNMP (simple network management protocol), tested by OverbyteIcsSnmpCliTst.

TSysLogClient in unit OverbyteIcsSysLogClient send syslog packets, tested by OverbyteIcsSysLogClientDemo.dpr.

TSysLogServer in unit OverbyteIcsSysLogServer receives syslog packets, tested by OverbyteIcsSysLogServerDemo.dpr.

TIcsWhoisCli in unit OverbyteIcsWhoisCli makes Whois requests to get details for the registrations of domain names and IP address ranges, tested by OverbyteIcsWhoisCliTst.dpr. The component has a large list of Whois servers for various countries around the world.

TIcsTimeClient and TIcsTimeServer in unit OverbyteIcsSntp support SNTP for getting and setting the correct time over the internet, tested using OverbyteIcsTimeTst.dpr.