Difference between revisions of "ICS Getting Started"

From Overbyte
Jump to navigation Jump to search
 
(6 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
ICS has a large number of sample application whose primary purpose is to test
 
ICS has a large number of sample application whose primary purpose is to test
 
all the components and to learn about using those components and how to use
 
all the components and to learn about using those components and how to use
them in your own applications.  There are often Multiple samples for a single
+
them in your own applications.  There are often several samples for a single
 
protocol with different purposes, so this section should help get you started
 
protocol with different purposes, so this section should help get you started
 
choosing the components and samples for your internet project.
 
choosing the components and samples for your internet project.
Line 13: Line 13:
 
have control over the commands.  The high level components are quicker and
 
have control over the commands.  The high level components are quicker and
 
easier to implement because they hide most of the protocol and offer complex
 
easier to implement because they hide most of the protocol and offer complex
methods instead such as download a file, they often include extra
+
methods instead such as download a file, they often include extra functionality.
functionality.
 
  
 
Historically, most ICS components are available on non-SSL and SSL versions,
 
Historically, most ICS components are available on non-SSL and SSL versions,
Line 22: Line 21:
 
higher level components mostly already include the SslContext and chain
 
higher level components mostly already include the SslContext and chain
 
checking and hide much of the SSL/TLS complexity making them faster to
 
checking and hide much of the SSL/TLS complexity making them faster to
implement and easier to maintain as SSL changes.  Note that any ICS
+
implement and easier to maintain as SSL changes.
applications using SSL/TLS need to redistribute two OpenSSL DLLs.
 
  
 +
 +
== ICS Code Snippets ==
 +
 +
The SSL sample OverbyteIcsSnippets contains small samples of codes for FTP,
 +
HTTP, sockets and email.  The unit includes several almost self contained
 +
methods each implementing a single functions, which are hopefully easier to
 +
follow than the normal samples used to develop ICS components and which often
 +
become very complicated due to all the different functionality supported. The
 +
snippets are heavily documented to try and explain usage.
 +
 +
Most of the snippets access Magenta Systems Ltd public ICS web and FTP servers
 +
and should just work without change, except for FTP uploading where you will
 +
need to request an account by emailing delphi@magsys.co.uk.  Snippets available
 +
include:
 +
 +
Snippet: View Local Directories - print a directory file listing.
 +
 +
Snippet: File Copy One File - copy a single file.
 +
 +
Snippet: File Copy Multiple Files - copy multiple files.
 +
 +
Snippet: FTP View Directories - print a remote directory listing from an FTP site.
 +
 +
Snippet: FTP Download One File - download a single file from an FTP site.
 +
 +
Snippet: FTP Download Multiple Files - downloads multiple files from an FTP site.
 +
 +
Snippet: FTP Upload One File - upload a single file to an FTP site.
 +
 +
Snippet: FTP Upload Multiple Files - upload multiple files to an FTP site.
 +
 +
Snippet: HTTP Download List of Files - downloads a list of files from a web site.
 +
 +
Snippet: HTTP Download Linked Files - downloads multiple files from a web site by  parsing HTML pages for links.
 +
 +
Snippet: HTTP REST Json Request - makes an HTTP GET request to a REST server receiving a Json response data.
 +
 +
Snippet: HTTP REST Download - makes an HTTP GET request to download a file, with optional resume of partial download.
 +
 +
Snippet: HTTP POST Upload File - makes a HTTP POST request to upload a file to a special upload web page.
 +
 +
Snippet: Local Socket Traffic - Send simple text traffic between two sockets on the same PC, using client server concepts.
 +
 +
Snippet: Remote Socket Traffic - Receive simple text traffic from a remote TCP Server.
 +
 +
Snippet: WebSocket Client - Connect to a remote WebSocket server to send and receive data.
 +
 +
Snippet: Send Email using Mail Queue - Runs a mail queue to send multiple emails with extended retries over many hours or days.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
 
== World Wide Web, HTTP Client ==
 
== World Wide Web, HTTP Client ==
  
There are three types of HTTP component, with many extra components used to
+
There are four types of HTTP component, with many extra components used to
 
extend their capabilities.
 
extend their capabilities.
  
 
TSslHttpCli in unit OverbyteIcsHttpProt is the low level HTTP protocol client
 
TSslHttpCli in unit OverbyteIcsHttpProt is the low level HTTP protocol client
that is tested using OverbyteIcsHttpsTst.dpr. It has buttons for GET and HEAD
+
that is tested using sample OverbyteIcsHttpsTst. It has buttons for GET and HEAD
 
commands and allows numerous SSL parameters to be specified. POST requests are
 
commands and allows numerous SSL parameters to be specified. POST requests are
tested with OverbyteIcsHttpPost.dpr and OverbyteIcsHttpPg.dpr. Other units
+
tested with samples OverbyteIcsHttpPost and OverbyteIcsHttpPg. Other units
 
containing components assisting HTTP include OverbyteIcsHttpCCodZLib,
 
containing components assisting HTTP include OverbyteIcsHttpCCodZLib,
 
OverbyteIcsHttpContCod, OverbyteIcsCookies, OverbyteIcsMimeUtils,
 
OverbyteIcsHttpContCod, OverbyteIcsCookies, OverbyteIcsMimeUtils,
 
OverbyteIcsFormDataDecoder, OverbyteIcsCharsetUtils, OverbyteIcsMsSslUtils,
 
OverbyteIcsFormDataDecoder, OverbyteIcsCharsetUtils, OverbyteIcsMsSslUtils,
MIME with OverbyteIcsMimeDemo.dpr, SSL certificate chains with
+
MIME with sample OverbyteIcsMimeDemo, SSL certificate chains with sample
OverbyteIcsMsVerify.dpr. Note TSslHttpCli requires an SslContext for SSL
+
OverbyteIcsMsVerify. Note TSslHttpCli requires an SslContext for SSL
 
configuration.  Note HTTP clients do not need SSL/TLS certificates, but
 
configuration.  Note HTTP clients do not need SSL/TLS certificates, but
 
generally should check the certificate chains received from HTTPS servers
 
generally should check the certificate chains received from HTTPS servers
Line 52: Line 100:
 
caching, content compression, content code page decoding, persistent
 
caching, content compression, content code page decoding, persistent
 
cookies, Json handling, logging and client SSL certificate support.  There
 
cookies, Json handling, logging and client SSL certificate support.  There
is also TRestOAuth for OAuth1/2 authentication and some REST examples
+
some REST examples TDnsQueryHttps, TIcsSms, TIcsTwitter and TIcsRestEmail.
TDnsQueryHttps, TIcsSms, TIcsTwitter and TIcsRestEmail. All tested using
+
All tested using sample OverbyteIcsHttpRestTst.
OverbyteIcsHttpRestTst.dpr.
+
 
 +
TRestOAuth in unit OverbyteIcsSslHttpOAuth handles OAuth1/2 authentication
 +
using either embedded EdgeBrowser or TWebBrowser to display the logins web
 +
pages, or they can be viewed using an external browser.  The unit supports
 +
various Microsoft User Authorities for corporate accounts. Note OAuth1/2 use
 +
requires a developer application account at Google or Microsoft, or other
 +
providers which includes Ids and secrets that need to be securely stored.
  
 
TIcsHttpMulti in unit OverbyteIcsHttpMulti is another high level HTTP client
 
TIcsHttpMulti in unit OverbyteIcsHttpMulti is another high level HTTP client
Line 60: Line 114:
 
or listed by parsing links from a web page, using a single function call. It
 
or listed by parsing links from a web page, using a single function call. It
 
also includes SSL configuration and certificate validation with a root bundle.
 
also includes SSL configuration and certificate validation with a root bundle.
Tested using OverbyteIcsXferTst.dpr.
+
Tested using sample OverbyteIcsXferTst.
 +
 
 +
TSslWebSocketCli in unit OverbyteIcsWebSocketCli is WebSocket client component
 +
that descends from TSslHttpRest so most of it's properties and events are common,
 +
but there are new methods and events to access WebSocket servers using ws:// or
 +
wss:// URLs.  WebSocket is a full duplex TCP protocol for web servers to support
 +
interactive web pages, typically dynamic updating such as chat sessions, spell
 +
checkers as you type, search hints, etc.
  
ICS has a visual web browser sample FrameBrowserIcs.dpr which needs
+
ICS has a visual web browser sample FrameBrowserIcs which needs the HtmlViewer
the HtmlViewer component to be installed, which will view simple web pages
+
component to be installed, which will view simple web pages that don't need
that don't need Javascript, it logs both HTTP and HTML protocol and can be
+
Javascript, it logs both HTTP and HTML protocol and can be very useful for
very useful for debugging.
+
debugging.
  
There are several simple HTTP samples to illustrate cookies, POST, threads,
+
There are two SSL samples OverbyteIcsHttpsTst and OverbyteIcsHttpRestTst
etc, but most of these are now included in OverbyteIcsHttpRestTst.dpr.
+
that illustrate HTTP GET and POST requests, authentication including OAuth2,
See OverbyteIcsConHttp.dpr, OverbyteIcsHttpAsy.dpr, OverbyteIcsHttpDmo.dpr,
+
file uploading and downloading, cookies, certificate chain verification,
OverbyteIcsHttpGet.dpr, OverbyteIcsHttpAsp.dpr and OverbyteIcsHttpThrd.dpr.
+
content encoding and decoding and WebSockets with a chat demo.
Another sample OverbyteIcsJoseTst.dpr can be used to test Json Object
+
 
 +
There are some older non-SSL demos for console and DLL and threads, see
 +
samples OverbyteIcsConHttp, OverbyteIcsHttpAsp and OverbyteIcsHttpThrd.
 +
Another sample OverbyteIcsJoseTst can be used to test Json Object
 
Signing (Jose) functions often used for REST requests, URL encoding and
 
Signing (Jose) functions often used for REST requests, URL encoding and
 
decoding and display of Json and XML data.
 
decoding and display of Json and XML data.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
 
== World Wide Web, HTTP Server ==
 
== World Wide Web, HTTP Server ==
  
There are three different HTTP web servers, which are based on
+
There are five different HTTP web servers, which are based on
 
TSslWSocketServer.
 
TSslWSocketServer.
  
 
TSslHttpServer in unit OverbyteIcsHttpSrv is the main web server, tested
 
TSslHttpServer in unit OverbyteIcsHttpSrv is the main web server, tested
with OverbyteIcsSslWebServ.dpr, while TSslHttpAppSrv in unit
+
with sample OverbyteIcsSslWebServr, while TSslHttpAppSrv in unit
 
OverbyteIcsHttpAppServer adds session support and page handlers for creating
 
OverbyteIcsHttpAppServer adds session support and page handlers for creating
dynamic page web applications. These servers only listen on one IP address
+
dynamic page web applications tested with sample OverbyteIcsSslWebAppServer.
and port, but you use multiple components for multiple listeners sharing the
+
These servers only listen on one IP address and port, but you use multiple
same events.  Note TSslHttpServer and OverbyteIcsHttpAppServer usually
+
components for multiple listeners sharing the same events.  Note TSslHttpServer
require an SslContext for SSL configuration. The samples are full web
+
and TSslHttpAppSrv require an SslContext for SSL configuration. The samples
servers with a lot of SSL configuration options for an SSL/TLS certificate,
+
are full web servers with a lot of SSL configuration options for an SSL/TLS
note HTTPS servers require an SSL certificate and will not start without one.
+
certificate, note HTTPS servers require an SSL certificate and will not start
Both samples include a number of dynamic web pages to illustrate basic
+
without one. Both samples include a number of dynamic web pages to illustrate
web server facilities, including a contact form that sends email.
+
basic web server facilities, including a contact form that sends email.
 +
 
 +
The WebSockets protocol is supported using the THttpWSSrvConn client class
 +
instead of THttpAppSrvConnection for only normal HTTP.
  
There is a third more advanced HTTP sample OverbyteIcsSslMultiWebServ.dpr
+
There is a third more advanced HTTP sample OverbyteIcsSslMultiWebServ
 
which configures TSslHttpAppSrv differently using collections of
 
which configures TSslHttpAppSrv differently using collections of
 
IcsHosts properties. This allows the web server to listen on multiple
 
IcsHosts properties. This allows the web server to listen on multiple
Line 103: Line 172:
 
them order free SSL/TLS certificates from Let's Encrypt (provided running
 
them order free SSL/TLS certificates from Let's Encrypt (provided running
 
on the public internet), and re-order them every three months before they
 
on the public internet), and re-order them every three months before they
expire.
+
expire.  IcsHosts can accept server certificates as PEM or PFX files, or
 +
from the Windows Certificate Store (but not from USB dongles). The sample
 +
OverbyteIcsSslMultiWebServr is different to most ICS samples in having a
 +
minimal GUI and being entirely configured using an INI file, it is really
 +
designed to be built as a Windows service application to run unattended
 +
in background.  It includes a separate web log for each host, and will
 +
send emails when it starts and stops.  It also includes some anti-hacking
 +
tests and will block abusive IP addresses. The sample is based on a
 +
commercial web server.
  
OverbyteIcsSslMultiWebServ is different to most ICS samples
+
Sample OverbyteIcsSslMultiWebServ also includes WebSockets support with
in having a minimal GUI and being entirely configured using an INI file,
+
simple echo servers and a chat server.
it is really designed to be built as a Windows service application to
+
 
run unattended in background.  It includes a separate web log for each
+
Sample OverbyteIcsDDWebService is very similar to OverbyteIcsSslMultiWebServ
host, and will send emails when it starts and stopsThe sample is based
+
but designed as a Windows service, although it will also run as a GUI for
on a commercial web server.
+
debuggingIt requires DDService service framework to be installed. It also
 +
includes a REST server with simple lookup responses from a SQL database, which
 +
requires DISQLite3 to be installed.
  
The third web server is TSimpleWebSrv in unit OverbyteIcsSslHttpRest which
+
The fifth web server is TSimpleWebSrv in unit OverbyteIcsSslHttpOAuth which
 
is a lightweight server with minimal functionality designed for embedding
 
is a lightweight server with minimal functionality designed for embedding
 
in applications needing OAuth2 or SSL/TLS certificate ordering that require
 
in applications needing OAuth2 or SSL/TLS certificate ordering that require
 
access to web server to check a host exists.  It has a single event that
 
access to web server to check a host exists.  It has a single event that
 
presents a request and returns a response. It supports SSL with IcsHosts.
 
presents a request and returns a response. It supports SSL with IcsHosts.
 +
There is no sample, but it is used by other ICS components.
  
 +
[https://{{SERVERNAME}}/arch/icsdemos-servers.zip '''Compiled ICS Server Samples Download''']
  
  
Line 125: Line 206:
  
 
TSslFtpClient in unit OverbyteIcsFtpCli is the low level FTP client that is
 
TSslFtpClient in unit OverbyteIcsFtpCli is the low level FTP client that is
tested with OverbyteIcsSslFtpTst.dpr.  It has about 50 buttons the test the
+
tested with sample OverbyteIcsSslFtpTst.  It has about 50 buttons the test the
 
various FTP commands in various ways, and allows numerous SSL parameters to
 
various FTP commands in various ways, and allows numerous SSL parameters to
 
be specified. Note TSslFtpClient requires an SslContext for SSL configuration.
 
be specified. Note TSslFtpClient requires an SslContext for SSL configuration.
Other FTP samples include OverbyteIcsBasFtp.dpr, OverbyteIcsConFtp.dpr,
+
Other older FTP samples include OverbyteIcsBasFtp, OverbyteIcsConFtp,
OverbyteIcsFtpAsy.dpr and OverbyteIcsFtpMulti.dpr.
+
OverbyteIcsFtpAsy and OverbyteIcsFtpMulti.
  
 
TIcsFtpMulti in unit OverbyteIcsFtpMulti is a high level FTP client that indexes,
 
TIcsFtpMulti in unit OverbyteIcsFtpMulti is a high level FTP client that indexes,
Line 138: Line 219:
 
be compared and files FTP uploaded or downloaded so they match.  It also
 
be compared and files FTP uploaded or downloaded so they match.  It also
 
includes SSL configuration and certificate validation with a root bundle,
 
includes SSL configuration and certificate validation with a root bundle,
SSL session caching and logging.
+
SSL session caching and logging. Use the sample OverbyteIcsXferTst to test
 +
TIcsFtpMulti.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
 
== File Transfer Protocol, FTP Server ==
 
== File Transfer Protocol, FTP Server ==
 +
 +
The FTP server is based on TSslWSocketServer.
  
 
The FTP server is based on TSslWSocketServer.
 
The FTP server is based on TSslWSocketServer.
  
 
TSslFtpServer in unit OverbyteIcsFtpSrv is the FTP server, tested using
 
TSslFtpServer in unit OverbyteIcsFtpSrv is the FTP server, tested using
OverbyteIcsSslFtpServ.drp. The FTP server only listens on one IP address
+
sample OverbyteIcsSslFtpServ. The FTP server only listens on one IP address
 
and port, but you use multiple components for multiple listeners sharing the
 
and port, but you use multiple components for multiple listeners sharing the
 
same events.  Note TSslFtpServer usually requires an SslContext for SSL
 
same events.  Note TSslFtpServer usually requires an SslContext for SSL
Line 153: Line 239:
 
certificate and will not start without one.
 
certificate and will not start without one.
  
There is a more advanced FTP server sample OverbyteIcsSslMultiFtpServ.dpr
+
There is a more advanced FTP server sample OverbyteIcsSslMultiFtpServ
 
which configures TSslFtpServer differently using collections of
 
which configures TSslFtpServer differently using collections of
 
IcsHosts properties. This allows the FTP server to listen on multiple
 
IcsHosts properties. This allows the FTP server to listen on multiple
Line 162: Line 248:
 
them order free SSL/TLS certificates from Let's Encrypt (provided running
 
them order free SSL/TLS certificates from Let's Encrypt (provided running
 
on the public internet), and re-order them every three months before they
 
on the public internet), and re-order them every three months before they
expire.
+
expire. OverbyteIcsSslMultiFtpServ is different to most ICS samples
 
 
OverbyteIcsSslMultiFtpServ is different to most ICS samples
 
 
in having a minimal GUI and being entirely configured using an INI file,
 
in having a minimal GUI and being entirely configured using an INI file,
 
it is really designed to be built as a Windows service application to run
 
it is really designed to be built as a Windows service application to run
 
unattended in background. The sample is based on a commercial FTP server.
 
unattended in background. The sample is based on a commercial FTP server.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-servers.zip '''Compiled ICS Server Samples Download''']
  
  
Line 176: Line 262:
  
 
TSslSmtpCli in unit OverbyteIcsSmtpProt is the low level SMTP client that
 
TSslSmtpCli in unit OverbyteIcsSmtpProt is the low level SMTP client that
is tested with OverbyteIcsSslMailSnd1.dpr.  It has about 16 buttons to
+
is tested with sample OverbyteIcsSslMailSnd1.  It has about 16 buttons to
 
test various SMTP commands and allow an email to be sent with attachments.
 
test various SMTP commands and allow an email to be sent with attachments.
Note TSslSmtpCli requires an SslContext for SSL configuration. Other test
+
Note TSslSmtpCli requires an SslContext for SSL configuration. Other older
samples include OverbyteIcsConSmtp.dpr, OverbyteIcsMailHtml.dpr and
+
test samples include OverbyteIcsConSmtp, OverbyteIcsMailHtml and
OverbyteIcsMailSndAsync.dpr.
+
OverbyteIcsMailSndAsync.
  
 
TIcsMailQueue in unit OverbyteIcsMailQueue is the high level SMTP client,
 
TIcsMailQueue in unit OverbyteIcsMailQueue is the high level SMTP client,
tested by OverbyteIcsMailQuTst.dpr.  It supports extended retries over many
+
tested by sample OverbyteIcsMailQuTst.  It supports extended retries over many
hours or days, and supports multiple SMTP relay servers or looks up MX
+
hours or days, and supports multiple SMTP relay servers or looks up MX servers
servers, while alleviating the need for the application to handle retries.
+
using DNS, while alleviating the need for the application to handle retries.
 
It spools emails as EML files, and can send them as well.  It includes SSL
 
It spools emails as EML files, and can send them as well.  It includes SSL
 
configuration and certificate validation with a root bundle and logging.
 
configuration and certificate validation with a root bundle and logging.
Line 191: Line 277:
 
TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of
 
TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of
 
sending email using HTTP REST requests to Google and Microsoft, instead
 
sending email using HTTP REST requests to Google and Microsoft, instead
of using SMTP. Tested using OverbyteIcsHttpRestTst.dpr.  This component
+
of using SMTP. Tested using sample OverbyteIcsHttpRestTst.  This component
 
also adds XOAuth2 authentication to the other SMTP components.
 
also adds XOAuth2 authentication to the other SMTP components.
 +
 +
All the mail components can use TRestOAuth in unit OverbyteIcsSslHttpOAuth
 +
to handle OAuth2 authentication using either embedded EdgeBrowser or
 +
TWebBrowser to display the logins web pages, or they can be viewed using an
 +
external browser.  The unit supports various Microsoft User Authorities for
 +
corporate accounts.  Note OAuth2 use requires a developer application account
 +
at Google or Microsoft, or other providers which includes Ids and secrets that
 +
need to be securely stored.
 +
 +
While mailboxes requiring OAuth2 require an initial interactive login, once
 +
that completes successfully a refresh token is returned which can be securely
 +
stored and treated like a password  for future access without requiring another
 +
login.  The refresh token can also be used by other applications provided they
 +
uses the same account Ids and secrets, allowing non-interactive applications
 +
like the ICS web, FTP and proxy servers to use GMail using TIcsMailQueue.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
Line 201: Line 304:
  
 
TSslPop3Cli in unit OverbyteIcsPop3Prot is the low level POP3 client that
 
TSslPop3Cli in unit OverbyteIcsPop3Prot is the low level POP3 client that
is tested with OverbyteIcsSslMailSnd1.dpr.  It has about 22 buttons to
+
is tested with sample OverbyteIcsSslMailSnd1.  It has about 22 buttons to
 
test various POP3 commands and allow emails to be retrieved from a mailbox.
 
test various POP3 commands and allow emails to be retrieved from a mailbox.
 
The unit OverbyteIcsMimeDec contains functions for decoding MIME encoded
 
The unit OverbyteIcsMimeDec contains functions for decoding MIME encoded
emails, tested with OverbyteIcsMimeDemo.dpr. AnOther test sample is
+
emails, tested with sample OverbyteIcsMimeDemo. AnOther older test sample
OverbyteIcsConPop3.dpr.  Note TSslPop3Cli requires an SslContext for SSL
+
is OverbyteIcsConPop3.  Note TSslPop3Cli requires an SslContext for SSL
 
configuration.
 
configuration.
  
 
TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of
 
TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of
 
receiving email using HTTP REST requests to Google and Microsoft, instead
 
receiving email using HTTP REST requests to Google and Microsoft, instead
of using POP3. Tested using OverbyteIcsHttpRestTst.dpr.  This component
+
of using POP3. Tested using sample OverbyteIcsHttpRestTst.  This component
also adds XOAuth2 authentication to the POP3 component.
+
also adds XOAuth2 authentication to the POP3 component. TRestOAuth in
 +
unit OverbyteIcsSslHttpOAuth handles OAuth2 authentication, see above.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
Line 218: Line 324:
 
TSslSmtpServer in unit OverbyteIcsSmtpSrv is an SMTP server that accepts
 
TSslSmtpServer in unit OverbyteIcsSmtpSrv is an SMTP server that accepts
 
emails from a client, making some checks and adding headers, which is
 
emails from a client, making some checks and adding headers, which is
tested by OverbyteIcsSslSmtpServ.drp which writes emails to an EML spool
+
tested by sample OverbyteIcsSslSmtpServ which writes emails to an EML spool
 
file.  Note neither component or sample support POP3 access, nor do they
 
file.  Note neither component or sample support POP3 access, nor do they
 
do anything with the EML file.  The TIcsMailQueue component could be
 
do anything with the EML file.  The TIcsMailQueue component could be
Line 224: Line 330:
 
for SSL configuration and SSL/TLS certificate, it does not yet support
 
for SSL configuration and SSL/TLS certificate, it does not yet support
 
IcsHosts.
 
IcsHosts.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-servers.zip '''Compiled ICS Server Samples Download''']
  
  
Line 239: Line 347:
 
FTP, SMTP, etc.  TSslWSocket can use TCP or UDP transmission, most
 
FTP, SMTP, etc.  TSslWSocket can use TCP or UDP transmission, most
 
protocols use TCP, except DNS and SNMP. TSslWSocket can be tested using
 
protocols use TCP, except DNS and SNMP. TSslWSocket can be tested using
OverbyteIcsSimpleSslCli.dpr, OverbyteIcsCliDemo.dpr, OverbyteIcsClient5.dpr,
+
samples OverbyteIcsSimpleSslCli, OverbyteIcsCliDemo, OverbyteIcsBinCliDemo,
OverbyteIcsClient7.dpr, OverbyteIcsUdpLstn.dpr, OverbyteIcsUdpSend.dpr
+
OverbyteIcsUdpLstn, OverbyteIcsUdpSend and many others.  Note TSslWSocket
and many others.  Note TSslWSocket requires an SslContext for SSL
+
requires an SslContext for SSL configuration.
configuration.
 
  
 
TIcsIpStrmLog in unit OverbyteIcsIpStreamLog is a higher level version
 
TIcsIpStrmLog in unit OverbyteIcsIpStreamLog is a higher level version
Line 250: Line 357:
 
connection attempts and retries on failure or loss of connection.
 
connection attempts and retries on failure or loss of connection.
 
TIcsIpStrmLog can be configured a client or server, TCP or UDP, and
 
TIcsIpStrmLog can be configured a client or server, TCP or UDP, and
is tested by OverbyteIcsIpStmLogTst.dpr which can run as client and
+
is tested by sample OverbyteIcsIpStmLogTst which can run as client and
 
server at the same time, sending data to itself.
 
server at the same time, sending data to itself.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-misc.zip '''Compiled ICS Miscelleous Samples Download''']
  
  
Line 270: Line 379:
  
 
TSslWSocketServer is mostly tested using the ICS HTTP and FTP servers,
 
TSslWSocketServer is mostly tested using the ICS HTTP and FTP servers,
but there are other samples, OverbyteIcsSslSmtpServ, OverbyteIcsTcpSrv.dpr,
+
but there are other samples, OverbyteIcsSimpleSslServer, OverbyteIcsTcpSrv,
OverbyteIcsTcpSrvIPv6.dpr, etc.
+
OverbyteIcsTcpSrvIPV6, OverbyteIcsThrdSrv, OverbyteIcsThrdSrvV2, etc.
  
 
TIcsIpStrmLog mentioned just above uses TSslWSocketServer for simpler
 
TIcsIpStrmLog mentioned just above uses TSslWSocketServer for simpler
Line 280: Line 389:
 
to avoid blocking on high load servers.  Beware this server does not yet
 
to avoid blocking on high load servers.  Beware this server does not yet
 
support IcsHosts and multiple IP addresses, nor is there a web server
 
support IcsHosts and multiple IP addresses, nor is there a web server
using it.  It is tested using OverbyteIcsThrdSrvV3.dpr.
+
using it.  It is tested using sample OverbyteIcsThrdSrvV3.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-misc.zip '''Compiled ICS Miscelleous Samples Download''']
  
  
 
== Forward or Reverse Proxy Server ==
 
== Forward or Reverse Proxy Server ==
  
TIcsProxy and TIcsHttpProxy in unit OverbyteIcsProxy are designed for
+
TIcsProxy and TIcsHttpProxy in unit OverbyteIcsProxy are designed
forward or reverse socket proxying and are tested by
+
for forward or reverse socket proxying and are tested by sample
OverbyteIcsProxySslServer.dpr.  Despite the component names, these
+
OverbyteIcsProxySslServer.  Despite the component names, these
 
components support SSL using IcsHosts with all the usual functions.
 
components support SSL using IcsHosts with all the usual functions.
 
TIcsProxy is protocol agnostic and may be used to proxy any TCP protocol,
 
TIcsProxy is protocol agnostic and may be used to proxy any TCP protocol,
Line 294: Line 405:
 
processing host names and URLs to match the source and destination.
 
processing host names and URLs to match the source and destination.
 
Note the sample has a minimal GUI and is configuring using an INI file.
 
Note the sample has a minimal GUI and is configuring using an INI file.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-servers.zip '''Compiled ICS Server Samples Download''']
 +
 +
 +
== Websockets Client and Server ==
 +
 +
See HTTP Client and HTTP Server above.
  
  
== Websockets Server ==
+
== MQ Telemetry Transport ==
  
TWebSocketSocket in unit OverbyteIcsWebSockets is a basic websockets server
+
TIcsMQTTServer and TIcsMQTTClient in OverbyteIcsMQTT handle the MQ Telemetry
based on TWSocketServer and websockets implementation ported from phpws
+
Transport message queuing service, tested by sample OverbyteIcsMQTTst which
project, tested by OverbyteIcsWebSocketSrv.
+
has both client and server.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
Line 306: Line 426:
  
 
TTnCnx in unit OverbyteIcsTnCnx implements the TCP/IP telnet protocol
 
TTnCnx in unit OverbyteIcsTnCnx implements the TCP/IP telnet protocol
including some options negotiations, tested by OverbyteIcsTnDemo.dpr.
+
including some options negotiations, tested by sample OverbyteIcsTnDemo.
There are samples that support telnet server, ANSI terminal emulation,
+
TTnEmulVT in unit OverbyteIcsTnEmulVT offers ANSI terminal emulation
and chat components.
+
tested using sample OverbyteIcsTelnetClient.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-misc.zip '''Compiled ICS Miscelleous Samples Download''']
  
  
Line 315: Line 437:
 
TNntpCli in unit OverbyteIcsNntpCli is a NNTP client, tested by
 
TNntpCli in unit OverbyteIcsNntpCli is a NNTP client, tested by
 
OverbyteIcsNewsReader.dpr with 28 buttons for the various commands,
 
OverbyteIcsNewsReader.dpr with 28 buttons for the various commands,
SSL is not supported yet.
+
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-clients.zip '''Compiled ICS Client Samples Download''']
  
  
Line 324: Line 447:
 
or client certificate, private key and one of more intermediate
 
or client certificate, private key and one of more intermediate
 
certificates, and has properties to display most of the certificate
 
certificates, and has properties to display most of the certificate
elements, all tested by OverbyteIcsPemtool.dpr.  TX509List contains
+
elements, all tested by sample OverbyteIcsPemtool.  TX509List contains
multiple certificates, typically a root store. The sample allow root
+
multiple certificates, typically a root store loaded from a PEM file.
stores and single certificates to be viewed, roots extracted from the
+
TMsX509List will load certificates from any Windows Certificate Store
Windows Certificate Store, and for SSL/TLS certificates, private keys
+
including private keys. TSslCertTools in unit OverbyteIcsSslX509Utils can
and certificate requests to be created and saved in multiple file formats
+
read, create and save certificates, private keys, certificate requests and
and bundles using TSslCertTools in unit OverbyteIcsSslX509Utils.
+
sign requests as a certificate authority.
 +
 
 +
The OverbyteIcsPemtool sample can be used to create certificates and
 +
private keys in various RSA, EC and other formats, create certificate
 +
requests and sign requests as a certificate authority, and convert
 +
certificate files between different formats, by reading as one and saving
 +
as another, also combining keys and certificates in a file.
 +
Root stores and single certificates may be to be viewed.  Certificates
 +
and private keys in Windows Stores may be viewed and deleted, and a PEM
 +
or PFX certificate bundle installed into any Windows Store.
  
TSslX509Certs in unit OverbyteIcsSslX509Certs, tested by
+
TSslX509Certs in unit OverbyteIcsSslX509Certs, tested by sample
OverbyteIcsX509CertsTst.dpr automatically downloads SSL/TLS X509
+
OverbyteIcsX509CertsTst which automatically downloads SSL/TLS X509
 
certificates from various issuers, including free certificates from Let's
 
certificates from various issuers, including free certificates from Let's
 
Encrypt, and  commercial certificates from CertCentre AG. Supports ACME
 
Encrypt, and  commercial certificates from CertCentre AG. Supports ACME
Line 338: Line 470:
 
validated certificates should generally  be issued without intervention,
 
validated certificates should generally  be issued without intervention,
 
other commercial certificates may take days to be approved. This unit may
 
other commercial certificates may take days to be approved. This unit may
be added to ICS server applications using IcsHosts while the sample may
+
be added to ICS server applications using IcsHosts for automatic ordering,
separately used to order certificates, including DNS validated wildcard
+
while the sample may be separately used to order certificates manually,
certificates from Let'S Encrypt.  All orders are kept in a database to
+
including DNS validated wildcard certificates from Let'S Encrypt.  All orders
allow automatic or manual re-ordering before expiry.
+
are kept in a database to allow automatic or manual re-ordering before expiry.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-tools.zip '''Compiled ICS Tool Samples Download''']
  
  
 
== Lookup Domain Names, DNS ==
 
== Lookup Domain Names, DNS ==
  
Simple DNS host look-ups are done using the DnsLookup method in TSslWSocket
+
Simple DNS host look-ups using the DNS servers configured for Windows are
and also ReverseDnsLookup, both fire an event with potentially multiple
+
done using the DnsLookup method in TSslWSocket and also the ReverseDnsLookup
results, tested by OverbyteIcsDnsLook.dpr and OverbyteIcsBatchDnsLookup.dpr.
+
method, both fire an event with potentially multiple results, tested by
These functions use Windows APIs that use the DNS servers configured for
+
sample OverbyteIcsDnsLook. These methods are used by almost all ICS
the PC.
+
components, although they currently only use the first IPv4 or IPv6 result
 +
returned, if more than one.
  
 
TDnsQuery in unit OverbyteIcsDnsQuery allows more complex DNS requests to
 
TDnsQuery in unit OverbyteIcsDnsQuery allows more complex DNS requests to
be made to any DNS server to get any DNS records, tested using
+
be made to specific DNS servers to get all DNS records such as MX or TXT,
OverbyteIcsNsLookup.dpr and OverByteIcsDnsResolver.dpr.
+
tested using sample OverbyteIcsNsLookup.  It includes a list of public DNS
 +
servers including Google, Cloudfare, OpenDNS and others, and will access
 +
these sequentially if one does not respond.  TDnsQueryHttps in unit
 +
OverbyteIcsSslHttpRest adds DNS over HTTPS for secure lookups. Also tested
 +
with sample OverbyteIcsNetTools.
 +
 
 +
TIcsDomNameCache and TIcsDomNameCacheHttps cache forward and reverse DNS lookup
 +
requests, mainly for diagnostic components but also for servers logging remote
 +
access. May be configured to use Windows lookup, UDP/TCP using TDnsQuery or
 +
HTTPS, testing using samples OverbyteIcsBatchDnsLookup and OverbyteIcsNetTools.
  
 
Unit OverbyteIcsWmi contains a number of functions for accessing a Windows
 
Unit OverbyteIcsWmi contains a number of functions for accessing a Windows
 
DNS Server (Windows Server 2012 and later) to list DNS zones and zone
 
DNS Server (Windows Server 2012 and later) to list DNS zones and zone
records, and to add zone records, tested by OverbyteIcsWmiTst.dpr.  The
+
records, and to add zone records, tested by sample OverbyteIcsWmiTst.  The
functions are also used by OverbyteIcsX509CertsTst.dpr to add DNS records
+
functions are also used by sample OverbyteIcsX509CertsTst to add DNS records
 
for the ACME DNS challenge.
 
for the ACME DNS challenge.
 +
 +
[https://{{SERVERNAME}}/arch/icsdemos-tools.zip '''Compiled ICS Tool Samples Download''']
  
  
 
== Network Diagnostic Tools ==
 
== Network Diagnostic Tools ==
  
TPing in unit OverbyteIcsPing is used to ping any host to see if it's
+
TPing and TPingThread in unit OverbyteIcsPing is used to ping any host to see
available on the internet, note some hosts may deliberately not reply,
+
if it's available on the internet, note some hosts may deliberately not reply,
tested by OverbyteIcsPingTst.dpr which includes trace route.
+
tested by samples OverbyteIcsPingTst and OverbyteIcsNetTools which both include
 +
trace route.
 +
 
 +
TIcsWhoisCli in unit OverbyteIcsWhoisCli makes Whois requests to get
 +
details for the registrations of domain names and IP address ranges,
 +
tested by samples OverbyteIcsWhoisCliTst and OverbyteIcsNetTools. The
 +
component has a large list of Whois servers for various countries
 +
around the world.
 +
 
 +
TIcsMonSocket in OverbyteIcsMonSock provides internet packet monitoring
 +
using raw sockets.  TIcsMonPcap in OverbyteIcsMonPcap provides internet
 +
monitoring using the Npcap NDIS driver.  There are both tested using
 +
sample OverbyteIcsNetMon which is similar to the WireShark diagnostic tool
 +
and can be used to monitor internet packets on a LAN, with filtering
 +
using TIcsMonFilterClass to include or exclude IPs, port or protocols.
 +
 
 +
TIcsIpChanges in OverbyteIcsIpHlpApi monitors IP address changes and calls
 +
an event for new IPs configured or old ones removed.  TIcsNeighbDevices
 +
in OverbyteIcsIpHlpApi builds a historic LAN MAC device and IPv4 and IPv6
 +
address table using ARP, neighbourhood and IP range scanning with reverse
 +
host lookup. Both are tested with sample OverbyteIcsNetTools which also
 +
uses several other IpHlp functions including IP Connections list, Network
 +
Adaptors and Interfaces, IP Routing and Path tables, ARP tables and Network
 +
Statistics.
  
 
TSnmpCli in unit OverbyteIcsSnmpCli does SNMP (simple network management
 
TSnmpCli in unit OverbyteIcsSnmpCli does SNMP (simple network management
protocol), tested by OverbyteIcsSnmpCliTst.
+
protocol), tested by sample OverbyteIcsSnmpCliTst.
  
 
TSysLogClient in unit OverbyteIcsSysLogClient send syslog packets, tested
 
TSysLogClient in unit OverbyteIcsSysLogClient send syslog packets, tested
by OverbyteIcsSysLogClientDemo.dpr.
+
by sample OverbyteIcsSysLogClientDemo.
  
 
TSysLogServer in unit OverbyteIcsSysLogServer receives syslog packets,
 
TSysLogServer in unit OverbyteIcsSysLogServer receives syslog packets,
tested by OverbyteIcsSysLogServerDemo.dpr.
+
tested by sample OverbyteIcsSysLogServerDemo.
 
 
TIcsWhoisCli in unit OverbyteIcsWhoisCli makes Whois requests to get
 
details for the registrations of domain names and IP address ranges,
 
tested by OverbyteIcsWhoisCliTst.dpr.  The component has a large list
 
of Whois servers for various countries around the world.
 
  
 
TIcsTimeClient and TIcsTimeServer in unit OverbyteIcsSntp support SNTP
 
TIcsTimeClient and TIcsTimeServer in unit OverbyteIcsSntp support SNTP
 
for getting and setting the correct time over the internet, tested
 
for getting and setting the correct time over the internet, tested
using OverbyteIcsTimeTst.dpr.
+
using sample OverbyteIcsTimeTst.
 +
 
 +
[https://{{SERVERNAME}}/arch/icsdemos-tools.zip '''Compiled ICS Tool Samples Download''']

Latest revision as of 18:04, 11 March 2024

Introduction

ICS has a large number of sample application whose primary purpose is to test all the components and to learn about using those components and how to use them in your own applications. There are often several samples for a single protocol with different purposes, so this section should help get you started choosing the components and samples for your internet project.

ICS often offers low and high level versions of components, the former allow your application to send the various commands used by the protocol but you need to send those commands in the correct order often dependent upon the result from earlier commands, so you need to understand the protocol, but have control over the commands. The high level components are quicker and easier to implement because they hide most of the protocol and offer complex methods instead such as download a file, they often include extra functionality.

Historically, most ICS components are available on non-SSL and SSL versions, these notes assume you are using SSL/TLS components which are often essential today. Note most low level component need SSL/TLS adding using an SslContext and need SSL certificate chain checking added to applications, while the higher level components mostly already include the SslContext and chain checking and hide much of the SSL/TLS complexity making them faster to implement and easier to maintain as SSL changes.


ICS Code Snippets

The SSL sample OverbyteIcsSnippets contains small samples of codes for FTP, HTTP, sockets and email. The unit includes several almost self contained methods each implementing a single functions, which are hopefully easier to follow than the normal samples used to develop ICS components and which often become very complicated due to all the different functionality supported. The snippets are heavily documented to try and explain usage.

Most of the snippets access Magenta Systems Ltd public ICS web and FTP servers and should just work without change, except for FTP uploading where you will need to request an account by emailing delphi@magsys.co.uk. Snippets available include:

Snippet: View Local Directories - print a directory file listing.

Snippet: File Copy One File - copy a single file.

Snippet: File Copy Multiple Files - copy multiple files.

Snippet: FTP View Directories - print a remote directory listing from an FTP site.

Snippet: FTP Download One File - download a single file from an FTP site.

Snippet: FTP Download Multiple Files - downloads multiple files from an FTP site.

Snippet: FTP Upload One File - upload a single file to an FTP site.

Snippet: FTP Upload Multiple Files - upload multiple files to an FTP site.

Snippet: HTTP Download List of Files - downloads a list of files from a web site.

Snippet: HTTP Download Linked Files - downloads multiple files from a web site by parsing HTML pages for links.

Snippet: HTTP REST Json Request - makes an HTTP GET request to a REST server receiving a Json response data.

Snippet: HTTP REST Download - makes an HTTP GET request to download a file, with optional resume of partial download.

Snippet: HTTP POST Upload File - makes a HTTP POST request to upload a file to a special upload web page.

Snippet: Local Socket Traffic - Send simple text traffic between two sockets on the same PC, using client server concepts.

Snippet: Remote Socket Traffic - Receive simple text traffic from a remote TCP Server.

Snippet: WebSocket Client - Connect to a remote WebSocket server to send and receive data.

Snippet: Send Email using Mail Queue - Runs a mail queue to send multiple emails with extended retries over many hours or days.

Compiled ICS Client Samples Download

World Wide Web, HTTP Client

There are four types of HTTP component, with many extra components used to extend their capabilities.

TSslHttpCli in unit OverbyteIcsHttpProt is the low level HTTP protocol client that is tested using sample OverbyteIcsHttpsTst. It has buttons for GET and HEAD commands and allows numerous SSL parameters to be specified. POST requests are tested with samples OverbyteIcsHttpPost and OverbyteIcsHttpPg. Other units containing components assisting HTTP include OverbyteIcsHttpCCodZLib, OverbyteIcsHttpContCod, OverbyteIcsCookies, OverbyteIcsMimeUtils, OverbyteIcsFormDataDecoder, OverbyteIcsCharsetUtils, OverbyteIcsMsSslUtils, MIME with sample OverbyteIcsMimeDemo, SSL certificate chains with sample OverbyteIcsMsVerify. Note TSslHttpCli requires an SslContext for SSL configuration. Note HTTP clients do not need SSL/TLS certificates, but generally should check the certificate chains received from HTTPS servers to ensure they are talking to the correct servers.

TSslHttpRest in unit OverbyteIcsSslHttpRest is the high level HTTP protocol client that has additional methods and properties for making GET, POST, PUT and HEAD REST (REpresentional State Transfer) client requests, but can still do everything TSslHttpCli does. It includes a TRestParams class to build and encode GET/PUT/POST parameter strings. It also includes SSL configuration and certificate validation with a root bundle, SSL session caching, content compression, content code page decoding, persistent cookies, Json handling, logging and client SSL certificate support. There some REST examples TDnsQueryHttps, TIcsSms, TIcsTwitter and TIcsRestEmail. All tested using sample OverbyteIcsHttpRestTst.

TRestOAuth in unit OverbyteIcsSslHttpOAuth handles OAuth1/2 authentication using either embedded EdgeBrowser or TWebBrowser to display the logins web pages, or they can be viewed using an external browser. The unit supports various Microsoft User Authorities for corporate accounts. Note OAuth1/2 use requires a developer application account at Google or Microsoft, or other providers which includes Ids and secrets that need to be securely stored.

TIcsHttpMulti in unit OverbyteIcsHttpMulti is another high level HTTP client that allows downloading of multiple files from an HTTP server using full URLs, or listed by parsing links from a web page, using a single function call. It also includes SSL configuration and certificate validation with a root bundle. Tested using sample OverbyteIcsXferTst.

TSslWebSocketCli in unit OverbyteIcsWebSocketCli is WebSocket client component that descends from TSslHttpRest so most of it's properties and events are common, but there are new methods and events to access WebSocket servers using ws:// or wss:// URLs. WebSocket is a full duplex TCP protocol for web servers to support interactive web pages, typically dynamic updating such as chat sessions, spell checkers as you type, search hints, etc.

ICS has a visual web browser sample FrameBrowserIcs which needs the HtmlViewer component to be installed, which will view simple web pages that don't need Javascript, it logs both HTTP and HTML protocol and can be very useful for debugging.

There are two SSL samples OverbyteIcsHttpsTst and OverbyteIcsHttpRestTst that illustrate HTTP GET and POST requests, authentication including OAuth2, file uploading and downloading, cookies, certificate chain verification, content encoding and decoding and WebSockets with a chat demo.

There are some older non-SSL demos for console and DLL and threads, see samples OverbyteIcsConHttp, OverbyteIcsHttpAsp and OverbyteIcsHttpThrd. Another sample OverbyteIcsJoseTst can be used to test Json Object Signing (Jose) functions often used for REST requests, URL encoding and decoding and display of Json and XML data.

Compiled ICS Client Samples Download


World Wide Web, HTTP Server

There are five different HTTP web servers, which are based on TSslWSocketServer.

TSslHttpServer in unit OverbyteIcsHttpSrv is the main web server, tested with sample OverbyteIcsSslWebServr, while TSslHttpAppSrv in unit OverbyteIcsHttpAppServer adds session support and page handlers for creating dynamic page web applications tested with sample OverbyteIcsSslWebAppServer. These servers only listen on one IP address and port, but you use multiple components for multiple listeners sharing the same events. Note TSslHttpServer and TSslHttpAppSrv require an SslContext for SSL configuration. The samples are full web servers with a lot of SSL configuration options for an SSL/TLS certificate, note HTTPS servers require an SSL certificate and will not start without one. Both samples include a number of dynamic web pages to illustrate basic web server facilities, including a contact form that sends email.

The WebSockets protocol is supported using the THttpWSSrvConn client class instead of THttpAppSrvConnection for only normal HTTP.

There is a third more advanced HTTP sample OverbyteIcsSslMultiWebServ which configures TSslHttpAppSrv differently using collections of IcsHosts properties. This allows the web server to listen on multiple IP addresses and ports at the same time, and to support multiple hosts on the same IP address serving different page content (as do most web servers). IcsHosts allow different SSL/TLS certificates to be specified for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire. IcsHosts can accept server certificates as PEM or PFX files, or from the Windows Certificate Store (but not from USB dongles). The sample OverbyteIcsSslMultiWebServr is different to most ICS samples in having a minimal GUI and being entirely configured using an INI file, it is really designed to be built as a Windows service application to run unattended in background. It includes a separate web log for each host, and will send emails when it starts and stops. It also includes some anti-hacking tests and will block abusive IP addresses. The sample is based on a commercial web server.

Sample OverbyteIcsSslMultiWebServ also includes WebSockets support with simple echo servers and a chat server.

Sample OverbyteIcsDDWebService is very similar to OverbyteIcsSslMultiWebServ but designed as a Windows service, although it will also run as a GUI for debugging. It requires DDService service framework to be installed. It also includes a REST server with simple lookup responses from a SQL database, which requires DISQLite3 to be installed.

The fifth web server is TSimpleWebSrv in unit OverbyteIcsSslHttpOAuth which is a lightweight server with minimal functionality designed for embedding in applications needing OAuth2 or SSL/TLS certificate ordering that require access to web server to check a host exists. It has a single event that presents a request and returns a response. It supports SSL with IcsHosts. There is no sample, but it is used by other ICS components.

Compiled ICS Server Samples Download


File Transfer Protocol, FTP Client

There are two types of FTP components for file transfers.

TSslFtpClient in unit OverbyteIcsFtpCli is the low level FTP client that is tested with sample OverbyteIcsSslFtpTst. It has about 50 buttons the test the various FTP commands in various ways, and allows numerous SSL parameters to be specified. Note TSslFtpClient requires an SslContext for SSL configuration. Other older FTP samples include OverbyteIcsBasFtp, OverbyteIcsConFtp, OverbyteIcsFtpAsy and OverbyteIcsFtpMulti.

TIcsFtpMulti in unit OverbyteIcsFtpMulti is a high level FTP client that indexes, uploads or downloads single or multiple files automatically, without needing to understand most FTP commands. One function indexes files and directories on an FTP server building a list compatible with the TIcsFileCopy component that indexes Windows directories, allowing local and remote directories to be compared and files FTP uploaded or downloaded so they match. It also includes SSL configuration and certificate validation with a root bundle, SSL session caching and logging. Use the sample OverbyteIcsXferTst to test TIcsFtpMulti.

Compiled ICS Client Samples Download


File Transfer Protocol, FTP Server

The FTP server is based on TSslWSocketServer.

The FTP server is based on TSslWSocketServer.

TSslFtpServer in unit OverbyteIcsFtpSrv is the FTP server, tested using sample OverbyteIcsSslFtpServ. The FTP server only listens on one IP address and port, but you use multiple components for multiple listeners sharing the same events. Note TSslFtpServer usually requires an SslContext for SSL configuration. The sample is a full FTP server for file uploads and downloads, with a lot of SSL configuration options for the SSL/TLS certificate and will not start without one.

There is a more advanced FTP server sample OverbyteIcsSslMultiFtpServ which configures TSslFtpServer differently using collections of IcsHosts properties. This allows the FTP server to listen on multiple IP addresses and ports at the same time, and to support multiple hosts on the same IP address. IcsHosts allow different SSL/TLS certificates to be specified for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire. OverbyteIcsSslMultiFtpServ is different to most ICS samples in having a minimal GUI and being entirely configured using an INI file, it is really designed to be built as a Windows service application to run unattended in background. The sample is based on a commercial FTP server.

Compiled ICS Server Samples Download


Sending Email, SMTP Client

There are three types of components for sending email using the SMTP protocol or HTTP REST protocol.

TSslSmtpCli in unit OverbyteIcsSmtpProt is the low level SMTP client that is tested with sample OverbyteIcsSslMailSnd1. It has about 16 buttons to test various SMTP commands and allow an email to be sent with attachments. Note TSslSmtpCli requires an SslContext for SSL configuration. Other older test samples include OverbyteIcsConSmtp, OverbyteIcsMailHtml and OverbyteIcsMailSndAsync.

TIcsMailQueue in unit OverbyteIcsMailQueue is the high level SMTP client, tested by sample OverbyteIcsMailQuTst. It supports extended retries over many hours or days, and supports multiple SMTP relay servers or looks up MX servers using DNS, while alleviating the need for the application to handle retries. It spools emails as EML files, and can send them as well. It includes SSL configuration and certificate validation with a root bundle and logging.

TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of sending email using HTTP REST requests to Google and Microsoft, instead of using SMTP. Tested using sample OverbyteIcsHttpRestTst. This component also adds XOAuth2 authentication to the other SMTP components.

All the mail components can use TRestOAuth in unit OverbyteIcsSslHttpOAuth to handle OAuth2 authentication using either embedded EdgeBrowser or TWebBrowser to display the logins web pages, or they can be viewed using an external browser. The unit supports various Microsoft User Authorities for corporate accounts. Note OAuth2 use requires a developer application account at Google or Microsoft, or other providers which includes Ids and secrets that need to be securely stored.

While mailboxes requiring OAuth2 require an initial interactive login, once that completes successfully a refresh token is returned which can be securely stored and treated like a password for future access without requiring another login. The refresh token can also be used by other applications provided they uses the same account Ids and secrets, allowing non-interactive applications like the ICS web, FTP and proxy servers to use GMail using TIcsMailQueue.

Compiled ICS Client Samples Download


Receiving Email, POP3 Client

There are two types of components for receiving email using the POP3 protocol or HTTP REST protocol.

TSslPop3Cli in unit OverbyteIcsPop3Prot is the low level POP3 client that is tested with sample OverbyteIcsSslMailSnd1. It has about 22 buttons to test various POP3 commands and allow emails to be retrieved from a mailbox. The unit OverbyteIcsMimeDec contains functions for decoding MIME encoded emails, tested with sample OverbyteIcsMimeDemo. AnOther older test sample is OverbyteIcsConPop3. Note TSslPop3Cli requires an SslContext for SSL configuration.

TIcsRestEmail in unit OverbyteIcsSslHttpRest is alternative means of receiving email using HTTP REST requests to Google and Microsoft, instead of using POP3. Tested using sample OverbyteIcsHttpRestTst. This component also adds XOAuth2 authentication to the POP3 component. TRestOAuth in unit OverbyteIcsSslHttpOAuth handles OAuth2 authentication, see above.

Compiled ICS Client Samples Download


Forwarding Email, SMTP Server

TSslSmtpServer in unit OverbyteIcsSmtpSrv is an SMTP server that accepts emails from a client, making some checks and adding headers, which is tested by sample OverbyteIcsSslSmtpServ which writes emails to an EML spool file. Note neither component or sample support POP3 access, nor do they do anything with the EML file. The TIcsMailQueue component could be used to forward EML files. Note TSslSmtpServer requires an SslContext for SSL configuration and SSL/TLS certificate, it does not yet support IcsHosts.

Compiled ICS Server Samples Download


Simple TCP Socket Client

TSslWSocket in unit OverbyteIcsWSocket is the root of most other ICS components opening a socket to either connect to a remote server, or to listen for connections from a remote server. The component always opens a socket by IP address, but will look-up that IP address from a host name if required, or provide a reverse look-up of host or domain name from an IP address. TSslWSocket sends or receives a stream of 8-bit binary characters, but does have methods to send and receive lines by checking or sending a CRLF line ending, which is the Telnet protocol, used for the headers all most other high level protocols like HTTP, FTP, SMTP, etc. TSslWSocket can use TCP or UDP transmission, most protocols use TCP, except DNS and SNMP. TSslWSocket can be tested using samples OverbyteIcsSimpleSslCli, OverbyteIcsCliDemo, OverbyteIcsBinCliDemo, OverbyteIcsUdpLstn, OverbyteIcsUdpSend and many others. Note TSslWSocket requires an SslContext for SSL configuration.

TIcsIpStrmLog in unit OverbyteIcsIpStreamLog is a higher level version of TSslWSocket, originally designed for IP stream logging with minimal events and extra coding, including an SslContext and full SSL/TLS certificate chain checking, with better line handling, multiple connection attempts and retries on failure or loss of connection. TIcsIpStrmLog can be configured a client or server, TCP or UDP, and is tested by sample OverbyteIcsIpStmLogTst which can run as client and server at the same time, sending data to itself.

Compiled ICS Miscelleous Samples Download


Simple TCP Socket Server

TSslWSocketServer in unit OverbyteIcsWSocketS is the main socket server accepting a few thousand remote clients using multiple IP addresses and ports, and separately allowing data to be sent and received from those remote clients, all in a single thread. Applications need to derive a client from TSslWSocketClient into which the required functionality is added. TSslWSocketServer supports using collections of IcsHosts properties. This allows the server to listen on multiple IP addresses and ports at the same time with different SSL/TLS certificates for each host using built-in SslContexts, will automatically create self signed SSL/TLS certificates so the server can start, and will them order free SSL/TLS certificates from Let's Encrypt (provided running on the public internet), and re-order them every three months before they expire.

TSslWSocketServer is mostly tested using the ICS HTTP and FTP servers, but there are other samples, OverbyteIcsSimpleSslServer, OverbyteIcsTcpSrv, OverbyteIcsTcpSrvIPV6, OverbyteIcsThrdSrv, OverbyteIcsThrdSrvV2, etc.

TIcsIpStrmLog mentioned just above uses TSslWSocketServer for simpler server applications with a small number of remote clients.

There is also a threaded version TSslWSocketThrdServer in unit OverbyteIcsWSocketTS where each client is created with a separate thread to avoid blocking on high load servers. Beware this server does not yet support IcsHosts and multiple IP addresses, nor is there a web server using it. It is tested using sample OverbyteIcsThrdSrvV3.

Compiled ICS Miscelleous Samples Download


Forward or Reverse Proxy Server

TIcsProxy and TIcsHttpProxy in unit OverbyteIcsProxy are designed for forward or reverse socket proxying and are tested by sample OverbyteIcsProxySslServer. Despite the component names, these components support SSL using IcsHosts with all the usual functions. TIcsProxy is protocol agnostic and may be used to proxy any TCP protocol, the sample includes SMTP, POP3, NNTP and telnet. TIcsHttpProxy is a full forward and reverse HTTP/HTTPS proxy with header and body parsing and processing host names and URLs to match the source and destination. Note the sample has a minimal GUI and is configuring using an INI file.

Compiled ICS Server Samples Download


Websockets Client and Server

See HTTP Client and HTTP Server above.


MQ Telemetry Transport

TIcsMQTTServer and TIcsMQTTClient in OverbyteIcsMQTT handle the MQ Telemetry Transport message queuing service, tested by sample OverbyteIcsMQTTst which has both client and server.

Compiled ICS Client Samples Download


Telnet Client

TTnCnx in unit OverbyteIcsTnCnx implements the TCP/IP telnet protocol including some options negotiations, tested by sample OverbyteIcsTnDemo. TTnEmulVT in unit OverbyteIcsTnEmulVT offers ANSI terminal emulation tested using sample OverbyteIcsTelnetClient.

Compiled ICS Miscelleous Samples Download


Network News Reader, NNTP Client

TNntpCli in unit OverbyteIcsNntpCli is a NNTP client, tested by OverbyteIcsNewsReader.dpr with 28 buttons for the various commands,

Compiled ICS Client Samples Download


Create, Order or Review SSL/TLS Certificates

ICS contains many functions for processing SSL/TLS X509 certificates and private keys. TX509Base in unit OverbyteIcsWSocket may contain and server or client certificate, private key and one of more intermediate certificates, and has properties to display most of the certificate elements, all tested by sample OverbyteIcsPemtool. TX509List contains multiple certificates, typically a root store loaded from a PEM file. TMsX509List will load certificates from any Windows Certificate Store including private keys. TSslCertTools in unit OverbyteIcsSslX509Utils can read, create and save certificates, private keys, certificate requests and sign requests as a certificate authority.

The OverbyteIcsPemtool sample can be used to create certificates and private keys in various RSA, EC and other formats, create certificate requests and sign requests as a certificate authority, and convert certificate files between different formats, by reading as one and saving as another, also combining keys and certificates in a file. Root stores and single certificates may be to be viewed. Certificates and private keys in Windows Stores may be viewed and deleted, and a PEM or PFX certificate bundle installed into any Windows Store.

TSslX509Certs in unit OverbyteIcsSslX509Certs, tested by sample OverbyteIcsX509CertsTst which automatically downloads SSL/TLS X509 certificates from various issuers, including free certificates from Let's Encrypt, and commercial certificates from CertCentre AG. Supports ACME V2 protocol, and REST protocols for specific vendors. Domain and DNS validated certificates should generally be issued without intervention, other commercial certificates may take days to be approved. This unit may be added to ICS server applications using IcsHosts for automatic ordering, while the sample may be separately used to order certificates manually, including DNS validated wildcard certificates from Let'S Encrypt. All orders are kept in a database to allow automatic or manual re-ordering before expiry.

Compiled ICS Tool Samples Download


Lookup Domain Names, DNS

Simple DNS host look-ups using the DNS servers configured for Windows are done using the DnsLookup method in TSslWSocket and also the ReverseDnsLookup method, both fire an event with potentially multiple results, tested by sample OverbyteIcsDnsLook. These methods are used by almost all ICS components, although they currently only use the first IPv4 or IPv6 result returned, if more than one.

TDnsQuery in unit OverbyteIcsDnsQuery allows more complex DNS requests to be made to specific DNS servers to get all DNS records such as MX or TXT, tested using sample OverbyteIcsNsLookup. It includes a list of public DNS servers including Google, Cloudfare, OpenDNS and others, and will access these sequentially if one does not respond. TDnsQueryHttps in unit OverbyteIcsSslHttpRest adds DNS over HTTPS for secure lookups. Also tested with sample OverbyteIcsNetTools.

TIcsDomNameCache and TIcsDomNameCacheHttps cache forward and reverse DNS lookup requests, mainly for diagnostic components but also for servers logging remote access. May be configured to use Windows lookup, UDP/TCP using TDnsQuery or HTTPS, testing using samples OverbyteIcsBatchDnsLookup and OverbyteIcsNetTools.

Unit OverbyteIcsWmi contains a number of functions for accessing a Windows DNS Server (Windows Server 2012 and later) to list DNS zones and zone records, and to add zone records, tested by sample OverbyteIcsWmiTst. The functions are also used by sample OverbyteIcsX509CertsTst to add DNS records for the ACME DNS challenge.

Compiled ICS Tool Samples Download


Network Diagnostic Tools

TPing and TPingThread in unit OverbyteIcsPing is used to ping any host to see if it's available on the internet, note some hosts may deliberately not reply, tested by samples OverbyteIcsPingTst and OverbyteIcsNetTools which both include trace route.

TIcsWhoisCli in unit OverbyteIcsWhoisCli makes Whois requests to get details for the registrations of domain names and IP address ranges, tested by samples OverbyteIcsWhoisCliTst and OverbyteIcsNetTools. The component has a large list of Whois servers for various countries around the world.

TIcsMonSocket in OverbyteIcsMonSock provides internet packet monitoring using raw sockets. TIcsMonPcap in OverbyteIcsMonPcap provides internet monitoring using the Npcap NDIS driver. There are both tested using sample OverbyteIcsNetMon which is similar to the WireShark diagnostic tool and can be used to monitor internet packets on a LAN, with filtering using TIcsMonFilterClass to include or exclude IPs, port or protocols.

TIcsIpChanges in OverbyteIcsIpHlpApi monitors IP address changes and calls an event for new IPs configured or old ones removed. TIcsNeighbDevices in OverbyteIcsIpHlpApi builds a historic LAN MAC device and IPv4 and IPv6 address table using ARP, neighbourhood and IP range scanning with reverse host lookup. Both are tested with sample OverbyteIcsNetTools which also uses several other IpHlp functions including IP Connections list, Network Adaptors and Interfaces, IP Routing and Path tables, ARP tables and Network Statistics.

TSnmpCli in unit OverbyteIcsSnmpCli does SNMP (simple network management protocol), tested by sample OverbyteIcsSnmpCliTst.

TSysLogClient in unit OverbyteIcsSysLogClient send syslog packets, tested by sample OverbyteIcsSysLogClientDemo.

TSysLogServer in unit OverbyteIcsSysLogServer receives syslog packets, tested by sample OverbyteIcsSysLogServerDemo.

TIcsTimeClient and TIcsTimeServer in unit OverbyteIcsSntp support SNTP for getting and setting the correct time over the internet, tested using sample OverbyteIcsTimeTst.

Compiled ICS Tool Samples Download