Sending and receiving data

Main page -> FAQ -> Sending and receiving data

General

There are several ways to transfer data. No matter if it is a file, an image, textual or raw data, it is all the same. What is needed is a common language between the data sender and the data receiver, in order for each to understand the other. This language defines a format. Formatting data is a way to "shape" it, describing its beginning and its end, in order for the riecever to know where data starts and ends.

Many formats are of course possible, mainly depending upon the kind of data itself. Basically, we can consider that we have two kinds of data :

• predictable data content
• unpredictable data content

In the first case, we know what kind of content the data contains - ie the byte set that can be used is known (byte values from 32 to 127, for example). In the other case, the data content is not known - ie the byte set can be any possible byte value (from 0 to 255). This major difference will mostly determine the format of the data to be exchanged, defining then the protocol.

You can use line mode or packet mode. Using line mode has the advantage that TWSocket will concatenate packets for you but the disadvantage that it will hold all data in memory before it will fire OnDataAvailable. The latter is of course only important if packets are very large. If you choose for packet mode then you have to concatenate data yourself but you have control on what is kept in memory or not.

Line mode

Base-64

Base64 is an encoding process using a 64 letter alphabet where each letter representing 6 bits in the input stream. It is described in RFC 2045. All encoding and decoding procedures can be found in TMimeDecode component. The encoded data is about 33 percent larger and not human readable. Every character not used in the Base64 alphabet can be used as control character, including line end.

ASCII-hex

ASCII-hex is used in many protocols. Every character is converted into his hexadecimal equivalent and sent as such. For example the string '123' is sent as '313233'. The encoded data is twice as long and difficult human readable. Every character except 0..9, A..F can be used as control character, including line end.

Escaping

Escaping is very often used. Control characters including line end has to be chosen in a way they are as less as possible in the original data. The principle is to precede the control characters with an escape character and replace them by other characters. Very often a NULL character is escaped as well. The data is only a little longer than original and good human readable.

Packet mode

Preceding each data packet with his length

This is a very common used technique. The first 1, 2 o 4 bytes of the data represent the length of the packet. Note that it is common habitude in communications to represent the length header in Big Endian format while Intel CPU use by design Little Endian format.

Less used but also a good technique is to represent the preceding length in hex format of 2, 4 or 8 bytes. Advantage is that this part of the header is human readable.

Conclusion

Difficult to explain something :)

Wilfried 20:18, 21 February 2006 (CET)