FAQ.Concatenating data

Main page -> FAQ -> Concatenating data

Why

Many times it is necessary to concatenate data. For example when receiving data chuncks with TWSocket, or to make a virtual HTML page with THttpServer.

One single advice is to not use strings in the form of:

Rcvd := Rcvd + Buffer;

Why not? Because every time you do that technically the following happens:

• Rcvd is reallocated memory. Not a big deal if there happens to be enough room right at the end of the block, but chance is low. So there is a new block allocated.
• The data in Rcvd is copied in the new block
• The buffer is appended to the data in the new block
• The original location memory is made free

You can imagine that your memory does get fragmented if that happens a lot, because other parts in your program also need to allocate / free memory.

Solution

The solution is to make a buffer for the data. The buffer has to allocate enough memory for the whole data, and if there is not enough it should reallocate in a cpu / memory friendly way.

Also the space has to be reused so that the allocation only happens one or a few time.

Code explanation

Creation

We need to create the class with an expected maximum Size. We also tell the class how to increment if size should getting too small. Both of the arguments are depending on the expecting data of course.

We allocate 1 byte more as we like to terminate the data with a NULL.

constructor TMemBuilder.Create(Size, Increment: integer);
begin
    FSize := Size + 1;
    GetMem(FData, FSize);
    FIncrement := Increment;
end;

Adding data

First we check if there is enough room. If not then we reallocate memory. Then we add the data to our buffer. We need in practice a few overloaded methods here, see therefore the complete code example.

procedure TMemBuilder.Add(P: PChar; Len: integer);
begin
    if FLen + Len >= FSize then begin
        FSize := FSize + Max(Len, FIncrement) + 1;
        ReAllocMem(FData, FSize);
    end;
    Move(P^, FData[FLen], Len);
    Inc(FLen, Len);
    FData[FLen] := #0;
end;

Reading data

Nothing can be more simple. We just have to public a property that point to the beginning of our data. In case it is binary data we need also a length property. If you want the data to be a string then you can do a typecast but think twice as this make an extra copy.

property Data: PChar read FData;
property Len: integer read FLen;

Clearing the buffer

We keep our memory for reuse, so all we need to do is reset the length.

procedure TMemBuilder.Clear;
begin
    FLen := 0;
    FData[0] := 0;
end;

Complete class

This is the complete class. You can copy / paste it to use it.

unit uMemBuilder;

interface

type
  TMemBuilder = class
  private
    FSize: integer;
    FLen: integer;
    FIncrement: integer;
    FData: PChar;
    function Max(a, b: integer): integer;
  public
    constructor Create(Size, Increment: integer);
    destructor  Destroy; override;
    procedure Clear;
    procedure Add(S: string); overload;
    procedure Add(P: PChar); overload;
    procedure Add(P: PChar; Len: integer); overload;
    property Data: PChar read FData;
    property Len: integer read FLen;
  end;

implementation

{ TMemBuilder }

procedure TMemBuilder.Add(S: string);
var
    Len: integer;
begin
    Len := Length(S);
    Add(PChar(S), Len);
end;

procedure TMemBuilder.Add(P: PChar);
var
    Len: integer;
begin
    Len := Length(P);
    Add(P, Len);
end;

procedure TMemBuilder.Add(P: PChar; Len: integer);
begin
    if FLen + Len >= FSize then begin
        FSize := FSize + Max(Len, FIncrement) + 1;
        ReAllocMem(FData, FSize);
    end;
    Move(P^, FData[FLen], Len);
    Inc(FLen, Len);
    FData[FLen] := #0;
end;

procedure TMemBuilder.Clear;
begin
    FLen := 0;
    FData[FLen] := #0;
end;

constructor TMemBuilder.Create(Size, Increment: integer);
begin
    FSize := Size + 1;
    GetMem(FData, FSize);
    FIncrement := Increment;
end;

destructor TMemBuilder.Destroy;
begin
    FreeMem(FData);
    inherited;
end;

function TMemBuilder.Max(a, b: integer): integer;
begin
    if a >= b then
        Result := a
    else
        Result := b;
end;

end.

rgds, Wilfried http://www.mestdagh.biz