1.1 Introduction

combine is primarily a program for merging files on a common key. While such things as table joins on keys are common in database systems like MySQL, there seems to be a marked failure in the availability for such processing on text files. combine is intended to fill that gap.

Another way of looking at combine is as the kernel of a database system without all the overhead (and safeguards) associated with such systems. The missing baggage that appeals most to me is the requirement to load data into somebody else’s format before working with it. combine’s design is intended to allow it to work with most data directly from the place where it already is.

In looking around for existing software that wanted to do what I wanted to do, the closest I came was the join utility in GNU and other operating systems. join has some limitations that I needed to overcome. In particular, in matching it works on only one field each in only two files. For such a match, on files whose fields are separated by a delimiter, I’m sure that join is a more efficient choice. Someday I’ll test that assumption.

Once I started writing the program, I had to come up with a name. Given that one of the earliest applications that I imagined for such a program would be to prepare data for storage in a data warehouse, I thought to where the things that are stored in physical warehouses come from. At that point, I came up with the name DataFactory. Unfortunately, just as I got ready to release it, I noticed that someone else has that as a registered trademark.

As a result, I have come up with the name combine. Like the farm implement of the same name, this program can be used to separate the wheat from the chaff. I also like it because it has a similarity to join reminiscent of the similarity of function.

 — Further Introduction —
 

This document was generated by Daniel P. Valentine on July 28, 2013 using texi2html 1.82.