Single binary QDP files are composed of a series of one or more application records. A single application record encodes a single QDP field or an array of QDP fields of the same data type. Physics metadata, managed at the convenience of the applications programmer, is associated with the file itself and with each application record as well. Above the API the QDP file is viewed as follows:
LatticeColorVector. The file metadata might include information about the gauge field configuration and the record metadata might encode the eigenvalue and an index for the eigenvector.
For another example, the gauge field configuration in four dimensions is represented in QDP as an array of four color matrix fields. The configuration is conventionally written so that the four color matrices associated with each site appear together. A file containing a single gauge field configuration would then consist of a single record containing the array of four color matrices.
The API permits mixing records of different datatypes in the same file. While this practice may be convenient for managing projects, it may be forbidden for archival files.
Additional metadata is automatically managed by QIO (without requiring intervention by the applications programmer) to facilitate the implementation and to check data integrity. Thus the file actually begins with QIO metadata and physics metadata and each application record consists of five logical records. Within QIO the file is viewed as a series of logical records as follows:
A new format called LIME (Lattice-QCD Interchange Message Encapsulation)
is used for packaging the logical records. A feature of this format is
the maximum record size is quite - bytes. However,
file system limitations may require splitting a single file into
multiple physical files. No provision is provided for such splitting.
It is expected that in time file systems will evolve to allow much
larger file sizes. In the interim, facilities like
tar can be used for file splitting.