UFO:AFTERMATH

Unofficial Virtual File System Format Specification Version 0.2

UFO:Aftermath is a trademark of Altar interactive. This document is not a product of Altar interactive, who should not be associated with it. Nor will they provide any support for it. You may use this document freely as long as you dont take out a fee for it. None of the information contained is guaranteed to be correct. The author can not be held responsible for any consequences related to the use or misuse of the information contained herein.

Introduction

The VFS-files are basically raw images of a FAT-style file system. As such they are easy to change dynamically but can easily get fragmented. That a compputer game uses a file system this way is very unusual. Its much more common to use simpler formats like .wad or .pak.

A VFS volume consists of these four segements:

The header describes the file system. It contains version information, sizes and limits of the system and an md5 checksum that is computed using the normal md5 algorithm. The header is followed by the fat table, which has one entry for each cluster in the volume and describes what clusters are used and how the clusters are linked together to form files. The root directory is a fixed-size area that contains the file entries for the root files/directories. Directories are normal files that contains sequences of file_entry. And last are the clusters.

Structures

struct header
{
  float  version             // format version, always 1.0
  uint32 clustersize         // observed values are 160 and 4096
  uint32 clustercount
  uint32 maxrootfiles        // 32 in save games, otherwise 64
  uint32 zero                // could be that maxrootfiles is 64bit, but its not very likely
  uint32 filenamelength      // always 64
  uint32 windowsize          // for compressed data, always 50000
  char   md5sum[16]          // calculated from offset 44 to EOF
  uint32 versionstringlength // always 256
  char   versionstring[256]
  uint32 usedclusters
}

struct file_entry
{
  char   name[64]             // padded with '\0'
  char   unknown1[4]          // unknown, not used by the game
  uint32 type                 // 1=file,2=directory,9=compressed file
  char   unknown2[4]          // always 0xFFFFFFFF
  uint32 startcluster         // one-based
  uint32 size
  uint32 size_uncompressed    // 0 if uncompressed
}

struct fat_entry
{
  uint32 usage                // 0=unused,1=used
  uint32 nextcluster          // 0xFFFFFFFF to signal end cluster
}

Compressed streams

The compressed files, type 9, are series of compressed chunks. The compression library is used is regular zlib. Each uncompressed chunk is at most the size of windowsize from the header. Before each chunk is a marker, uint32, that describes the size of the chunk. Then follows the compressed data. On some files the last marker is repeated after the last chunk, this is probably caused by a bug in the generator tool.

Notes

Always respect the size of a file. One many of the files the last cluster contains some extra data that isnt used. This is especially true for directories. Apparently Altar's vfs packing utility reused buffers when creating directories, because of this the last part of directories contain data from previous directories. So if you overread directories you'll get a file system with hard links.

In the type field of file_entry the gap between 2 and 9 could be the result of bit 3 being a flag that signals compression, this way, theoretically there can be compressed directories.

Cluster indices range from 1 and up to and including clustercount

Issues of the format

Problems and downsides of the way this format was designed.

• Clusters arent stored at efficient offsets.
  Cluster sizes found on ntfs and fat are always 2^x and the most common size nowadays is probably 4096. Due to this fact reads and writes to files are more efficient when they align to cluster boundaries. For example, when reading 500 bytes and the first 200 is in one cluster and the rest is in the next cluster two clusters will have to be read in. While if the whole 500 bytes were in the same cluster there would be one cluster read. The VFS-format could exploit this by storing its clusters on even cluster boundaries, so that a vfs cluster perfectly matches a cluster on disk. One way of achieving this as the format looks now would be to extend to version string so that the fat-table starts at offset 4096. And then use a fat-table with a size dividable in 4096, since a fat_entry is 8 bytes and 4096/8 is 512, use multiples of 512 as your clustercount. I'm still unsure of what performance increases this can give.

Future

There is still alot of loose ends. Since version 0.1 a few of them have been resolved.

• What does the mysterious field unknown1 in file_entry really mean?
  The game has no problem reading vfs files with unknown1 set to 0. So the game doesnt need them, its probably something thats used when volume is filled by Altar's tools.
• Can the filenames really be longer than 64?
  This is still to be tested. But my guess is that this value is hardcoded in the game.
• Can the version string have lengths other then 256?
  Yes, apparently they can.
• Can the compression window size be any other value than 50000?
  Havent tested this yet. But its probably a hard coded value.
• Can there be compressed directories?
  This isnt very likely. But i havent ruled it out yet.
• The md5sum is zeroed out in gamedata.vfs 1.1a, is this legal for all VFS?
  Its legal for some files, and not for others.
• What do the files contained in the archive mean? The amount of reconfiguration possible is still unexplored.
  The unoffical UFO:AM spec does a great job at this. If you find something there thats incorrect or someting thats missing, mail me about it.
• The contents of directories are sorted. Is this a requirement?
  No it isnt, the game had no problem reading a gamedata.vfs whose directories i had rearranged randomly.

Credits

Fnurg gave me the version field and the uncompressed_size field. Actually, my first extractor didnt use the uncompressed_size since i didnt know about it. It's zero on all the files in the beginning of gamedata.vfs so i discarded it as being an unused field.

If you find something thats missing or incorrect please let me know by sending an e-mail to the address below. If you implement something using this spec i'd love to here about it.