Description of the GRF file format.

[Tiera]

On the Internet it is impossible to find a description of the format of this file type, namely Gravity Patch File. In developing KPatcher I gathered enough information that you could share it with the community.

To simplify the writing and understanding of the text, we introduce a small definition for all GRF/GPF files and we will call them shortly - container.

Version of the containers differ only by the methods encrypt the content.

In his description, I will talk only about 0x200 version container.

The description also is present code in C/C++ examples.

So let's begin.

To start analyze a simplified structure of the container:

1) Cap the container in which to store the number of files in a container, a compressed file table address container, the container version and a bit of official information;

2) The table container file contains the file name, file size, the compressed file size, compressed file size is aligned, the file position in the container and the attributes of a file in a container;

3) Compressed files.

All this can be described as structures of the C/C++

To cap structure is as follows:

struct grf_header
{
char Magic [16];			/// "Master of Magic" +\0
char Key [14];				/​​// 0x01 -> 0x0E, or 0x00 -> 0x00 (no encryption)
uint32_t FileTableOffset;   /// The location of the file table
uint32_t Seed;				/// What is the value of randomly
uint32_t FilesCount;		/// The actual number of files = FilesCount - Seed - 7
uint32_t Version;		   /// GRF file version: 0x102, 0x103, 0x200
};

We analyze this structure in more detail:

1) Magic [16] - an identifier that indicates the program running with a container that is in front of her GRF/GPF container, the keyword "Master of Magic";

2) Key [14] - a key that tells what the data inside the container encryption, obsolete value is not used;

3) FileTableOffset - well, everything is clear, there is stored the offset location of the file table, I want to see that all the specified offset is not the beginning of the file, and the end cap of the container which is equal to 'sizeof (grf_header)' or 46;

4) Seed - a value that randomly invented by Koreans do not know why, probably used in conjunction with Key [14] in earlier versions of the container to protect the information inside the container, which is obsolete value is always 0;

5) FilesCount - the number of files in the container, this value is not absolute, but a bit distorted.

The actual value can be obtained by doing a simple mathematical operation RealFilesCount = FilesCount - Seed - 7;

6) Version - this is how it was not hard to guess, the version container.

Now, using this knowledge, try to read this information from any arbitrary container.

To do this, I will use the programming language C/C++

#include <...>
#include <...>
#include <...>

# Pragma pack (push, 1)		/// Align in-memory structure of 1 byte
struct grf_header
{
char Magic [16];		/// "Master of Magic" +\0
char Key [14];			/// 0x01 -&gt; 0x0E, or 0x00 -&gt; 0x00 (no encryption)
uint32_t FileTableOffset;   /// The location of the file table
uint32_t Seed;			/// What is the value of randomly
uint32_t FilesCount;		/// The actual number of files = FilesCount - Seed - 7
uint32_t Version;		/// GRF file version: 0x102, 0x103, 0x200
};
# Pragma pack (pop)

# Define GRF_HEADER "Master of Magic"
# Define GRF_HEADER_SIZE 46

int main ()
{
FILE * FD = NULL;
struct grf_header Header = {0};
uint32_t RealFilesCount = 0;

FD = fopen ("data.grf", "rb");

/// Read the header container
fread (&amp; Header, GRF_HEADER_SIZE, 1, FD);

/// Check if this is a GRF/GPF container or not
if (strncmp (Header.Magic, GRF_HEADER, 16)
{
return 0;
}

/// Now the structure is stored Header informations about current container
/// And can easily access them

/// The variable is now RealFilesCount the actual number of files in a container
/// On this can start reading the information about files
RealFilesCount = Header.FilesCount - Header.Seed - 7;

return 0;
}

In this example, I make out the structure of GRF/GPF files and shows how to use the C/C++, you can determine that this is the GRF/GPF file, which version of the file and how many files are contained in this container.

And so, to deal with the basic description of the container and having considered the example of obtaining information about the container down to work with the data container.

The first thing to get a table with data in the file container.

Fully describe the entire structure of the table files can be very simple:

table: [<zsize>. l <size>. l [<fname>. s <fentry> .17 b] *]
fentry: [<zsize>. l <zsizeal>. l <size>. l <flags>. b <offset>. l] .17 b

From this simple circuit structure is visible to full data is located on the very structure of the file table and file table

With this information, can easily get information about all files stored in the container.

Table with a container files can also be described as a structure:

struct grf_fentry
{
uint32_t zSize;		/// Size of packed data
uint32_t zSizeAl;   /// The same thing only with alignment
uint32_t Size;		/// Size of uncompressed data
uint8_t Flags;		/// Flag file (GRF_FLAG_FILE, GRF_FLAG_MIXCRYPT, GRF_FLAG_DES)
uint32_t Offset;	/// offset into the GRF file (starts immediately after grf_header)
};

Describe in detail the structure, I will not, in the comments should explain everything quite clear.

#include <...>
#include <...>
#include <...>
.................................................. .................
int main ()
{
.................................................. .........
uint8_t * pBuffer = NULL, * pZBuffer = NULL;
uint32_t TableSize = 0, zTableSize = 0;

/// Move the file pointer at the beginning of the file table
fseek (FD, FileTableOffset + GRF_HEADER_SIZE, SEEK_SET);

/// Read compressed and the actual size of the MFT
if (! fread (&amp; zTableSize, 4, 1, FD) | |! fread (&amp; TableSize, 4, 1, FD))
{
return 0;
}

/// Allocate the necessary memory volume and read the file table
pBuffer = new uint8_t [TableSize]; pZBuffer = new uint8_t [zTableSize];
if (! fread (pZBuffer, zTableSize, 1, FD))
{
delete [] pBuffer, delete [] pZBuffer;
return 0;
}

/// Extract the file table
/// If you want to use the code presented here will have to connect the library zlib
/// This function is part of GrfLib for KPatcher
/// The variable pBuffer now stored unpacked and ready to read the file table
if (zio:: zlib_uncompress (pBuffer, TableSize, pZBuffer, zTableSize)! = TableSize)
{
delete [] pBuffer, delete [] pZBuffer;
return 0;
}
delete [] pZBuffer;

return 0;
}

We now have a fully ready to read the file table. Now it remains the case for small, properly read and write to it, we will write to the memory of which optionally can be written to a file.

For that would work with the data it was convenient to define another structure that will remind you what struct grf_fentry, but will carry more useful information:

struct _FileNode
{
std:: string FileName;	  /// File Name
uint32_t NameHash;	   /// Hash the file name
uint32_t FileOffset;	   /// file offset in a container
uint32_t FileSize;	   /// Size
uint32_t zFileSize;		/// compressed file size
uint32_t zFileSizeAl;	  /// compressed file size aligned
uint8_t FileFlags;	   /// Flags file
int32_t Cycle;			   /// Cycle (used for encrypted files)
};

This structure can sozherzhat information about only one file, and must keep information about all files, but it requires some realties store, which will be convenient to work:

typedef std:: list <_FileNode> _FilesList;

Now that is all that is necessary, we can begin reading the information about files

#include <...>
#include <...>
#include <...>
.................................................. .................
.................................................. .................
struct grf_fentry
{
uint32_t zSize;		/// Size of packed data
uint32_t zSizeAl;	  /// The same thing only with alignment
uint32_t Size;		 /// Size of uncompressed data
uint8_t Flags;		 /// Flag file (GRF_FLAG_FILE, GRF_FLAG_MIXCRYPT, GRF_FLAG_DES)
uint32_t Offset;	   /// offset into the GRF file (starts immediately after grf_header)
};

# Define GRF_TABLE_SIZE 17
# Define GRF_FLAG_FILE a
# Define GRF_FLAG_MIXCRYPT 2
# Define GRF_FLAG_DES 4

struct _FileNode
{
std:: string FileName;	/// File Name
uint32_t NameHash;		/// Hash the file name
uint32_t FileOffset;	  /// file offset in a container
uint32_t FileSize;	  /// Size
uint32_t zFileSize;	   /// compressed file size
uint32_t zFileSizeAl;	 /// compressed file size aligned
uint8_t FileFlags;		/// Flags file
int32_t Cycle;			/// Cycle (used for encrypted files)
};
typedef std:: list &lt;_FileNode&gt; _FilesList;

int main ()
{
.................................................. .........
.................................................. .........

_FilesList FilesList;
for (uint32_t Offset = 0; Offset <tablesize;) {="" get="" the="" file="" name="" length="" size_t="" fn_size="strlen" ((char="" *)="" (pbuffer="" +="" offset));="" if="" (fn_size=""> = 0x100)
{
delete [] pBuffer;
return 0;
}


grf_file_table FileEntry = {0};
/// Get a pointer to the file name
char * FileName = (char *) (pBuffer + Offset);
Offset + = FN_Size + 1;

/// Get the file information
memcpy (&amp; FileEntry, (pBuffer + Offset), GRF_TABLE_SIZE);
Offset + = GRF_TABLE_SIZE;

/// Skip the directories and files whose size is equal to 0
if (! (FileEntry.Flags &amp; GRF_FLAG_FILE) | | FileEntry.Size == 0)
continue;

/// Fill the structure with the file information
_FileNode Node;
Node.FileName.append (FileName, FN_Size);
Node.FileFlags = FileEntry.Flags;
Node.NameHash = zio:: zlib_crc32 (FileName, FN_Size);
Node.FileOffset = FileEntry.Offset;
Node.FileSize = FileEntry.Size;
Node.zFileSize = FileEntry.zSize;
Node.zFileSizeAl = FileEntry.zSizeAl;
Node.Cycle = -1;

/// Get the information needed to decrypt the file
if (Node.FileFlags &amp; GRF_FLAG_MIXCRYPT)
{
Node.Cycle = 1;
for (uint32_t i = 10; Node.zFileSize&gt; = i; i = i * 10)
Node.Cycle + +;
}
if (Node.FileFlags &amp; GRF_FLAG_DES)
Node.Cycle = 0;

/// Add all the information on file in the repository.
FilesList.push_back (Node);
}

return 0;
}

Now, the variable contains FilesList informations about each file in the container and can easily burn any data to a file.

Now we can unite all that is written above together and we get:

#include <stdio.h>
#include <string.h>
#include <string>
#include <list>
#include <stdint.h>

#include <zlib.h>

using namespace std;

#pragma pack(push, 1)		   /// Align in-memory structure of 1 byte
struct grf_header
{
char Magic[16];			 /// "Master of Magic" +\0
char Key[14];			 	 /// 0x01 -> 0x0E, or 0x00 -> 0x00 (no encryption)
uint32_t FileTableOffset;   /// The location of the file table
uint32_t Seed;				/// What is the value of randomly
uint32_t FilesCount;		/// The actual number of files = FilesCount - Seed - 7
uint32_t Version;	   	 /// GRF file version: 0x102, 0x103, 0x200
};
#pragma pack(pop)

#define GRF_HEADER "Master of Magic"
#define GRF_HEADER_SIZE 46

#pragma pack(push, 1)   /// Align in-memory structure of 1 byte
struct grf_fentry
{
uint32_t zSize;		/// Size of packed data
uint32_t zSizeAl;   /// The same thing only with alignment
uint32_t Size;		/// Size of uncompressed data
uint8_t Flags;		/// Flag file (GRF_FLAG_FILE, GRF_FLAG_MIXCRYPT, GRF_FLAG_DES)
uint32_t Offset;	/// offset into the GRF file (starts immediately after grf_header)
};
#pragma pack(pop)

#define GRF_TABLE_SIZE 17

#define GRF_FLAG_FILE 1
#define GRF_FLAG_MIXCRYPT 2
#define GRF_FLAG_DES 4

struct _FileNode
{
std::string FileName;   /// File Name
uint32_t NameHash;	  /// Hash the file name
uint32_t FileOffset;	/// file offset in a container
uint32_t FileSize;	  /// Size
uint32_t zFileSize;	 /// compressed file size
uint32_t zFileSizeAl;   /// compressed file size aligned
uint8_t FileFlags;	  /// Flags file
int32_t Cycle;		  /// Cycle (used for encrypted files)
};
typedef std::list <_FileNode> _FilesList;


int main()
{
/// Some declarations
FILE *FD = NULL;
struct grf_header Header;
uint32_t RealFilesCount = 0;
uint8_t * pBuffer = NULL, * pZBuffer = NULL;
uint32_t TableSize = 0, zTableSize = 0;
_FilesList FilesList;

FD = fopen("naoro.grf", "rb");
fread(&Header, GRF_HEADER_SIZE, 1, FD);

if ( strncmp(Header.Magic, GRF_HEADER, 16) )
	 return 0;

/// Now the structure is stored Header informations about current container
/// And can easily access them

/// The variable is now RealFilesCount the actual number of files in a container
/// On this can start reading the information about files
RealFilesCount = Header.FilesCount - Header.Seed - 7;

/// Move the file pointer at the beginning of the file table
fseek(FD, Header.FileTableOffset + GRF_HEADER_SIZE, SEEK_SET);

/// Read compressed and the actual size of the MFT
if ( !fread (&zTableSize, 4, 1, FD) || !fread (&TableSize, 4, 1, FD) )
{
	return 0;
}

/// Allocate the necessary memory volume and read the file table
pBuffer = new uint8_t [TableSize]; pZBuffer = new uint8_t [zTableSize];
if( !fread (pZBuffer, zTableSize, 1, FD) )
{
	delete [] pBuffer, delete [] pZBuffer;
	return 0;
}
fclose(FD);

/// Extract the file table
/// If you want to use the code presented here will have to connect the library zlib
/// This function is part of GrfLib for KPatcher
/// The variable pBuffer now stored unpacked and ready to read the file table
if ( uncompress((Bytef*)pBuffer, (uLongf*)&TableSize, (Bytef*)pZBuffer, zTableSize) != Z_OK )
{
	delete [] pBuffer, delete [] pZBuffer;
	return 0;
}
delete [] pZBuffer;

for ( uint32_t Offset = 0; Offset < TableSize; )
{
	size_t FN_Size = strlen((char*)(pBuffer+Offset));
	if ( FN_Size >= 0x100 )
	{
		delete[] pBuffer;
		return 0;
	}

	grf_fentry FileEntry = {0};
	/// Get a pointer to the file name
	char *FileName = (char*)(pBuffer+Offset);
	Offset += FN_Size + 1;
	/// Get the file information
	memcpy(&FileEntry, (pBuffer+Offset), GRF_TABLE_SIZE);
	Offset += GRF_TABLE_SIZE;

	/// Skip the directories and files whose size is equal to 0
	if ( !(FileEntry.Flags&GRF_FLAG_FILE) || FileEntry.Size == 0 )
		continue;

	/// Fill the structure with the file information
	_FileNode Node;
	Node.FileName.append(FileName, FN_Size);
	Node.FileFlags = FileEntry.Flags;
	Node.NameHash = crc32(0, (Bytef*)FileName, FN_Size);
	Node.FileOffset = FileEntry.Offset;
	Node.FileSize = FileEntry.Size;
	Node.zFileSize = FileEntry.zSize;
	Node.zFileSizeAl = FileEntry.zSizeAl;
	Node.Cycle = -1;

	/// Get the information needed to decrypt the file
	if ( Node.FileFlags&GRF_FLAG_MIXCRYPT )
	{
		Node.Cycle = 1;
		for( uint32_t i = 10; Node.zFileSize >= i; i = i*10 )
			Node.Cycle++;
	}
	if ( Node.FileFlags&GRF_FLAG_DES )
	{
		Node.Cycle = 0;
	}

	/// Add all the information on file in the repository.
	FilesList.push_back(Node);
}
delete[] pBuffer;

/// Creating empty file
FD = fopen("files_list.txt", "w+");

_FilesList::iterator itr = FilesList.begin();
_FilesList::iterator itr_end = FilesList.end();

/// Fill the newly created file data.
fprintf(FD, "FileName - FileSize - FilePos\n");
for ( ; itr != itr_end; itr++)
	fprintf(FD, "%s - %d - %d\n", itr->FileName.c_str(), itr->FileSize, itr->FileOffset+GRF_HEADER_SIZE);
fclose(FD);

return 0;
}

Fully working source code that reads the GRF/GPF file and prints the contents a file.

main.rar

[Mercurial]

We can actually put this in the Wiki ;O

[Tiera]

Yes, I do not mind.

[Tiera]

Updated.

[Zwei]

Very interesting info.

Thank you for share it (:

Edited January 10, 2012 by Zwei

[manabeast]

so many word xD! for ppl dun like study like me k.o ~ in the middle reading. . can tell me shortcut? what use for? just for knowleght?

[kojex 2.0]

n1 information

[trenx]

ill use this to make an online grf maker.

[poshu]

Just started to play with this (thanks a lot btw!) there is a mistake though:

4) Seed - a value that randomly invented by Koreans do not know why, probably used in conjunction with Key [14] in earlier versions of the container to protect the information inside the container, which is obsolete value is always 0;

5) FilesCount - the number of files in the container, this value is not absolute, but a bit distorted.

The actual value can be obtained by doing a simple mathematical operation RealFilesCount = FilesCount - Seed - 7;

if seed is always 0 (which is not the case with my test grf, but it might be corrupted or something) it would not be of any use to subtract it from filescount to get the realfilescount.

Still pretty cool, I gonna have some fun :3

[Aeomin]

Just started to play with this (thanks a lot btw!) there is a mistake though:

4) Seed - a value that randomly invented by Koreans do not know why, probably used in conjunction with Key [14] in earlier versions of the container to protect the information inside the container, which is obsolete value is always 0;

5) FilesCount - the number of files in the container, this value is not absolute, but a bit distorted.

The actual value can be obtained by doing a simple mathematical operation RealFilesCount = FilesCount - Seed - 7;

if seed is always 0 (which is not the case with my test grf, but it might be corrupted or something) it would not be of any use to subtract it from filescount to get the realfilescount.

Still pretty cool, I gonna have some fun :3

There is no rule says the seed always be 0.. Tiera probably made assumption since most tools use 0.

[clydelion]

very informative!