How to Read and Write Binary Data for Your Custom File Formats

In my previous article, Create Custom Binary File Formats for Your Game's Data, I covered the topic of using custom binary file formats to store game assets and resources. In this short tutorial we will take a quick look at how to actually read and write binary data.

Note: This tutorial uses pseudo-code to demonstrate how to read and write binary data, but the code can easily be translated to any programming language that supports basic file I/O operations.

Bitwise Operators

If this is all unfamiliar territory for you, you will notice a few strange operators being used in the code, specifically the `&`, `|`, `<<` and `>>` operators. These are standard bitwise operators, available in most programming language, which are used for manipulating binary values.

Endianness and Streams

Before we can read and write binary data successfully, there are two important concepts that we need to understand: endianness and streams.

Endianness dictates the order of multiple-byte values within a file or within a chunk of memory. For example, if we had a 16-bit value of `0x1020`, that value can either be stored as `0x10` followed by `0x20` (big-endian) or `0x20` followed by `0x10` (little-endian).

Streams are array-like objects that contain a sequence of bytes (or bits in some cases). Binary data is read from and written to these streams. Most programming will provide an implementation of binary streams in one form or another; some are more convoluted than others, but they all essentially do the same thing.

Let's start by defining some properties in our code. Ideally these should all be private properties:

Here is an example of what a basic class constructor might look like:

The following functions will read unsigned integers from the stream:

These functions will read signed integers from the stream:

Writing Binary Data

Let's start by defining some properties in our code. (These are more or less the same as the properties we defined for reading binary data.) Ideally these should all be private properties:

Here is an example of what a basic class constructor might look like:

The following functions will write unsigned integers to the stream:

And, again, these functions will write signed integers to the stream. (The functions are actually aliases of the `writeU*()` functions, but they provide API consistency with the `readS*()` functions.)

Note: These aliases work because binary data is always stored as unsigned values; for instance, a single byte will always have a value in the range 0 to 255. The conversion to signed values is done when the data is read from a stream.

Conclusion

My goal with this short tutorial was to complement my previous article on creating binary files for your game's data with some examples of how to do the actual reading and writing. I hope it's achieved that; if there's more you'd like to know about the topic, please speak up in the comments!