If I were to write a game engine today ...

These are some musings on writing an engine in the modern era should I happen to have a couple of hundred million dollars burning a hole in my pocket.

Introduction

THIS DOCUMENT IS VERY MUCH WORK IN PROGRESS!
Some of these items will appear minor and or irrelevant, but are listed here anyway. Some other items are self evident and no rational being could do it any other way. These are musings, and by no means an even remotely full engine design. Items marked in [] are bullet points to be expanded into full sentences.

Goals

  • Iteration time is absolutely paramount.
  • The client install should be small and quick to generate.
  • The engine should be modular and API driven.
  • It should be easy to add/ignore platforms and modules.
  • There is extensive support for procedural meshes and textures.
  • The engine should be deterministic for use in simulations.
  • A C++ core (for performance) with a C# scripting language (for ease of use).

    • Direction

  • There should be a mantra of read anything, write a single correct run time format.
  • The source assets should be kept distinct from run time assets.
  • The asset name should match the path name.
  • All modules should have built in tests to make sure they are functioning correctly, and more extensive tests on request (to be run overnight or on check-in).
  • All strings and localization should be UTF-8. Everything is case sensitive.
  • Configuration files are in json.
  • File extensions are presumed.

    • Implementation

    C# scripting.

    The biggest issue is C++ calling C# functions - this is not easy. However, there is no reason that there couldn't be a header file generated using reflection of public members in assemblies. I'm currently experimenting with this in a side project. There would need to be automatic conversions between managed and native memory in the header file to make this practical.

    C# is a modern language that is performant enough for most tasks and very debuggable. It has many threading capabilities built in and is easy to read. Attributes can be used to decorate fields and functions to note extra functionality for the core engine.

    API Driven.

    Interface classes are the way to go; pure in C++ and an actual interface class in C#. This virtual function approach avoids excessive inlining leading to excessive compile times and client code bloat.

    Folder Layout - Code

    There is a tendency to mix editor and run time content together resulting in a client footprint that is larger than it needs to be. To resolve this, there should be editor source and run time source. Furthermore, the source should be split per platform as much as possible. There are only three practical editor platforms at the moment; Win-x64, Mac-x64, and Linux-x64.
    Having the editor resources local to the plugin like this means the UI cannot display options that can't be used; if it doesn't exist, it can't be misused!
    
	Compression/LZMA/
	Compression/LZMA/Editor/
	Compression/LZMA/Editor/API/		// The API required by the editor. The Editor API is free to call client API methods.
	Compression/LZMA/Editor/Common/		// Code common to all platforms but only used in the editor
	Compression/LZMA/Editor/Win-x64/	// Code specific to Windows x64
	Compression/LZMA/Editor/Mac-x64/
	Compression/LZMA/Editor/Resources	// Resources used by the editor (PNG format preferred)
	Compression/LZMA/Runtime/
	Compression/LZMA/Runtime/API/		// The API required by the client
	Compression/LZMA/Runtime/Common/	// Code common to all platforms that is the minimal set required for the client
	Compression/LZMA/Runtime/Android/	// Code specific to Android
	Compression/LZMA/Runtime/XboxOne/	// There are more client/run time platforms than editor platforms.
	Compression/LZMA/Runtime/Win-x64/
	Compression/LZMA/Runtime/Mac-x64/
    If the Compression/LZMA folder does not exist, then everything will compile fine, but the API will not have any implemented functions. This could be done with a static initializer registering the availability of LZMA. For most third party software, the common folders will contain the vast majority of the code, and the platform specific folders will effectively be build instructions. [Use folders to infer platforms, or config file?]
    [Plugin reference counts]

    Folder Layout - Assets

    Following a similar concept to the code, the source and run time assets should be kept distinct. Any update to the SourceAssets should be automatically updated in the parallel RunTime folder for the appropriate platform(s). Packaging a game then becomes a matter of iterating over all the platform folders and copying the files.
    The meta data for each asset should be part of the run-time version of the data. Everything should be self contained as much as possible. The editor version of the asset should have the meta data as a json file with strong versioning. If the version of the editor meta data changes, the run-time asset data is regenerated.
    Each asset should have a parallel json file listing the assets that reference it and the assets it references. If there are no assets referencing it, the run-time version can be deleted. Using a json format for this allows external apps to analyze the hierarchy for profiling purposes.
    
	ProjectName/SourceAssets/Meshes/A/B/SimpleMesh.fbx
	ProjectName/RunTimeAssets/Win-x64/Meshes/A/B/SimpleMesh.runtime
	ProjectName/RunTimeAssets/Android/Meshes/A/B/SimpleMesh.runtime
	ProjectName/RunTimeAssets/XboxOne/Meshes/A/B/SimpleMesh.runtime

    Solution generation.

    Using the above folder structure, there would need to be a program that iterates over the available folders for the available modules to create a Visual Studio solution.

    Json configuration files.

    XML files would work here too, but it is critical that the format of the file can be validated independently and there is no ambiguity. Using a standard format like this allows third party apps to be written that can audit assets.

    Localization/Internationalization.

    Localization - all in UTF8 for reasons listed here. All locales should follow the language-region schema defined by the IETF language tag. e.g. en-US, de-DE, af-ZA. Typically, language-region definition is overkill, but this could be extended even further if necessary. ICU is also overkill for a game engine. Using string interpolation allows the localizer to do whatever is needed to make the grammar correct. e.g. $"Congratulations! You got {KillCount} kills!"

    Configurations

  • Debug - The slowest but most debuggable with all the sanity checks and tests enabled.
  • Dev - The typical use case for programmers and editors. Code optimization and sanity checks are enabled. Stats and debugging tools are enabled.
  • Ship - The slower and more thorough code optimizations are enabled. There are no sanity checks, tests, nor debugging tools.
  • QA - This is very similar to Ship, but has just enough of the debugging tools available for QA to thoroughly test.

    • Targets

  • Editor - The editor build to design and create assets.
  • Client - The client build to run a game.
  • Server - A dedicated server.

    • Rendering System!

    Audio System!