Using a properly setup Perforce to handle Unreal Engine streams can be a boon to productivity. This article explores how to set this up using Perforce best practices and what I've learned over the years. Please email me any questions and/or clarifications that you require.
The example used here is UE4, but the same principles apply to UE5. The only difference is trimming the footprints at the bottom of this article.
Setting up the Depot
The best process I've found here is to go two deep when creating the depot. This allows categorization of the streams and prevents an explosion of streams in the 'root' folder.
From Helix Admin:
The key part to note is 'Stream Depth' needs to be at least 2. For the example given of UE4/1/2
Whenever you create a new branch, add it under the correct category. To my knowledge you cannot change the category of an existing branch after it has been created. The only way to move it is to obliterate and then recreate.
After some months of work and general development, you should end up with something like the following:
You'll note that the 'Task-Linux-GL' branch was created in the wrong location.
Tips for branch creation:
Adding UE4 to Perforce
The default install starts off hidden in the bowels of Windows and the projects are created in the 'My Documents' folder. This does not facilitate storing in Perforce or a reasonable branching strategy. The only way this is feasible is if you never make any engine code changes and only make content.
Set up a decent typemap. Typemap (GitHub link)
Make a new workspace (using your own conventions) and put the workspace root at the root of one of your drives. My convention is to use the machine name and branch name with the root folder being named the same. e.g. the MACHINE-Main workspace resides in d:\MACHINE-main.
Copy the contents of the UE4/Engine folder and the files in the root of the UE4 folder to your workspace root after setup.bat has been run, but before any compilation has been done. Mark these files for add and submit them. Generate project files, compile and run the engine. Generate a new project for your content and store it in the root of your workspace. I'd also recommend adding a TestBed project for experimentation and throwaway content. After all this, you should end up with something like:
d:\MACHINE-Main
d:\MACHINE-Main\Engine\
d:\MACHINE-Main\TestBed\
d:\MACHINE-Main\CoolNewGame\
d:\MACHINE-Main\GenerateProjectFiles.bat
d:\MACHINE-Main\UE4Games.uprojectdirs
Workflow
Perforce best practices state that you should merge down and copy up. There is plenty of literature online about this so I won't delve too deep here.
Given:
Trimming the Perforce footprint (UE4.27.2)
UE4 comes with all the code for all editors and all platforms (except the targets that require a special license). I have no intention of targeting certain platforms or using a Mac editor, so I trim these source files out. This cuts down the the number of files in the depot by about a third, speeds up editor loading time, and ensures no-one goes down any blind alleys. Not everyone has an Epic machine, so every little bit of performance helps. The downside is that engine changes are not tested on platforms you've disabled, but if you're never going to target that platform, who cares?
This method uses Perforce Virtual Streams to mask out the unwanted files gracefully. The UnrealBuildTool project file is changed significantly, but otherwise there is only an accessor added to the code. Several build cs files also had to be updated to make the changes forward and backward compatible.
Setup virtual streams
Virtual streams allows restriction of the workspace view i.e. allows masking out of certain files. Virtual Streams. To this end, they can be set up to mask out platform specific files.
Sample virtual stream ignore set for HoloLens (GitHub link), Apple (GitHub link), and Android (GitHub link)
Implement code changes
All changes are wrapped in the comments // ETERNAL BEGIN and // ETERNAL END
Add the following five files to //Engine/Source/Programs/UnrealBuildTool/Platform
The NoApple and NoHoloLens branches will generate project files, compile, and run without issue.
The NoAndroid branch requires the following C++ changes to generate project files, compile, and run. The changes purely comment out Android specific code.
The end result of masking out these five platforms in reducing the engine folder from about 36GB to about 19GB.
File size | File size delta | File count | File count delta | |
Vanilla | 36,331,715,884 | 0 | 178,257 | 0 |
No Apple | 24,980,271,332 | 11.3GB | 168,625 | 9,632 |
No Android | 32,100,783,594 | 4.2GB | 174,034 | 4,223 |
No HoloLens | 35,132,865,156 | 1.2GB | 177,490 | 767 |
Trimming the Perforce footprint (UE5.2.1)
UE5 has made some noticeable improvements to UBT:
Setup virtual stream
Virtual streams allows restriction of the workspace view i.e. allows masking out of certain files. Virtual Streams. To this end, they can be set up to mask out platform specific files.
Sample virtual stream ignore set for Mac, iOS, Android, HoloLens, TVOS (GitHub link)
Implement code changes
All changes are wrapped in the comments // ETERNAL BEGIN and // ETERNAL END
Add the following three files to //Engine/Source/Programs/UnrealBuildTool/Platform (Lumin is deprecated and HoloLens is automatic)
Update the following 8 files to get UnrealBuildTool to compile (this is less than ideal)
Fix the engine compile issues with the following 4 files
To get the tools to compile
The end result of masking out these five platforms in reducing the engine folder from about 57GB to about 44GB.
File size | File size delta | File count | File count delta | |
Vanilla | 56.8GB | 0 | 255,734 | 0 |
Trimmed | 43.5GB | 13.3GB | 232,287 | 23,447 |
There seems to be some effort to automatically populate the UnrealTargetPlatform enum in UBT with discovered platforms. This would be awesome! For example, if UnrealTargetPlatform.HoloLens existed in UBT, it could be checked in the various Build.cs files to include only the relevant files and reduce the number of include folders that needed to be searched. It could also be used to generate a preprocessor define (e.g. UE_BUILD_TARGET_HOLOLENS) that could be used to add HoloLens specific UI only if that target was available. Eventually, Unreal Engine could have an installer that allows you to only install the components you wish (a bit like the Visual Studio installer). Which editors do you want? (Windows, Mac, Linux) and which targets do you want? (Windows, Xbox8, Mac, iOS, Android, PS9, etc.)
In an ideal world
Future work