What Is Shader Cache

Hi the shader cache location seems to have changed in the latest drivers and I can't find it, I use it for a few games copying the shader cache files and then putting them back in when I next want to launch the games making them stutter less. Oct 29, 2019. However, if you believe your DirectX Shader Cache is corrupt or too large, you may delete it. The stuff in it is permanently deleted - but the cache will regenerate & fill again. It may take a reboot to get it back, though. I ran DxDiag (ask Cortana) & viewed videos on the WEB.

DDC stands for Derived Data Cache.

Many Unreal Engine Assets require additional 'derived data' before they can be used. (A simple example would be a Material that has a shader). Before the Material can be rendered, the shader must be compiled for the platform the editor is running on.

Because the derived data is large, and at times may need regenerating, it is not checked into source control. Instead it's kept in the Derived Data Cache (DDC).

Where the DDC Is Stored

Depending on how your project and system are configured, there can be several DDC caches in a hierarchy that ranges from fast to slow. When assessing derived data, your system will do the following to determine how quickly it can access it:

  1. When a piece of derived data is needed, the fastest cache is checked first, then the next fastest, and so on, until the data is found.

  2. When the data is found, it is copied into the fastest local cache so that it is quicker to access next time.

  3. If the data is not found, then it is first generated, then asynchronously copied into the caches so it is available for you (and potentially your team) in the future.

Content stored in the DDC is disposable, and can be regenerated at any time using the data stored in the .uasset file. Storing these derived formats externally makes it possible to easily add or change the formats used by the engine without needing to modify the source Asset file.

Types of DDCs

Typically, you will have at least the following two caches:

  1. Boot DDC: Startup files that are loaded into memory to increase boot times.

    1. For Example: ProjectDir/DerivedDataCache/Boot.ddc

  2. Local DDC: Derived data for your projects.

    1. For Example: EngineDir/DerivedDataCache

However, your project may also be set up to use a DDC Pak or Shared DDC.

DDC Pak

If you download Unreal Engine from the Epic Games Store, the engine will come with a DDC Pak (.ddp). The DDC Pak contains derived data for all engine content, so you can start working without compiling shaders and so on. Similarly, some samples are shipped with a DDC Pak for the same reason.

  1. Engine DDC Pak

    1. For Example: EngineDir/DerivedDataCache/Compressed.ddp

  2. Project DDC Pak

    1. For Example: ProjectDir/DerivedDataCache/Compressed.ddp

Shared DDC

One of our strongest recommendations for teams in the same location is that they set up a shared DDC. This is a network drive that all team members and build machines can read/write to. This then writes off the cost of creating any required DDC data across the entire team. For example, when an artist edits a shader, the DDC data will be written straight to the DDC share.

  • Shared DDC: Will be either a network or mapped drive.

An example of a shared drive would be: epicgames.netrootDDC-Global-Fortnite

Using a Shared DDC

A studios should use a shared DDC that all users in a particular location can access. This way, only one person needs to build the derived Asset format(s) and they will be automatically available to all other users. There will occasionally be stalls when Assets need to be processed, but the results are stored and shared. Even on a fairly small team, sharing Asset processing work in this way will eliminate most processing time.

It is not advised to copy an entire DDC across the internet, or back up your DDC, or restore a DDC from a backup. While not necessarily harmful, this would be a waste of time—it takes longer to transfer the amount of data stored in the DDC than it does to generate it from scratch locally. If you have a large project and want to distribute pre-built DDC data, you should generate a DDC Pak.

Setting Up Shared DDC

As you may see in BaseEngine.ini, the editor is already set up to use a Shared DDC that can be enabled one of three ways.

Here are the options from most to least recommended.

  1. Add an override (as pictured below) in the DefaultEngine.ini for your project that sets the path to a valid location for your team.

    [DerivedDataBackendGraph] Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=YourCompanyServerDDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)

  2. Set an environment variable (as pictured below) of UE-SharedDataCachePath (On Mac/Linux UE_SharedDataCachePath) to the folder to use.

  3. In the editor, set the SharedDerivedDataCache variable.

Click image for full size.

You don't have to do anything special to YourCompanyServerDDC; as long as it is writable, the engine will create the structure it needs.

Disabling a Shared DDC

Developers working remotely with a project that has a shared DDC configured may experience poor performance due to the fact that it takes longer to access DDC data than it does to generate it. To temporarily disable a shared DCC, use one of the methods below:

  • Pass -ddc=noshared on the command line.

  • Set the environment variable to a local drive hard drive:

    • UE-SharedDataCachePath=None

    • On Mac: UE_SharedDataCachePath=None

Building Derived Data

The user who imports the Asset is the one who builds the derived data since they will most likely be using and testing that Asset in the Engine. However, there may be times when a new Asset needs to be processed. This happens automatically on an as-needed basis and should not result in much of an impact when running on fast hardware, though there may be occasional stalls.

You can fill your DDCs at any time by running the following command:

Epic Games does this on a nightly basis to ensure that the DDC is always primed, but it is not necessary as the general automatic caching feature should suffice.

Mounting Your DDC Folder

For the DDC to be used over your internal network, the folder that contains the DDC must be mounted to a network drive. If you are unfamiliar with how to do this, see Map A Network Drive on Windows.

Distributing with a DDC

Cooking is the preferred method for packaging games since cooked builds do not need or use a DDC. However, DDCs can be packaged for distribution if the need arises.

Cooking is the preferred method for packaging games since cooked builds do not need or use a DDC. However, DDCs can be packaged for distribution if the need arises.

To package a DDC:

  1. Run UE4Editor.exe from the UE4/Engine/Binaries/Win64 directory, passing the arguments shown below:

    UE4Editor.exe ProjectName -run=DerivedDataCache -fill -DDC=CreatePak

  2. This creates a DDC.ddp file in the UE4ProjectNameDerivedDataCache directory.

  3. The engine automatically detects and uses the .ddp file.

DDC Settings

Where are DDC settings kept?

DDC settings are kept in DefaultEngine.ini under the [DerivedDataBackendGraph] section. You can view the defaults in BaseEngine.ini.

Common Questions

Q: Can I have multiple DDC settings?

Yes! Just create a new [YourDDCSettings] entry in DefaultEngine.ini and then run the editor with -ddc=YourDDCSettings.

At Epic, we do this for three reasons:

  1. For teams in offices that want to use their own DDC instead of the global one.

  2. When creating DDC Paks we use one of several options that control what goes into the .pak file, such as [CreateInstalledEnginePak].

  3. For people working from home who don't want to use a Shared DDC due to slower internet or not having VPN enabled.

You'll see that BaseEngine.ini contains several DDC entries by default, including NoShared, which is used for #3 above (-ddc=NoShared).

Q: I am short on disk space. Can I move the Local DDC elsewhere?

Yes! Just set an environment variable of UE-LocalDataCachePath to a path of your choice. For example: UE-LocalDataCachePath=d:DDC.

  1. At the command prompt, type 'setx UE-LocalDataCachePath d:DDC'.

  2. Restart Unreal Engine, along with any application you launch it from, such as Epic Launcher, UGS, or Visual Studio.

This can be used when you are short of space on your primary drive, or when you are working on multiple branches and wish to ensure the data isn't duplicatednote

Q: I have a slow network connection. Can I turn off the Shared DDC?

Yes! There are two options, depending on whether you want this to be temporary or permanent:

  • If you set the environment variable for a DDC to None it will disable it. In this case, you would set UE-SharedDataCachePath=None.

  • Launch the editor with -ddc=noshared.

Q: Can I change the Shared DDC path?

Yes! You can change it by setting the UE-SharedDataCachePath mentioned above.

However, you should never set UE-SharedDataCachePath to a local path! Doing this means you would have both Local and Shared caches on your machine—twice the disk space for zero gain!

Q: How can I diagnose DDC issues?

If you think your editor is not correctly reading DDC data, then search your logfile for LogDerivedDataCache.

Here you can see:

  • There's no project or Engine Pak cache, which is what would be expected as I compiled this build myself from Perforce.

  • My local data cache path is mapped to d:DDC and is writable.

  • I am using the Epic shared cache, which is writable.

You can also run with -logcmds='LogDerivedDataCache Verbose' to turn on verbose logging.

Q: What is the S3 DDC?

The Fortnite DDC is extremely large due to the amount of content we have. When Covid19 struck, we quickly realized that it was very inefficient for developers to pull all this data from a network share over the limited bandwidth of a VPN to the office, so instead we distribute the most commonly used data via S3.

Setting up the engine to use the S3DDC is extremely easy, but you will need to set up automation that generates the DDC content and uploads it to S3.

Q: What is IDDCUtilsModuleInterface?

This interface allows you to use ? Name arguments in your DDC settings that can then be resolved dynamically at runtime based on platform or domain.

Unfortunately, we don't ship an example of this, but implementing it is straightforward:

Click image for full size.

Q: How do I create a DDC Pak?

Creating a project-specific DDC:

  • UE4Editor.exe ProjectName -run=DerivedDataCache -fill -DDC=CreatePak

Direct Shader X Files

What is directx shader cache

This commandlet will create a DDC for all content in the project. Alternatively, you can simply supply -DDC=CreatePak while running automation or even a user session to generate a more targeted set of content.

Creating an Engine DDC:

What Is Directx Shader Cache

  • UE4Editor.exe -run=DerivedDataCache -fill -DDC=CreatePak