circle-loader
0
by
0/ 108/ /0

This time we will share a technical topics which are related to Program development. It is recommended to read for 15 minutes. Any unique insights or discoveries, please feel free to contact us or discuss.

UWA Q&A Community:answer.uwa4d.com


TextureStreaming

Q1: TextureStreamingJob crash on Android, iOS, and PC.

 

The call stack of crash is as follows:

Unity.exe!TextureStreamingJob(struct TextureStreamingJobData *)
Unity.exe!JobQueue::Exec(struct JobInfo *,__int64,int)
Unity.exe!JobQueue::Steal(class JobGroup *,struct JobInfo *,__int64,int,bool)
Unity.exe!JobQueue::ExecuteJobFromQueue(void)
Unity.exe!JobQueue::ProcessJobs(void *)
Unity.exe!JobQueue::WorkLoop(void *)
Unity.exe!Thread::RunThreadWrapper(void *)
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()

 

or on iOS:

1 app TextureStreamingJob (FloatConversion.h:127)

2 app Exec (JobQueue.cpp:412)

3 app Steal (JobQueue.cpp:673)

4 app ExecuteJobFromQueue (JobQueue.cpp:832)

5 app ProcessJobs (JobQueue.cpp:890)

6 app WorkLoop (JobQueue.cpp:976)

7 app RunThreadWrapper (Thread.cpp:76)

 

It’s similar on Android.

Here is a report which has submitted a Bug Report. After locating the problem site according to the call stack and analyzing the context, the pseudo code is roughly as follows:

void TextureStreamingJob(TexutreStreamingJobData* jobdata)
{
    // Above omitted
    auto& sharedData = jobdata->sharedData;
    auto smallestMip = jobdata->smallestMip;
    auto largestMip = jobdata->largestMip;
    count = sharedData.textures.size();
    
    auto& currBatchDesiredMipLevels = jobdata->results->desiredMipLevels[jobdata->batchIndex];
    
    if (count)
    {
        for (int i = 0; i < count; i++)
        {
            int8_t mipLevel = -1;
            
            if (sharedData.textures[i].unknownFloatValue >= 0.0)
                mipLevel = sharedData.textures[i].mipLevel;
            
            if (mip < 0)
                mipLevel = -1;
            
            if (mip >= smallestMip)
                mipLevel = smallestMip;
            
            if (mip <= largestMip)
                mipLevel = largestMip;
            
            currBatchDesiredMipLevels[i].mipLevel = mipLevel;    // << Crash here
            currBatchDesiredMipLevels[i].unknownField = CONST_VALUE;
        }
    }
    
    // The following is omitted...
}

The crash occurred in this sentence: currBatchDesiredMipLevels[i].mipLevel = mipLevel;.

 

According to the analysis, TextureStreamingManager will divide current tasks into groups according to streamingMipmapsRenderersPerFrame parameters in UnityEngine.QualitySettings.You can refer to: QualitySettings.streamingMipmapsRenderersPerFrame

 

For example, streamingMipmapsRenderersPerFrame = 2;

(1) If there are 2 streaming objects in the scene, there will be only one set of tasks (we will call it batchCount in the following, and the index is called batchIndex), that is, batchCount = 1;

 

(2) If there are 3 streaming objects, batchCount = 2, the number of tasks in the first group is 2, and the second group is 1;

 

(3) If there are 4 streaming objects, batchCount = 2, the number of tasks in the first group is 2, and the second group is 2;

 

(4) If there are 5 streaming objects, batchCount = 3, the number of tasks in the first group is 2, the second group is 2, and the third group is 1;

 

And so on.

 

The first set of tasks and the second one mentioned here are called batches. jobdata->results->desiredMipLevels is an array, which is stored by batch.

 

(1) jobdata->results->desiredMipLevels[

This shows that jobdata->results->desiredMipLevels.size() should be equal to batchCount.

// Type name 🙁
struct TextureMipLevelInfo;
// The engine has its own data structure dynamic_array, more intuitive to use std::vector to express the meaning
std::vector<std::vector<TextureMipLevelInfo>> desiredMipLevels;

According to the crash site, jobdata->results->desiredMipLevels[batchIndex] is NULL when the crash occurs. So further investigate the size() and capacity() of desiredMipLevels. The size is 1, and the capacity is 8.

 

jobdata->results->desiredMipLevels1, at this time, jobdata->results->desiredMipLevels.size() == 1

 

An out-of-bounds index has occurred, so the source of batchIndex needs to be analyzed. After some investigation, I learn that batchIndex comes from TextureStreamingManager::Update()

TextureStreamingManager::Update()
{
    // Above omitted
    if (this->jobBatchIndex > this->results->batchCount)
    {
        this->jobBatchIndex = 0;
    }
    // Omit
        TextureStreamingManager::InitJobData(/*Parameter omitted...*/)    // initialization this->jobData
        ScheduleJobInternal(this->jobFence, &TextureStreamingJob, this->jobData, 0);
    // Omit
}

It can be inferred that the status should be correct when the task is ScheduleJobInternal. If jobBatchIndex = 1 and this->results->batchCount = 1, then this->jobBatchIndex should return to zero.

 

Therefore, it can be inferred that after ScheduleJobInternal and before the execution of TextureStreamingJob, the task data has changed.

 

The batchIndex task will not be modified after initialization. If the state changes, it must be that this->results->batchCount has changed.

 

What kind of operations can affect batchCount, naturally comes to mind: streamingMipmapsRenderersPerFrame parameters and the number of mappings which need to participate in streaming.

 

With speculation, it needs to be further verified, and for reproduction, the following conditions need to be met:

(1) After the task data of TextureStreamingJob of each frame is initialized and before the task is actually executed, batchCount will reduce;

(2) BatchIndex during task initialization >= reduced batchCount.

 

After some trial and error, I choose a better solution:

(1) Make the execution timing of TextureStreamingJob controllable through hooks (at the end of each frame);

(2) Switch RenderersPerFrame by setting two Quality;

(3) Set smaller Renderers Per Frame for one QualitySettings, and set larger values for the other one;

(4) Put some Renderers participating in TextureStreaming in the scene to make batchCount just reach 2;

(5) Add a button and switch to QualitySettings with a larger Renderers Per Frame parameter when the button is clicked;

 

At the end of the frame when the button is clicked, TextureStreamingJob is executed-Crash!

Some supplements about the problem:

(1) Although the project used to reproduce the problem is triggered by modifying Renderers Per Frame, it can also be triggered without modifying it.

 

(2) In addition to the crash at the beginning of the TextureStreamingJob, there is also access to the desiredMipLevels at the back of the function, so there will also have a crash after the address.

 

(3) In TextureStreamingManager::InitJobData, the sharedData has a reference count plus one, and wherever it is used, it will be Unshare(), so it is not easy to cause problems, but the results are not. Although the results call Unshare(), the reference count is 1, in fact the JobSystem thread and the main thread accessed and modified the same TextureStreamingResults, which cause a bug.

 

At present, this problem has been reported to Unity, waiting for the official fix. It has been tested and exists in all current versions, 2018.4.1, 2019.1.

 

TextureStreaming can effectively reduce memory, but it has a lot of bugs and is known to have a crash. TextureStreaming currently conflicts with the QualitySettings.masterTextureLimit setting. Under certain circumstances, setting 1/2 will result in actually becoming 1/4 (effective twice). Switching textureStreamingActive will also produce some unexpected phenomena. If the project development is close to the later stage, it is not recommended to connect at this time.

Thanks to Zhuang Qin for providing the answer above.


This is the 71th UWA Technology Sharing for Unity Development. As we all know, Our life has a limit but knowledge has none. These problems are only the tip of the iceberg and there are more technical problems during our program development which deserve to discuss. Welcome to join UWA Q&A community, let‘s explore and share knowledge together!

UWA website:en.uwa4d.com

UWA Blog:blog.en.uwa4d.com

UWA Q&A community:answer.uwa4d.com

Post a Reply