build.cmd does not regenerate managed PGO data on changes

[jakobbotsch]

StandardOptimizationData.mibc does not seem to be regenerated when it exists, even when the opt data package changes:

git checkout d435388^
rm .\artifacts\bin\coreclr\windows.x64.Checked\StandardOptimizationData.mibc
.\build.cmd clr -rc checked -lc release
(gci .\artifacts\bin\coreclr\windows.x64.Checked\StandardOptimizationData.mibc).Length
# prints 8000512
git checkout d435388
.\build.cmd clr -rc checked -lc release
(gci .\artifacts\bin\coreclr\windows.x64.Checked\StandardOptimizationData.mibc).Length
# prints 8000512 again
rm .\artifacts\bin\coreclr\windows.x64.Checked\StandardOptimizationData.mibc
.\build.cmd clr -rc checked -lc release
(gci .\artifacts\bin\coreclr\windows.x64.Checked\StandardOptimizationData.mibc).Length
# prints 9137664

d435388 is a commit where the PGO data changed significantly.

cc @davidwrighton @AndyAyersMS

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[AndyAyersMS]

Think this is a build system issue, somehow we need to understand that the optimization data is out of date -- basically the step that restores the data needs to be dependent on the props file or something, so it reruns if the props file changes.

[davidwrighton]

@jakobbotsch do the files in artifacts/mibc change across those builds? I believe our build infra is designed to detect changes to those, but I'm not confident about the bits in artifacts/mibc getting updated correctly.

[jakobbotsch]

@davidwrighton Yes, it seems like those files do change.

[jakobbotsch]

The problem seems to be that the timestamps of the files in artifacts/mibc are the original timestamps from inside the NuGet package. That means they are days in the past. Once we create the merged StandardOptimizationData.mibc, the timestamp becomes the time at which the tool was invoked. When we get newly optimized data in artifacts/mibc this still remains older than StandardOptimizationData.mibc, and thus it doesn't rerun.

Ideally the artifacts/mibc/**.mibc files would have their timestamp reset when they are placed, but that does not seem that easy to do while keeping incremental builds (it would be better if we could set the times after package restore has determined a new optimization data package was needed). Instead, I guess we can change the merged output .mibc file timestamp to be the same as the input files. Then at least dotnet-pgo will rerun automatically when one fetches newer commits (but potentially not if going back in history).