BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs #5356
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
During the Incremental build GenerateByteArrayValue used to generate the ByteArrayValue even when there is no change in the PCD/VPDs. which is time consuming API based on the number of PCD/VPDs and SKU IDs. The optimization is that GenerateByteArrayValue is used to store the StructuredPcdsData in a JSON file for each of the arch. and during the Incremental build this API will check, if there is any change in the Structured PCD/VPDs then rest of the flow remains the same. if there is no change then it will return the provious build data. Flow: during the 1st build StructuredPcdsData.json is not exists, StructuredPcdsData will be dumped to json file. and it will copy the output.txt as well. Note: as the output.txt are different for different Arch, so it will be stored in the Arch folder. During the Incremental build check if there is any change in Structured PCD/VPD. if there is a change in Structured VPD/PCD then recreate the StructuredPcdsData.json, and rest of the flow remains same. if there is no change in VPD/PCD read the output.txt and return the data Unit Test: Test1: Modified the Structured Pcds default from DEC file. current flow is executing. Test2: Override the default value of the PCD from DEC file. current flow is executing. Test3: Modified/Override the PCD from DSC file. current flow executing Test4: Modified/Override the FDF from DSC file. current flow executing Test5: update the default value from Command Line.current flow executing Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the proposed changes will be executing, and the return data remains the same with and without the changes. Test7: Build with and without modified the include headers of Structured PCDs. if there is any change in those Structured PCD header then current flow will be executed. With these changes it's helping to save around ~2.5min to ~3.5min of Incremental build time in my build environment. Sample PR: tianocore/edk2-basetools#113 Cc: Yuwei Chen <[email protected]> Cc: Rebecca Cran <[email protected]> Cc: Liming Gao <[email protected]> Cc: Bob Feng <[email protected]> Cc: Amy Chan <[email protected]> Cc: Sai Chaganty <[email protected]> Cc: Digant H Solanki <[email protected]> Signed-off-by: Ashraf Ali S <[email protected]> Reviewed-by: Yuwei Chen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During the Incremental build GenerateByteArrayValue used to generate the ByteArrayValue even when there is no change in the PCD/VPDs. which is time consuming API based on the number of PCD/VPDs and SKU IDs.
The optimization is that GenerateByteArrayValue is used to store the StructuredPcdsData in a JSON file for each of the arch. and during the Incremental build this API will check, if there is any change in the Structured PCD/VPDs then rest of the flow remains the same. if there is no change then it will return the provious build data.
Flow:
during the 1st build StructuredPcdsData.json is not exists, StructuredPcdsData will be dumped to json file. and it will copy the output.txt as well.
Note: as the output.txt are different for different Arch, so it will be stored in the Arch folder.
During the Incremental build check if there is any change in Structured PCD/VPD. if there is a change in Structured VPD/PCD then recreate the StructuredPcdsData.json, and rest of the flow remains same. if there is no change in VPD/PCD read the output.txt and return the data
Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow is executing.
Test2: Override the default value of the PCD from DEC file. current flow is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing Test4: Modified/Override the FDF from DSC file. current flow executing Test5: update the default value from Command Line.current flow executing Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the proposed changes will be executing, and the return data remains the same with and without the changes.
Test7: Build with and without modified the include headers of Structured PCDs. if there is any change in those Structured PCD header then current flow will be executed.
With these changes it's helping to save around ~2.5min to ~3.5min of Incremental build time in my build environment.
Sample PR: tianocore/edk2-basetools#113
Cc: Yuwei Chen [email protected]
Cc: Rebecca Cran [email protected]
Cc: Liming Gao [email protected]
Cc: Bob Feng [email protected]
Cc: Amy Chan [email protected]
Cc: Sai Chaganty [email protected]
Cc: Digant H Solanki [email protected]
Reviewed-by: Yuwei Chen [email protected]