-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(GH-68) Move Puppetization Automation helper functions into the module & document usage #189
(GH-68) Move Puppetization Automation helper functions into the module & document usage #189
Conversation
This commit updates the util functions based on needs from latest release cycle. Future commits will pull each of these functions into the Puppet.Dsc module and add documentation/unit tests.
Prior to this commit, several functions for interacting with versions were instantiated and used in the util script file; this commit extracts those functions into the module as private functions, adding testing and documentation.
This commit moves the `Get-PowerShellDscModule` function from the util script into the public functions folder of the module, making it available to users. This commit adds unit testing and documentation for the function, increasing safety over dot-sourcing the util script.
This commit adds the ability to return all modules in a given namespace on the forge with the `Get-ForgeModuleInfo` public function, replacing the functionality previously found in the `Get-ForgeDscModules` function in the util script. Calling `Get-ForgeModuleInfo` without any parameters now returns all of the Puppet modules in the DSC namespace by default (with a pagination of 5) and only specified modules if the `Name` parameter is used. Updates tests and documentation for new functionality.
This commit moves the `Export-PuppetModule` function from the Util script into the module as a public function. This does minimal handling of calling `pdk build` against a Puppet module for the purposes of our automation. Includes unit tests and documentation.
This commit refactors the `Publish-PuppetModule` public function to include the enhancements from its reimplementation in the Util script as `Publish-PuppetDscModule` and to call the new `Export-PuppetModule` public function when needed. This commit also updates the function to use the `FORGE_TOKEN` environment variable as a default forge token if one is not specifically passed. The updated function is a nearly full-rewrite of the original implementation with updated tests and documentation.
This commit moves the `Get-UnpuppetizedDscModuleVersion` function from the Util script into the module as a public function, adding tests and documentation. In the process, it refactors the function for some UX improvements, enabling users to specify the PowerShell repository to check as well as the Forge URI and NameSpace to compare against. It also adds two options for filtering the unpuppetized versions - `OnlyNewer`, which returns only the versions found in the PSRepository for each module which are newer than the newest version on the Forge; and `MinimumVersion`, which filters out any versions older than the version specified. In the process of implementing this function, it was necessary to refactor the `Get-PowerShellDscModule` function to take the `Repository` parameter for searching repositories other than the default. Further, this function was refactored to write verbose messaging for the queries being executed against the PowerShell repository and to add some error handling to ensure that errors for failed queries are emitted without stopping processing *and* only successful queries write output to the pipeline. In the process of implementing `Get-UnpuppetizedDscModuleVersion`, it was also necessary to modify the `Get-ForgeModuleInfo` function to improve error handling, messaging, and logging for the same reasons as with `Get-PowerShellDscModule` above.
This commit refactors the Forge-related parameters in the Get- ForgeModuleInfo function to: 1. Make the `ForgeNameSpace` parameter mandatory and drop the default value which was Puppet-specific and therefore not useful to users. 2. Rename the `ForgeUri` parameter to `ForgeSearchUri` for the sake of clarity and parity with the similar parameter in Publish-PuppetModule This commit also updates the tests to match.
This commit refactors the `ConvertTo-VersionBuild` function to ensure that it accurately splits the specified version and Puppet build (previously, any build number greater than 9 would break the output) as well as properly sort the versions (previously, versions were sorted as strings which meant `1.10.0` would be lower in the list than `1.2.0`).
This commit explicitly adds support for `ShouldProcess` to the `Publish-PuppetModule` function, which was already implied by the existance of the `Force` parameter.
This commit refactors the Forge-related parameters in the Get- UnpuppetizedDscModuleVersion function to: 1. Make the `ForgeNameSpace` parameter mandatory 2. Rename the `ForgeUri` parameter to `ForgeSearchUri` for the sake of clarity and parity with `Get-ForgeModuleInfo` This commit also updates the tests to match.
This commit moves the `Update-ForgeDscModule` function from the Util script into the module, refactoring it to take advantage of prior updates and improvements to the module functions, as well as implement some minimal filtering and quality-of-life improvements. Includes documentation and tests.
This commit moves the `Publish-NewDscModuleVersion` function from the Util script into the module as a public function, refactoring it for non-Puppet employee workflows and bringing it into parity with other existing functions. This commit also adds unit tests and documentation for the function.
Manually updating existing Puppetized DSC ModulesYou can run the following command to update ever version of every module in the $UpdateParameters = @{
ForgeNameSpace = 'dsc'
ForgeToken = 'my_forge_token'
BuildFolderPath = 'C:\dsc'
PackageFolderPath = 'C:\pkg'
}
Update-ForgeDscModule @UpdateParameters However, as this is hundreds of versions across dozens of modules, it's very slow. $UpdateParameters = @{
ForgeNameSpace = 'dsc'
ForgeToken = 'my_forge_token'
BuildFolderPath = 'C:\dsc'
PackageFolderPath = 'C:\pkg'
# This ensures we don't rebuild dozens of releases for
# each module, only the latest 5
MaximumVersionCountToRebuild = 5
ErrorAction = 'Stop'
}
$ModulesToUpdate = Get-ForgeModuleInfo -ForgeNameSpace 'dsc' | Select-Object -ExpandProperty Name
"Number of Modules to Update: $($ModulesToUpdate.Count)"
Update-ForgeDscModule @UpdateParameters -Name $ModulesToUpdate[0..9]
Update-ForgeDscModule @UpdateParameters -Name $ModulesToUpdate[10..19]
Update-ForgeDscModule @UpdateParameters -Name $ModulesToUpdate[20..29]
# Continue until out of modules to update Manually publishing missing versions of PowerShell modules with DSC Resources to the ForgeYou can run the following command to Puppetize and publish every version of every PowerShell module with DSC Resources found on the PowerShell Gallery and not in the $PublishNewVersionParameters = @{
ForgeNameSpace = 'dsc'
ForgeToken = 'my_forge_token'
BuildFolderPath = 'C:\dsc'
PackageFolderPath = 'C:\pkg'
}
Publish-NewDscModuleVersion @PublishNewVersionParameters However, as this is hundreds of versions across more than 300 modules, it's very slow. $PublishNewVersionParameters = @{
ForgeNameSpace = 'dsc'
ForgeToken = 'my_forge_token'
BuildFolderPath = 'C:\dsc'
PackageFolderPath = 'C:\pkg'
# This ensures we don't try building older versions
# of already Puppetized modules
OnlyNewer = $true
# This ensures we only try to build 10 versions in
# a single execution, shrinking our batch processing
MaxBuildCount = 10
ErrorAction = 'Stop'
}
# Repeat until no new versions are published
Publish-NewDscModuleVersion @PublishNewVersionParameters If you encounter any errors, stop and then document and file a bug. |
I added the documentation here instead of anywhere long lived as we're planning to obviate it in the next ticket with automation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR moves the helper functions for automating the Puppetization of PowerShell DSC modules from the util script into the module itself as both private and public functions. In the process, it adds unit tests and documentation and refactors as necessary to improve UX and bring the module functions into alignment as described in detail in each commit.
TODO:
Update-ForgeDscModule
into the module as a public function with unit tests and documentationPublish-NewDscModuleVersion
into the module as a public function with unit tests and documentationUtil.ps1
script file