sf package version create erroneous missing metadata #2455

ClayChipps · 2023-09-09T02:40:40Z

Summary

When using the sf package version create command with the following configurations:

CLI v2.9.2 Packaging v1.24.0
CLI v2.9.1 Packaging v1.24.0
CLI v2.7.11 Packaging v1.22.2
CLI v2.7.0 Packaging v1.22.2

The package version creation fails with a litany of errors indicating missing metadata, which in fact is not missing from the project.

Around 1PM EDT on 9/8/2023, we had successfully built this exact project using the following configuration, which succeeded:

CLI v2.7.0 Packaging v1.22.2

Around 5PM EDT on 9/8/2023, we attempted to build this exact same project using the same configuration, which failed:

CLI v2.7.0 Packaging v1.22.2

We were using the exact same Git ref for both of these builds, so absolutely nothing changed in the project structure.

We then attempted with the following configurations, all of which failed:

CLI v2.9.2 Packaging v1.24.0
CLI v2.9.1 Packaging v1.24.0
CLI v2.7.11 Packaging v1.22.2

Steps To Reproduce

Run the sf package version create command
Command fails with multiple errors

We cannot provide the repo as this is a client implementation that we cannot share.

Expected result

The package should have been built

Actual result

Multiple metadata elements are reported as missing, when they are in fact not

System Information

CLI v2.9.2 Packaging v1.24.0
1694228554943-diagnosis.txt
1694228554943-command-debug.log
1694228554943-command-stdout.log

    "cliVersion": "@salesforce/cli/2.9.2",
    "architecture": "win32-x64",
    "nodeVersion": "node-v18.14.2",
    "osVersion": "Windows_NT 10.0.19045",
    "shell": "C:\\Program Files\\Git\\usr\\bin\\bash.exe",
    "rootPath": "C:\\Users\\BCCA1JV\\AppData\\Local\\Programs\\node-v18.14.2-win-x64\\node_modules\\@salesforce\\cli",
    "pluginVersions": [
      "@dx-cli-toolbox/sfdx-toolbox-package-utils 0.8.4 (user)",
      "@oclif/plugin-autocomplete 2.3.8 (core)",
      "@oclif/plugin-commands 2.2.25 (core)",
      "@oclif/plugin-help 5.2.19 (core)",
      "@oclif/plugin-not-found 2.4.1 (core)",
      "@oclif/plugin-plugins 3.5.0 (core)",
      "@oclif/plugin-search 0.0.22 (core)",
      "@oclif/plugin-update 3.2.3 (core)",
      "@oclif/plugin-version 1.3.10 (core)",
      "@oclif/plugin-warn-if-update-available 2.1.0 (core)",
      "@oclif/plugin-which 2.2.32 (core)",
      "@salesforce/cli 2.9.2 (core)",
      "apex 2.3.14 (core)",
      "auth 2.8.16 (core)",
      "community 2.4.0 (user)",
      "data 2.5.8 (core)",
      "deploy-retrieve 1.17.9 (core)",
      "info 2.6.40 (core)",
      "limits 2.3.33 (core)",
      "login 1.2.29 (core)",
      "marketplace 0.2.0 (core)",
      "org 2.10.6 (core)",
      "packaging 1.24.0 (user)",
      "schema 2.3.25 (core)",
      "settings 1.4.28 (core)",
      "sobject 0.2.6 (core)",
      "source 2.10.33 (core)",
      "telemetry 2.3.1 (core)",
      "templates 55.5.11 (core)",
      "trust 2.6.9 (core)",
      "user 2.3.32 (core)"

CLI v2.9.1 Packaging v1.24.0
1694224776340-diagnosis.txt
1694224776340-command-debug.log.log
1694224776340-command-stdout.log.log

"cliVersion": "@salesforce/cli/2.9.1",
    "architecture": "win32-x64",
    "nodeVersion": "node-v18.14.2",
    "osVersion": "Windows_NT 10.0.19045",
    "shell": "C:\\Program Files\\Git\\usr\\bin\\bash.exe",
    "rootPath": "C:\\Users\\BCCA1JV\\AppData\\Local\\Programs\\node-v18.14.2-win-x64\\node_modules\\@salesforce\\cli",
    "pluginVersions": [
      "@dx-cli-toolbox/sfdx-toolbox-package-utils 0.8.4 (user)",
      "@oclif/plugin-autocomplete 2.3.8 (core)",
      "@oclif/plugin-commands 2.2.25 (core)",
      "@oclif/plugin-help 5.2.19 (core)",
      "@oclif/plugin-not-found 2.4.1 (core)",
      "@oclif/plugin-plugins 3.4.2 (core)",
      "@oclif/plugin-search 0.0.22 (core)",
      "@oclif/plugin-update 3.2.3 (core)",
      "@oclif/plugin-version 1.3.10 (core)",
      "@oclif/plugin-warn-if-update-available 2.1.0 (core)",
      "@oclif/plugin-which 2.2.32 (core)",
      "@salesforce/cli 2.9.1 (core)",
      "apex 2.3.14 (core)",
      "auth 2.8.16 (core)",
      "community 2.4.0 (user)",
      "data 2.5.8 (core)",
      "deploy-retrieve 1.17.8 (core)",
      "info 2.6.40 (core)",
      "limits 2.3.33 (core)",
      "login 1.2.29 (core)",
      "marketplace 0.1.3 (core)",
      "org 2.10.6 (core)",
      "packaging 1.24.0 (user)",
      "schema 2.3.25 (core)",
      "settings 1.4.28 (core)",
      "sobject 0.2.6 (core)",
      "source 2.10.33 (core)",
      "telemetry 2.3.1 (core)",
      "templates 55.5.11 (core)",
      "trust 2.6.9 (core)",
      "user 2.3.32 (core)"

CLI v2.7.11 Packaging v1.22.2
1694222844488-diagnosis.txt
1694222844488-command-debug.log.log
1694222844488-command-stdout.log.log

    "cliVersion": "@salesforce/cli/2.7.11",
    "architecture": "win32-x64",
    "nodeVersion": "node-v18.14.2",
    "osVersion": "Windows_NT 10.0.19045",
    "shell": "C:\\Program Files\\Git\\usr\\bin\\bash.exe",
    "rootPath": "C:\\Users\\BCCA1JV\\AppData\\Local\\Programs\\node-v18.14.2-win-x64\\node_modules\\@salesforce\\cli",
    "pluginVersions": [
      "@dx-cli-toolbox/sfdx-toolbox-package-utils 0.8.4 (user)",
      "@oclif/plugin-autocomplete 2.3.6 (core)",
      "@oclif/plugin-commands 2.2.24 (core)",
      "@oclif/plugin-help 5.2.18 (core)",
      "@oclif/plugin-not-found 2.4.0 (core)",
      "@oclif/plugin-plugins 3.3.2 (core)",
      "@oclif/plugin-search 0.0.22 (core)",
      "@oclif/plugin-update 3.2.0 (core)",
      "@oclif/plugin-version 1.3.9 (core)",
      "@oclif/plugin-warn-if-update-available 2.0.49 (core)",
      "@oclif/plugin-which 2.2.32 (core)",
      "@salesforce/cli 2.7.11 (core)",
      "apex 2.3.12 (core)",
      "auth 2.8.14 (core)",
      "community 2.3.14 (user)",
      "data 2.5.7 (core)",
      "deploy-retrieve 1.17.6 (core)",
      "info 2.6.40 (core)",
      "limits 2.3.32 (core)",
      "login 1.2.28 (core)",
      "marketplace 0.1.1 (core)",
      "org 2.10.5 (core)",
      "packaging 1.22.2 (user)",
      "schema 2.3.24 (core)",
      "settings 1.4.27 (core)",
      "sobject 0.2.6 (core)",
      "source 2.10.32 (core)",
      "telemetry 2.3.1 (core)",
      "templates 55.5.10 (core)",
      "trust 2.6.7 (core)",
      "user 2.3.31 (core)"

The text was updated successfully, but these errors were encountered:

github-actions · 2023-09-09T02:40:57Z

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

github-actions · 2023-09-09T02:41:00Z

Hello @ClayChipps 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

Make sure you've provided detailed steps to reproduce your issue.
- A repository that clearly demonstrates the bug is ideal.
Make sure you've installed the latest version of Salesforce CLI. (docs)
- Better yet, try the rc or nightly versions. (docs)
Try running the doctor command to diagnose common issues.
Search GitHub for existing related issues.

Thank you!

avesolovksyy · 2023-09-09T15:34:06Z

We are experiencing the same issue.
All package version creation commands in all our projects suddenly start failing since Friday's night (Sep 8) and continue doing so permanently when retrying.
Everything was just working fine till Friday, and some projects haven't got new incoming changes for a few days (neither metadata, nor CI configs), but still they are failing with some ridiculous errors mostly compilation related (like invalid field name in an object, missing report type etc., dozens of errors like that - despite that metadata is perfectly valid and has been successfully deployed into empty scratch org).
Seems like it is the global issue, maybe related to some release happening on Friday's night(

jeroen-binnekamp · 2023-09-09T22:42:12Z

We experience the same unexpected behaviour, except that package creation works but the created package won't install while no changes to the code since the previous package created earlier.

ClayChipps · 2023-09-10T00:27:54Z

Just as an update, we were able to get around this issue by using a much older version (1.17.1) of the packaging plugin.
Specifically, these configurations are tested as working in our scenario:

CLI v2.7.0 Packaging v1.17.1
1694300590857-command-debug.log
1694300590857-command-stdout.log
1694300590857-diagnosis.txt
CLI v2.7.11 Packaging v1.17.1
1694303000937-command-debug.log
1694303000937-command-stdout.log
1694303000937-diagnosis.txt

mshanemc · 2023-09-10T02:43:32Z

Is there any chance you have CustomIndex in your packaged metadata?

ClayChipps · 2023-09-10T02:48:50Z

Is there any chance you have CustomIndex in your packaged metadata?

A quick search of my codebase doesn't return any CustomIndex results. I should also clarify this is specifically for 2GP Unlocked Packages (in our scenario).

mshanemc · 2023-09-10T03:20:08Z

I believe all of you. FWIW, there was definitely something bad with packaging on Sept 7
This is the overall error rate for all versions of package:create. For context, there's about 12,000 calls of package version creates per day. If there's a bug, it may not be very widespread (ex: it's very specific to certain metadata being packaged that's now failing).

One hint is that the "much older 1.17.1" seems to work ok. I'd be interested in knowing if running the command on a newer plugin but with an --api-version might cause a similar improvement.

jeroen-binnekamp · 2023-09-10T05:59:33Z

Hello, I have reported before we are having similar problems. I have found the cause in our situation, unsure if this is really related to this Issue or maybe to #2448.

In our metadata we have a Custom Object named Finance_Account__c that has a RecordType named Profit_Loss. In this RecordType we have an & in the label:

<label>Profit & Loss</label>

when trying to build a package with this label we get amongst all other errors:

(63) Finance_Account__c: Error parsing file: The entity name must immediately follow the '&' in the entity reference.

When I change the label to something like

<label>Profit and Loss</label>

everything works fine.

Now we have this label (with the &) in our code for quite some time and never had any problems creating packages because of this.

Hopefully this helps.

qwikag · 2023-09-10T06:55:53Z

@ClayChipps, did you/can you provide the actual errors you are receiving and examples of metadata, to compare.
also package XML and files naming too...
Salesforce has many work arounds in effect on many different metadata types and as they fix I have historically seen old ways of doing things becomes technically incorrect as they fix their old mistakes. FOr exmaple there are many naming clashes currently in play.

More detail the better. :)

Maybe provide some examples that are non descript, and do not impact your customer. I am sure they want you operating efficiently.

avesolovksyy · 2023-09-10T12:30:04Z

@mshanemc @qwikag
Wouldn't it be a wise decision to revert recently released changes to packaging plugin to unblock people, and then continue doing debugging/investigation with some beta/pre-released version in separate branch or whatever if it becomes obvious that it has introduced some regression issues.
We are happy to help, but don't want to be a 'guinea pig' while our teams of devs and qas are waiting for new package version to be created.

In our case, it is also about unlocked packages, some of which really have customIndex packaged, but some of them not - all are still failing.

Are there any workaround, i.e. how can we use specific version of packaging plugin (e.g. older one) in CI/CD?
Currently we are just installing sf cli with a single command on our pipelines:
install @salesforce/cli@latest --global --loglevel verbose

As I understand due to lazy loading approach, packaging plugin is actually installed implicitly when we first issue 'package version create' command. Even if we provided specific version of cli itself, it would still install latest version of packaging plugin implicitly on first call. Any advice/suggestion here?

ClayChipps · 2023-09-10T12:36:23Z

@avesolovksyy You can install a specific plugin version like so sf plugins install @salesforce/[email protected]

avesolovksyy · 2023-09-10T14:39:49Z

@ClayChipps
Thanks, seems like it's working when we have installed explicitly packaging plugin of version 1.17.0 before issuing 'package version create' command.
At least package version has been created successfully, but not sure what further implications this may have down the line e.g. for install/upgrade and dependent packages.

mshanemc · 2023-09-10T20:30:43Z

Hey, all. We've got this updated, I think. Would love to hear confirmation that it's solved this problem.

avesolovksyy · 2023-09-10T22:56:10Z

@mshanemc
Thanks a lot.
We've created successfully package versions for 2 previously failing projects so far.

ClayChipps added the investigating We're actively investigating this issue label Sep 9, 2023

mshanemc mentioned this issue Sep 10, 2023

Sm/revert-xml-parsing-entities forcedotcom/source-deploy-retrieve#1109

Merged

shetzel closed this as completed in forcedotcom/source-deploy-retrieve#1109 Sep 10, 2023

This was referenced Sep 10, 2023

fix: bump SDR salesforcecli/plugin-deploy-retrieve#752

Merged

SF V2 - some HTML entities being incorrectly escaped during push #2448

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sf package version create erroneous missing metadata #2455

sf package version create erroneous missing metadata #2455

ClayChipps commented Sep 9, 2023 •

edited

Loading

github-actions bot commented Sep 9, 2023

github-actions bot commented Sep 9, 2023

avesolovksyy commented Sep 9, 2023

jeroen-binnekamp commented Sep 9, 2023

ClayChipps commented Sep 10, 2023

mshanemc commented Sep 10, 2023

ClayChipps commented Sep 10, 2023 •

edited

Loading

mshanemc commented Sep 10, 2023 •

edited

Loading

jeroen-binnekamp commented Sep 10, 2023 •

edited

Loading

qwikag commented Sep 10, 2023

avesolovksyy commented Sep 10, 2023

ClayChipps commented Sep 10, 2023 •

edited

Loading

avesolovksyy commented Sep 10, 2023

mshanemc commented Sep 10, 2023

avesolovksyy commented Sep 10, 2023

sf package version create erroneous missing metadata #2455

sf package version create erroneous missing metadata #2455

Comments

ClayChipps commented Sep 9, 2023 • edited Loading

Summary

Steps To Reproduce

Expected result

Actual result

System Information

github-actions bot commented Sep 9, 2023

github-actions bot commented Sep 9, 2023

avesolovksyy commented Sep 9, 2023

jeroen-binnekamp commented Sep 9, 2023

ClayChipps commented Sep 10, 2023

mshanemc commented Sep 10, 2023

ClayChipps commented Sep 10, 2023 • edited Loading

mshanemc commented Sep 10, 2023 • edited Loading

jeroen-binnekamp commented Sep 10, 2023 • edited Loading

qwikag commented Sep 10, 2023

avesolovksyy commented Sep 10, 2023

ClayChipps commented Sep 10, 2023 • edited Loading

avesolovksyy commented Sep 10, 2023

mshanemc commented Sep 10, 2023

avesolovksyy commented Sep 10, 2023

ClayChipps commented Sep 9, 2023 •

edited

Loading

ClayChipps commented Sep 10, 2023 •

edited

Loading

mshanemc commented Sep 10, 2023 •

edited

Loading

jeroen-binnekamp commented Sep 10, 2023 •

edited

Loading

ClayChipps commented Sep 10, 2023 •

edited

Loading