Some Specific Wave metadata type Components are not added in Output directory #286

anish1310 · 2022-04-08T08:23:12Z

Issue verification check:

is the current repository fully deployable at the commit SHA provided with the 'from' parameter of the command?

What is the problem?

We are getting a problem with generating delta with source code in a separate folder for wave component (Tableau Components)

There are some wave component differences between the branches, and the package.xml for wave components generated after running the source:delta command is correct but it didn't add the following wave metadata type components in the output directory:

WaveApplication
WaveDataset
WaveXmd

What is the parameter and the value you used with it?

sfdx sgd:source:delta --to "source-branch" --from "target-branch" --output output-dir/ -d

What is the expected result?

WaveApplication, WaveDataset, WaveXmd type components should be added in the output directory, even though they are generated in the package.xml which got generated

What is the actual result?

No WaveApplication, WaveDataset, WaveXmd added in the output directory
Sample package.xml

<types>
        <members>Changed_Wave_App</members>
        <name>WaveApplication</name>
</types>
<types>
        <members>Changed_Wave_Dataset</members>
        <name>WaveDataset</name>
   </types>
<types>
        <members>Changed_Wave_Xmd</members>
        <name>WaveXmd</name>
 </types>

Steps to reproduce

Modify or add some WaveApplication, WaveDataset, WaveXmd
commit into a branch
Execute command: sfdx sgd:source:delta --to "source-branch" --from "target-branch" --output output-dir/ -d
Check for the mentioned wave components in the output-dir/

Execution context

Operating System: MacOS Big Sur 11.6
yarn version: …

node version: …

git version: 2.30.1 (Apple Git-130)

sfdx version: sfdx-cli/7.144.2 darwin-x64 node-v16.14.2

sgd plugin version: sfdx-git-delta 5.1.2

More information (optional)

The text was updated successfully, but these errors were encountered:

scolladon · 2022-04-08T08:25:30Z

Hi @anish1310

Thanks for raising this issue.
Sorry for the inconvenience.

I'll have a look in the coming days to qualify this issue.

Stay tuned!

scolladon · 2022-04-08T15:07:21Z

Hi @anish1310

I have investigated a little bit.
I was not able to reproduce... On my side it is working fine:

the package.xml contains the wave component ✅
the output folder contains the wave component files ✅

I created a branch issue/286 on the reproduction playground to do so.
You can checkout this branch and try locally to run sgd:

$ git clone https://github.com/scolladon/sfdx-git-delta-reproduction-playground.git
$ cd sfdx-git-delta-reproduction-playground
$ git checkout issue/286
$ sfdx sgd:source:delta -d -f "HEAD~1"
$ cat output/package/package.xml
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>WaveApplicationTest</members>
        <name>WaveApplication</name>
    </types>
    <types>
        <members>WaveDataSetTest</members>
        <name>WaveDataset</name>
    </types>
    <types>
        <members>WaveXmlTest</members>
        <name>WaveXmd</name>
    </types>
    <version>54.0</version>
</Package>
$ tree output/sgd
output/sgd
└── reproduction
    └── playground
        └── wave
            ├── WaveApplicationTest.wapp
            ├── WaveDataSetTest.wds
            └── WaveXmlTest.xmd

Could you try on your side and see if you still have the issue ?
Also I'm wondering how is your file structure, and specifically in which folder the wave files are located on your side ?
Could you give us an extract scoped for wave components ($ tree your-force-app-folder/) ?

My environment

Darwin Kernel Version 21.4.0: Mon Feb 21 20:34:37 PST 2022; root:xnu-8020.101.4~2/RELEASE_X86_64
yarn 1.22.15
node v14.19.0
git 2.35.1
sfdx-cli/7.143.0 darwin-x64 node-v14.19.0
sfdx-git-delta 5.1.2

vaibhavdangar-woolies · 2022-04-21T09:58:31Z

Hi @scolladon , trust you doing good!
We are facing a similar issue with 'wave' metadata types (WaveDashboard, WaveApplication, WaveDataset, WaveXmd), where the sgd plugin seems unable to add support file in directory, later causing to fail the validation step with Salesforce sandbox

Details below:

Source file (as seen in Github PR - Files changed): force-app/main/default/wave/ABC_Landing_Dashboard.wdash

Get delta command:
export SOURCE_API_VERSION=$(cat sfdx-project.json | jq -r .sourceApiVersion)
echo "In sfdx-project.json sourceApiVersion=$SOURCE_API_VERSION"
export LAST_DEPLOYED_COMMIT=$(git show-ref -s refs/remotes/origin/POC)

sfdx sgd:source:delta -t HEAD -f $LAST_DEPLOYED_COMMIT --output "/__w/1/s/artifact_1234" --generate-delta --api-version=$SOURCE_API_VERSION

Output:
/__w/1/s/store-wpay-merchant-portal
In sfdx-project.json sourceApiVersion=54.0
{
"error": null,
"output": "/__w/1/s/artifact_1234",
"success": true,
"warnings": []
}

**Package.xml in artifacts**

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>ABC_Landing_Dashboard</members>
        <name>WaveDashboard</name>
    </types>
    <version>54.0</version>
</Package>

 **cat .\destructiveChangesPost.xml**

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <version>54.0</version>
</Package>

Physical files in output: Delta package generated by the sgd plugin and has only one file.

└───artifact_1234
│ .forceignore
│ sfdx-project.json
│
├───destructiveChanges
│ destructiveChanges.xml
│ package.xml
│
├───force-app
│ └───main
│ └───default
│ └───wave
│ ABC_Landing_Dashboard.wdash
│
└───package
destructiveChangesPost.xml
Package.xml

Validation error :
Command:
sfdx force:source:deploy -l NoTestRun --checkonly -x /__w/1/s/artifact_1234/package/package.xml --postdestructivechanges /__w/1/s/artifact_1234/package/destructiveChangesPost.xml -u [email protected] --verbose --wait 360

Output: ERROR running force:source:deploy: No source-backed components present in the package.

With another third party tool: same object generate 2 files as output and validates successfully with Salesforce sandbox.

ABC_Landing_Dashboard.wdash
ABC_Landing_Dashboard.wdash-meta.xml (<---file absent via sgd plugin)

cat ABC_Landing_Dashboard.wdash-meta.xml
<?xml version="1.0" encoding="UTF-8"?>
<WaveDashboard xmlns="http://soap.sforce.com/2006/04/metadata" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <content xsi:nil="true"/>
    <application>ABC_App</application>
    <description>ABC_Landing_Dashboard - final</description>
    <masterLabel>ABC Landing Dashboard</masterLabel>
</WaveDashboard>

Environment:
Azure DevOps pipelines, using container: salesforce/salesforcedx:latest-full
Ubuntu 20.04.4 LTS
Version: 20220410.2

git version 2.35.1
git-lfs/3.1.2 (GitHub; linux amd64; go 1.17.6)

sfdx-cli/7.146.1 linux-x64 node-v16.13.2
@salesforce/cli/1.24.0 linux-x64 node-v16.13.2

@salesforce/sfdx-scanner 2.13.1
sfdx-git-delta 5.1.2

scolladon · 2022-04-21T10:10:52Z

Hi @vaibhavdangar-woolies !

Thanks for those information.
I will trouble shoot it in the coming days and try to propose a fix.

I was not aware of the dashboard metadata having meta file.

Other question, what is the other third party tool you mentioned ?

vaibhavdangar-woolies · 2022-04-21T10:50:25Z

Hi @scolladon ,
Appreciate your quick response.

Files with extensions in wave directory
.wdash, .wdash-meta.xml
.wds-meta.xml,
.xmd-meta.xml,
.wdf, .wdf-meta.xml
.wpr, .wdpr-meta.xml
.wlens, .wlens-meta.xml
third party tool - Gearset

I hope you are able to resolve this issue quickly, as we are blocked at the moment.
Thank you !

scolladon · 2022-04-21T10:51:07Z

I wonder why you are facing this issue as the deployment is done based on the package.xml content and not from the generated source. Normally, using sfdx force:source:deploy -x, it should deploy from your code base and not from the delta source generated, unless the delta generated folder (/__w/1/s/artifact_1234/) is listed in your sfdx-project.json right ?

vaibhavdangar-woolies · 2022-04-21T11:14:51Z

In the Azure DevOps pipeline steps:

we copy certain files:

cp ./sfdx-project.json ./artifact_1234
cp ./.forceignore ./artifact_1234
cp /__w/1/s/artifact_1234/destructiveChanges/destructiveChanges.xml /__w/1/s/artifact_1234/package/destructiveChangesPost.xml

In validation step,
cd /__w/1/s/artifact_1234 to the delta generated folder and then run the sfdx force:source:deploy -x
In sfdx-project.json from source

{
  "packageDirectories": [
    {
      "path": "force-app",
      "default": true
    }
  ],
  "name": "AppName",
  "namespace": "",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "54.0"
}

The plugin install

echo 'y' | sfdx plugins:install sfdx-git-delta
echo 'y' | sfdx plugins:install @salesforce/sfdx-scanner
echo 'List plugins installed'
sfdx plugins

output:

This plugin is not digitally signed and its authenticity cannot be verified. Continue installation y/n?: Finished digital signature check.
Installing plugin sfdx-git-delta... yarn add v1.22.18
Installing plugin sfdx-git-delta... info No lockfile found.Installing plugin sfdx-git-delta... [1/4] Resolving packages...Installing plugin sfdx-git-delta... [2/4] Fetching packages...Installing plugin sfdx-git-delta... [3/4] Linking dependencies...Installing plugin sfdx-git-delta... [4/4] Building fresh packages...Installing plugin sfdx-git-delta... success Saved lockfile.Installing plugin sfdx-git-delta... success Saved 250 new dependencies.Installing plugin sfdx-git-delta... info Direct dependenciesInstalling plugin sfdx-git-delta... └─ [email protected] plugin sfdx-git-delta... info All dependenciesInstalling plugin sfdx-git-delta... ├─ @types/[email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... ├─ [email protected] plugin sfdx-git-delta... └─ [email protected] plugin sfdx-git-delta... Done in 27.68s.Installing plugin sfdx-git-delta... installed v5.1.2
warning sfdx-git-delta > @salesforce/core > jsforce > [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
warning sfdx-git-delta > @salesforce/core > jsforce > request > [email protected]: this library is no longer supported
warning sfdx-git-delta > @salesforce/core > jsforce > request > [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
Polling for new version(s) to become available on npm... attempt: 1 of 300
Polling for new version(s) to become available on npm... done
Successfully validated digital signature for @salesforce/sfdx-scanner.
Finished digital signature check.
Installing plugin @salesforce/sfdx-scanner... yarn add v1.22.18
Installing plugin @salesforce/sfdx-scanner... [1/4] Resolving packages...Installing plugin @salesforce/sfdx-scanner... [2/4] Fetching packages...Installing plugin @salesforce/sfdx-scanner... [3/4] Linking dependencies...Installing plugin @salesforce/sfdx-scanner... [4/4] Building fresh packages...Installing plugin @salesforce/sfdx-scanner... success Saved lockfile.Installing plugin @salesforce/sfdx-scanner... success Saved 149 new dependencies.Installing plugin @salesforce/sfdx-scanner... info Direct dependenciesInstalling plugin @salesforce/sfdx-scanner... └─ @salesforce/[email protected] plugin @salesforce/sfdx-scanner... info All dependenciesInstalling plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @babel/[email protected] plugin @salesforce/sfdx-scanner... ├─ @lwc/[email protected] plugin @salesforce/sfdx-scanner... ├─ @oclif/[email protected] plugin @salesforce/sfdx-scanner... ├─ @oclif/[email protected] plugin @salesforce/sfdx-scanner... ├─ @oclif/[email protected] plugin @salesforce/sfdx-scanner... ├─ @salesforce/[email protected] plugin @salesforce/sfdx-scanner... ├─ @salesforce/[email protected] plugin @salesforce/sfdx-scanner... ├─ @salesforce/[email protected] plugin @salesforce/sfdx-scanner... ├─ @types/[email protected] plugin @salesforce/sfdx-scanner... ├─ @types/[email protected] plugin @salesforce/sfdx-scanner... ├─ @typescript-eslint/[email protected] plugin @salesforce/sfdx-scanner... ├─ @typescript-eslint/[email protected] plugin @salesforce/sfdx-scanner... ├─ @typescript-eslint/[email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... ├─ [email protected] plugin @salesforce/sfdx-scanner... └─ [email protected] plugin @salesforce/sfdx-scanner... Done in 16.75s.Installing plugin @salesforce/sfdx-scanner... installed v2.13.1
warning @salesforce/sfdx-scanner > [email protected]: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
warning @salesforce/sfdx-scanner > @salesforce/command > [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
warning @salesforce/sfdx-scanner > @salesforce/eslint-config-lwc > [email protected]: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
List plugins installed
@salesforce/sfdx-scanner 2.13.1
sfdx-git-delta 5.1.2

scolladon · 2022-04-21T11:30:03Z

@vaibhavdangar-woolies Thanks for your answers !

We have created a fix for this issue, the PR is currently under review.
Could you test it on your side and let us know if it does the job from your point of view please ?
You can follow those steps to do so

vaibhavdangar-woolies · 2022-04-21T11:39:56Z

Hi @scolladon, I will test and inform. thank you !

vaibhavdangar-woolies · 2022-04-21T13:40:21Z

Hi @scolladon , Please could you merge the PR and make it live, I'm unable to get the sfdx plugins:link work locally from docker. It would be easier with latest version installed by
echo 'y' | sfdx plugins:install sfdx-git-delta

scolladon · 2022-04-21T13:46:09Z

New version released (5.2.0) with the fix @vaibhavdangar-woolies & @anish1310
You can give feedback here

anish1310 · 2022-04-21T14:00:14Z

@scolladon Thanks for fixing the issue quickly, will test and let you know about further results.

vaibhavdangar-woolies · 2022-04-21T14:02:32Z

Hi @scolladon , Issue resolved, able to see the meta files in the delta output directory.
Thanks a ton !

scolladon · 2022-04-21T14:06:36Z

Happy incremental deployment guys !

anish1310 added the bug Something isn't working label Apr 8, 2022

anish1310 assigned scolladon Apr 8, 2022

scolladon added the good first issue Good for newcomers label Apr 8, 2022

scolladon added the reproduction steps needed Need reproduction steps to further with the analysis label Apr 8, 2022

scolladon mentioned this issue Apr 21, 2022

fix: wave meta file delta generation #293

Merged

7 tasks

scolladon closed this as completed in #293 Apr 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some Specific Wave metadata type Components are not added in Output directory #286

Some Specific Wave metadata type Components are not added in Output directory #286

anish1310 commented Apr 8, 2022

scolladon commented Apr 8, 2022

scolladon commented Apr 8, 2022

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading

scolladon commented Apr 21, 2022 •

edited

Loading

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading

scolladon commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading

scolladon commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

scolladon commented Apr 21, 2022

anish1310 commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

scolladon commented Apr 21, 2022

Some Specific Wave metadata type Components are not added in Output directory #286

Some Specific Wave metadata type Components are not added in Output directory #286

Comments

anish1310 commented Apr 8, 2022

What is the problem?

What is the parameter and the value you used with it?

What is the expected result?

What is the actual result?

Steps to reproduce

Execution context

More information (optional)

scolladon commented Apr 8, 2022

scolladon commented Apr 8, 2022

My environment

vaibhavdangar-woolies commented Apr 21, 2022 • edited Loading

Details below:

@salesforce/sfdx-scanner 2.13.1 sfdx-git-delta 5.1.2

scolladon commented Apr 21, 2022 • edited Loading

vaibhavdangar-woolies commented Apr 21, 2022 • edited Loading

scolladon commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022 • edited Loading

scolladon commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

scolladon commented Apr 21, 2022

anish1310 commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022

scolladon commented Apr 21, 2022

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading

@salesforce/sfdx-scanner 2.13.1
sfdx-git-delta 5.1.2

scolladon commented Apr 21, 2022 •

edited

Loading

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading

vaibhavdangar-woolies commented Apr 21, 2022 •

edited

Loading