Export changed files via builtin environment variables

[6543]

add CI_PIPELINE_FILES to builtin env vars

close #853

---

Sponsored by Kithara Software GmbH

[woodpecker-bot]

Deployment of preview was successful: https://woodpecker-ci-woodpecker-pr-2935.surge.sh

[qwerty287]

I think this fixes #853?

Can you check the comments there, especially #853 (comment) about the memory/performance with many changed files?

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (23f58fc) 33.89% compared to head (981c4db) 33.87%.

❗ Current head 981c4db differs from pull request most recent head 1671637. Consider uploading reports for the commit 1671637 to get more accurate results

Files	Patch %	Lines
pipeline/frontend/metadata/environment.go	0.00%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2935      +/-   ##
==========================================
- Coverage   33.89%   33.87%   -0.02%     
==========================================
  Files         219      219              
  Lines       13856    13863       +7     
==========================================
  Hits         4696     4696              
- Misses       8802     8809       +7     
  Partials      358      358              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[anbraten]

I wouldn't do this as the string can easily be extremely long. Getting the changed files can if needed be done using git already.

[6543]

I'm for adding it.

first:

1. long env variables dont create any issue
2. on bigger projects this is a question of efficience (if checked via git in pipeline, it would e.g. add 9min just to checkout the whole code to get the changed files)
3. we already have the info in metadata and database anyway

[6543]

if long env var are a concern they posibly can only make issues on windows, I can add a check for this case if desired

linux handle it just fine!

[anbraten]

Still feels wrong to possibly have those super long env vars, but as it seems to be supported its fine to me.

fine for me

[lafriks]

I would rather see this implemented differently than env variable, maybe having a token available for some special api call to the woodpecker server from the pipeline that would allow it to retrieve such data?

[6543]

So basically: #2291 (comment) ?

[confusedsushi]

For the user it is so convenient to just have that information. Of course he could get this information via git, or maybe later via some sort of API call. But shouldn't you make it for the user as easy as possible? You already have that information, why not forward it to the user? If the user need to bother with some git command, some HTTP API call or some other external tool he will just make mistakes.

[6543]

It now only export if changed files if maxChangedFiles (500) is not exceeded

[xoxys]

I'm also not sure about the current way of implementation. Why not use the approach from the linked issue and let e.g. plugin-git create a file that contains all changed files, one per line? The current JSON string can't be easily parsed by bash/shell.

[6543]

well first, we do shallow clone by default.
so recover the information the server already has is just realy inefficient.

just to point out some numbers: big repos will rise clone time from ~3min to 12min just to get that info

[6543]

and json can be parsed by jq or yq witch are widely available. powershell (windows) support it natively.

[xoxys]

Ah yeah that make sense. Missed this point.

[6543]

I think this fixes #853?

yes it does :) - updated pull description

[xoxys]

Code LGTM.