feat(brotli): support brotli compression format

[yigaldviri]

Solving this issue

Checklist

• [V] I have ensured my pull request is not behind the main or master branch of the original repository.
• [V] I have rebased all commits where necessary so that reviewing this pull request can be done without having to merge it first.
• [V] I have written a commit message that passes commitlint linting.
• [V] I have ensured that my code changes pass linting tests.
• [V] I have ensured that my code changes pass unit tests.
• [V] I have described my pull request and the reasons for code changes along with context if necessary.

[yigaldviri]

Hi @titanism is there a chance you can have a look at this one?

[titanism]

I think you may need to use re2 package to check for reged ddos attack vector in utils for isGzipOrDefaultEncoding and isBrotliEncoding. What happens if malicious arbitrarily long payload was passed for Content-Encoding header?

[yigaldviri]

I think you may need to use re2 package to check for reged ddos attack vector in utils for isGzipOrDefaultEncoding and isBrotliEncoding. What happens if malicious arbitrarily long payload was passed for Content-Encoding header?

Thanks for the feedback. Actually, I moved the gzip/ deflate regex in to the file and added another regex (pretty much the same) for Brotli so I guess this vulnerability already exists. Shouldn't we handle it in a dedicated PR? WDYT?

[yigaldviri]

I think you may need to use re2 package to check for reged ddos attack vector in utils for isGzipOrDefaultEncoding and isBrotliEncoding. What happens if malicious arbitrarily long payload was passed for Content-Encoding header?

Thanks for the feedback. Actually, I moved the gzip/ deflate regex in to the file and added another regex (pretty much the same) for Brotli so I guess this vulnerability already exists. Shouldn't we handle it in a dedicated PR? WDYT?

Hi @titanism , did you had time to look at it?

[titanism]

As requested you need to use something like RE2 or add tests to test against Regex Bomb / long string attack vector

[yigaldviri]

As requested you need to use something like RE2 or add tests to test against Regex Bomb / long string attack vector

I had a look at it and RE@ is Node only. And the other option of xregexp fails the build because of lint issue in the dist file. BTW, Im not sure why even linting the dist file - maybe you can shed some light

[yigaldviri]

As requested you need to use something like RE2 or add tests to test against Regex Bomb / long string attack vector

Hi @titanism can you have a look? I added what you've asked for

[titanism]

You did not do what we asked, we asked for re2 support, not xregexp. That package you used doesn't even provide regex bomb safety.

[titanism]

As requested you need to use something like RE2 or add tests to test against Regex Bomb / long string attack vector

I had a look at it and RE@ is Node only. And the other option of xregexp fails the build because of lint issue in the dist file. BTW, Im not sure why even linting the dist file - maybe you can shed some light

Hi there - apologies as we did not see your earlier reply quoted above here. For browsers you can take the approach we did here:

https://github.com/spamscanner/email-regex-safe/blob/9844448d9f617935da2a5197391787a6d8cb0eab/src/index.js#L33-L46

https://github.com/spamscanner/email-regex-safe/blob/9844448d9f617935da2a5197391787a6d8cb0eab/package.json#L6-L8

[yigaldviri]

As requested you need to use something like RE2 or add tests to test against Regex Bomb / long string attack vector

I had a look at it and RE@ is Node only. And the other option of xregexp fails the build because of lint issue in the dist file. BTW, Im not sure why even linting the dist file - maybe you can shed some light

Hi there - apologies as we did not see your earlier reply quoted above here. For browsers you can take the approach we did here:

https://github.com/spamscanner/email-regex-safe/blob/9844448d9f617935da2a5197391787a6d8cb0eab/src/index.js#L33-L46

https://github.com/spamscanner/email-regex-safe/blob/9844448d9f617935da2a5197391787a6d8cb0eab/package.json#L6-L8

Done. Please have a look

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/[email protected]	environment, eval	+5	165 kB	goto-bus-stop

🚮 Removed packages: npm/[email protected]

View full report↗︎

[yigaldviri]

Hi @titanism - did you had a chance to have a look at it? :)

[titanism]

This code needs cleaned up dc58a76#diff-3274f1a37032fb0ae4e2823def0007c634e869ae0dfc304ff6a12c36513c3a52R1-R15

[yigaldviri]

This code needs cleaned up dc58a76#diff-3274f1a37032fb0ae4e2823def0007c634e869ae0dfc304ff6a12c36513c3a52R1-R15

Code refactored

[yigaldviri]

This code needs cleaned up dc58a76#diff-3274f1a37032fb0ae4e2823def0007c634e869ae0dfc304ff6a12c36513c3a52R1-R15

Code refactored

Hi @titanism can you please have a look?

[titanism]

@yigaldviri what was changed? it doesn't look like anything was changed

did you run npm test locally? does npx xo --fix make the code linked better and less indented?

[socket-security]

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Alert	Package	Note	Source	CI
Install scripts	npm/[email protected]	Install script: postinstall Source: node -e "try{require('./postinstall')}catch(e){}"	package.json	🚫

View full report↗︎

Next steps

What is an install script?

Install scripts are run when the package is installed. The majority of malware in npm is hidden in install scripts.

Packages should not be running non-essential scripts during install and there are often solutions to problems people solve with install scripts that can be run at publish time instead.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/[email protected] or ignore all packages with @SocketSecurity ignore-all

• @SocketSecurity ignore npm/[email protected]

[yigaldviri]

@yigaldviri what was changed? it doesn't look like anything was changed

did you run npm test locally? does npx xo --fix make the code linked better and less indented?

Sorry I forgot to git add :)

And yes - tests passed successfully

[yigaldviri]

Hi @titanism - can you please have a look?

[yigaldviri]

Hi @titanism - I understand that you are probably busy. Do you know another maintainer that can have a look at this?

[titanism]

@yigaldviri v10.0.0 is released to npm, can you please try it out and let us know if it works OK? 🙏

Thank you for your contribution 🎉

Ref: https://github.com/ladjs/superagent/releases/tag/v10.0.0

[yigaldviri]

I think we have an issue. When using Yarn and Node 14 you cant download superagent since the re2 package contains [email protected] that requires Node 16 (only yarn fails it. npm not).
We can downgrade RE2 version a bit to [email protected] and it will solve it so it.

[titanism]

PR welcome?

[yigaldviri]

PR welcome?

Seems like this package (combined with yarn) isnt compatible on Node 14.18. I tried older version but some other dependency fails us there. Do you think there's a different package we ca can use instead of it?

PR welcome?

Here you go - #1813

[titanism]

We should just drop Node v14 support. Honestly would be best to support Node >= 16 or just Node v18+ if necessary.

You can modify your PR as such @yigaldviri

[yigaldviri]

We should just drop Node v14 support. Honestly would be best to support Node >= 16 or just Node v18+ if necessary.

You can modify your PR as such @yigaldviri

You are right. However, I'm using Node 14. Upgrading now isnt an option (on our roadmap) so dropping support for 14 will be problematic