Privacy Sql Tracking Detection Using Easylist Adservers #3730

hadiamjad · 2024-08-14T15:10:46Z

New queries added for this year's chapter

easylist_tracker_detection.sql

New util added for this year's chapter

populate_easylist_adserver.py

* Add standard GA4 web-vital fields * Add value

Co-authored-by: tunetheweb <[email protected]>

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.1.0 to 4.1.1. - [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md) - [Commits](GoogleChrome/web-vitals@v4.1.0...v4.1.1) --- updated-dependencies: - dependency-name: web-vitals dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.10.0 to 22.10.1. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.10.0...puppeteer-v22.10.1) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [prettier](https://github.com/prettier/prettier) from 3.3.1 to 3.3.2. - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@3.3.1...3.3.2) --- updated-dependencies: - dependency-name: prettier dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.10.1 to 22.11.0. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.10.1...puppeteer-v22.11.0) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.11.0 to 22.11.2. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.11.0...puppeteer-v22.11.2) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.1.1 to 4.2.0. - [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md) - [Commits](GoogleChrome/web-vitals@v4.1.1...v4.2.0) --- updated-dependencies: - dependency-name: web-vitals dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.11.2 to 22.12.0. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.11.2...puppeteer-v22.12.0) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: tunetheweb <[email protected]>

* Remove deploy.zip * Remove from ignore files

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.12.0 to 22.12.1. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.12.0...puppeteer-v22.12.1) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump treosh/lighthouse-ci-action from 11.4.0 to 12.1.0 Bumps [treosh/lighthouse-ci-action](https://github.com/treosh/lighthouse-ci-action) from 11.4.0 to 12.1.0. - [Release notes](https://github.com/treosh/lighthouse-ci-action/releases) - [Commits](treosh/lighthouse-ci-action@11.4.0...12.1.0) --- updated-dependencies: - dependency-name: treosh/lighthouse-ci-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> * Upgrade to Node 20 --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Barry Pollard <[email protected]>

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.0 to 4.2.1. - [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md) - [Commits](GoogleChrome/web-vitals@v4.2.0...v4.2.1) --- updated-dependencies: - dependency-name: web-vitals dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.0 to 5.1.1. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5.1.0...v5.1.1) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.12.1 to 22.13.0. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.12.1...puppeteer-v22.13.0) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.13.0 to 22.15.0. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.13.0...puppeteer-v22.15.0) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [jsdom](https://github.com/jsdom/jsdom) from 24.1.0 to 24.1.1. - [Release notes](https://github.com/jsdom/jsdom/releases) - [Changelog](https://github.com/jsdom/jsdom/blob/main/Changelog.md) - [Commits](jsdom/jsdom@24.1.0...24.1.1) --- updated-dependencies: - dependency-name: jsdom dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.1 to 4.2.2. - [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md) - [Commits](GoogleChrome/web-vitals@v4.2.1...v4.2.2) --- updated-dependencies: - dependency-name: web-vitals dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [prettier](https://github.com/prettier/prettier) from 3.3.2 to 3.3.3. - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@3.3.2...3.3.3) --- updated-dependencies: - dependency-name: prettier dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.2 to 4.2.3. - [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md) - [Commits](GoogleChrome/web-vitals@v4.2.2...v4.2.3) --- updated-dependencies: - dependency-name: web-vitals dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: rviscomi <[email protected]>

* add file to translation * done translation cdn.md Makes progress on #505

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.15.0 to 23.0.2. - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json) - [Commits](puppeteer/puppeteer@puppeteer-v22.15.0...puppeteer-v23.0.2) --- updated-dependencies: - dependency-name: puppeteer dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: tunetheweb <[email protected]>

* Replace object with iframe for embedded SVG * Translations

* Fix the build to ignore 2024 chapters (for now) * Remove test line

Co-authored-by: tunetheweb <[email protected]>

sql/util/populate_easylist_adserver.py

+import requests
+import pandas as pd
+from google.cloud import bigquery
+import os


max-ostapenko · 2024-08-15T00:07:55Z

Before pushing the changes, you can lint the code in your branch:

python3 -m venv src/.venv
source src/.venv/bin/activate
pip install -r src/requirements.txt
sqlfluff fix sql/2024/privacy/

sql/2024/privacy/tracking-detection/easylist_adservers.csv

max-ostapenko · 2024-08-15T09:29:27Z

@hadiamjad FYI I pointed this PR base to the Privacy chapter PR, so that we can merge all chapter in one step eventually.
Looking forward to your linting adjustments to merge this.

tunetheweb · 2024-08-15T14:10:55Z

@hadiamjad FYI I pointed this PR base to the Privacy chapter PR, so that we can merge all chapter in one step eventually. Looking forward to your linting adjustments to merge this.

@max-ostapenko you might wanna update the Privacy chapter PR to latest version of main to hide the irrelevant changes in this PR (and stop those annoying diff comments being added to this PR, which happens when it thinks you're changing certain things).

max-ostapenko · 2024-08-15T14:43:50Z

@max-ostapenko you might wanna update the Privacy chapter PR to latest version of main to hide the irrelevant changes in this PR (and stop those annoying diff comments being added to this PR, which happens when it thinks you're changing certain things).

@tunetheweb would you suggest to do it with merge commit or rebase to make it clean?

tunetheweb · 2024-08-15T14:46:06Z

I think a merge commit on the privacy-sql-2024 branch will be fine. I hate rebasing :-)

max-ostapenko · 2024-08-16T21:37:30Z

I think a merge commit on the privacy-sql-2024 branch will be fine. I hate rebasing :-)

I merged main into Privacy PR and then also here - the diff was only growing 🤔
Seems I still need to learn about git behavior :)

tunetheweb and others added 30 commits June 10, 2024 16:03

Add GA4 fields to match documentation (#3679)

1fcf6a5

* Add standard GA4 web-vital fields * Add value

Update Timestamps (#3680)

523ae0d

Co-authored-by: tunetheweb <[email protected]>

Translation of security chapter to Japanese (#3685)

91a5fb9

Update Timestamps (#3691)

1a8586a

Co-authored-by: tunetheweb <[email protected]>

Remove deploy.zip step of deployment (#3692)

5107c81

* Remove deploy.zip * Remove from ignore files

Translation of mobile-web chapter to Japanese (#3700)

e0c67bd

Update Timestamps (#3716)

90e7e8a

Co-authored-by: rviscomi <[email protected]>

tracking detection using easylist adservers

b637da1

easylist_adserver tracking detection and query

f7b1283

2022 cdn portuguese (#3725)

f428a3e

* add file to translation * done translation cdn.md Makes progress on #505

Update Timestamps (#3726)

74330d7

Co-authored-by: tunetheweb <[email protected]>

Replace <object> with <iframe> for embedded SVG (#3727)

5b7997e

* Replace object with iframe for embedded SVG * Translations

Hadi Amjad and others added 3 commits August 14, 2024 10:43

auto upload easylist data to table

bf7fee0

Fix the build to ignore 2024 chapters (for now) (#3728)

da71b3e

* Fix the build to ignore 2024 chapters (for now) * Remove test line

Update Timestamps (#3729)

6e01d17

Co-authored-by: tunetheweb <[email protected]>

github-advanced-security bot found potential problems Aug 14, 2024

View reviewed changes

sql/util/populate_easylist_adserver.py Outdated

import requests

import pandas as pd

from google.cloud import bigquery

import os

Check notice

Code scanning / CodeQL

Unused import Note

Import of 'os' is not used.

max-ostapenko mentioned this pull request Aug 14, 2024

Privacy 2024 queries #3653

Open

40 tasks

max-ostapenko requested changes Aug 15, 2024

View reviewed changes

sql/2024/privacy/tracking-detection/easylist_adservers.csv Outdated Show resolved Hide resolved

max-ostapenko changed the base branch from main to privacy-sql-2024 August 15, 2024 09:14

Hadi Amjad added 2 commits August 15, 2024 09:23

liniting

26965e7

liniting

3107288

HTTPArchive deleted a comment from github-actions bot Aug 15, 2024

Hadi Amjad added 4 commits August 15, 2024 09:38

linting

bac6526

linting

e6ec23d

linting

8331ecf

linting

d48eafc

HTTPArchive deleted a comment from github-actions bot Aug 15, 2024

fixes of Simplified Chinese translation for 2020 Performance (#3734)

90be0cf

max-ostapenko approved these changes Aug 16, 2024

View reviewed changes

Merge branch 'main' into privacy-sql-tracking-detection-easylist

3128f3a

max-ostapenko merged commit baf490d into privacy-sql-2024 Aug 16, 2024
9 checks passed

max-ostapenko deleted the privacy-sql-tracking-detection-easylist branch August 16, 2024 21:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Privacy Sql Tracking Detection Using Easylist Adservers #3730

Privacy Sql Tracking Detection Using Easylist Adservers #3730

hadiamjad commented Aug 14, 2024

max-ostapenko commented Aug 15, 2024

max-ostapenko commented Aug 15, 2024

tunetheweb commented Aug 15, 2024

max-ostapenko commented Aug 15, 2024

tunetheweb commented Aug 15, 2024

max-ostapenko commented Aug 16, 2024

Privacy Sql Tracking Detection Using Easylist Adservers #3730

Privacy Sql Tracking Detection Using Easylist Adservers #3730

Conversation

hadiamjad commented Aug 14, 2024

max-ostapenko commented Aug 15, 2024

max-ostapenko commented Aug 15, 2024

tunetheweb commented Aug 15, 2024

max-ostapenko commented Aug 15, 2024

tunetheweb commented Aug 15, 2024

max-ostapenko commented Aug 16, 2024