Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QA: update LTE "classic" analysis engine to set extended blocking flags #2640

Closed
bassosimone opened this issue Dec 1, 2023 · 1 comment
Closed

QA: update LTE "classic" analysis engine to set extended blocking flags #2640

bassosimone opened this issue Dec 1, 2023 · 1 comment
Assignees
@bassosimone
Labels
funder/drl2022-2024 priority/high

Comments

@bassosimone
Copy link
Contributor

This issue is about updating LTE "classic" analysis engine, which currently only emulates v0.4 top-level keys, to additionally write keys that provide a more nuanced overview of the blocking types, pretty much like the "orig" engine does.
@bassosimone bassosimone self-assigned this Dec 1, 2023
@bassosimone bassosimone mentioned this issue Jan 16, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
refactor(webconnectivitylte): make analysis flags public (#1447)
4bf2ae2 
This PR contains yak shaving before doing
ooni/probe#2640.

We extracted this PR from #1446.
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
refactor(minipipeline): allow avoiding linear analysis
327a6ac 
This is yak shaving for ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 16, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
refactor(minipipeline): allow avoiding linear analysis (#1448)
75780fd 
This is yak shaving for ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 16, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
feat(minipipeline): add DNSLookupSuccessWithBogonAddresses (#1449)
dfaeed9 
This diff adds the DNSLookupSuccessWithBogonAddresses analysis field to
minipipeline.

We'll need this field to complete
ooni/probe#2640.

While there, implement `io.Stringer` for the `minipeline.Set` type,
which we will also need to have.
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
chore(minipipeline): increase code coverage
11b19e3 
KTLO work as part of ooni/probe#2640
@bassosimone bassosimone mentioned this issue Jan 16, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 16, 2024
@bassosimone
chore(minipipeline): increase code coverage (#1450)
2f83da4 
I spent some time increasing code coverage for `internal/minipipeline`.

This is KTLO work as part of ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 18, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 18, 2024
@bassosimone
feat(minipipeline): add ControlFinalResponseExpectations (#1451)
0e44897 
This PR adds the ControlFinalResponseExpectations field to
WebObservationsContainer. We need this field to determine whether
unexplained failures observed by the probe (i.e., failures occurring
during redirects) are to be expected (because for some reason also the
TH failed) or unexpected (because the TH succeeded).

Part of ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 18, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
feat(minipipeline): add ControlFinalResponseExpectations (2/2) (#1452)
64255d5 
This diff completes the work started by #1451 by adding support for
ControlFinalResponseExpectations to the analysis data structures, which
will enable us to use this information for Web Connectivity LTE.

Part of ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 19, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
feat(minipipeline): add DNS and HTTP unexplained failures (#1453)
bdb519d 
We need to know about all possible unexplained failures. With this
information and information about ControlFinalResponseExpectations
introduced in #1451 and #1452, we are able to transform unexplained
failures (i.e., failures occurring during redirects) into explained
failures when we have a control-based expectation to compare to.

Part of ooni/probe#2640
@bassosimone bassosimone mentioned this issue Jan 19, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
feat(webconnectivitylte): classic computes XBlockingFlags (#1446)
41bedb5 
This diff modifies the "classic" analysis engine to compute
XBlockingFlags. While there, make sure that the flags we compute make
sense thus fixing analysis bugs in the previously used "orig" engine.

Part of ooni/probe#2640.
@bassosimone bassosimone mentioned this issue Jan 19, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
feat(webconnectivitylte): classic supports XNullNullFlags (#1454)
5ddbf8b 
This diff extends webconnectivitylte's classic analysis engine and
minipipeline to generate `XNullNullFlags`.

With this change implemented, we have successfully replaced the "orig"
engine with "classic".

This work is part of ooni/probe#2640.

While this diff is large, most of the changes are in the generated files
for the minipipeline test suite.
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
cleanup(webconnectivitylte): remove the orig engine
915d5a8 
This commit removes the "orig" data analysis engine of the
webconnectivitylte experiment now that the "classic" engine
is able to generate the same or better results.

Part of ooni/probe#2640
@bassosimone bassosimone mentioned this issue Jan 19, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
cleanup(webconnectivitylte): remove the orig engine (#1455)
b6cfb0a 
This commit removes the "orig" data analysis engine of the
webconnectivitylte experiment now that the "classic" engine is able to
generate the same or better results.

Part of ooni/probe#2640
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
cleanup(webconnectivitytle): avoid code duplication
1411878 
Use a single algorithm for determining whether there's HTTP diff.

Part of ooni/probe#2640
@bassosimone bassosimone mentioned this issue Jan 19, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 19, 2024
@bassosimone
cleanup(webconnectivitytle): avoid code duplication (#1456)
b664281 
Use a single algorithm for determining whether there's HTTP diff. We can
do this by migrating the HTTP diff flags to use `optional.Value` and
then we can always use code written for the "ext" sub-engine to do that.

Part of ooni/probe#2640
@bassosimone
Copy link
Contributor Author

All that was required has been done! 🥳

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
refactor(webconnectivitylte): make analysis flags public (ooni#1447)
d6239ef 
This PR contains yak shaving before doing
ooni/probe#2640.

We extracted this PR from ooni#1446.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
refactor(minipipeline): allow avoiding linear analysis (ooni#1448)
f5cd0c0 
This is yak shaving for ooni/probe#2640.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(minipipeline): add DNSLookupSuccessWithBogonAddresses (ooni#1449)
b9a76a6 
This diff adds the DNSLookupSuccessWithBogonAddresses analysis field to
minipipeline.

We'll need this field to complete
ooni/probe#2640.

While there, implement `io.Stringer` for the `minipeline.Set` type,
which we will also need to have.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
chore(minipipeline): increase code coverage (ooni#1450)
dcb5d5a 
I spent some time increasing code coverage for `internal/minipipeline`.

This is KTLO work as part of ooni/probe#2640.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(minipipeline): add ControlFinalResponseExpectations (ooni#1451)
99cf201 
This PR adds the ControlFinalResponseExpectations field to
WebObservationsContainer. We need this field to determine whether
unexplained failures observed by the probe (i.e., failures occurring
during redirects) are to be expected (because for some reason also the
TH failed) or unexpected (because the TH succeeded).

Part of ooni/probe#2640.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(minipipeline): add ControlFinalResponseExpectations (2/2) (ooni#…
0aa3919 
…1452)

This diff completes the work started by ooni#1451 by adding support for
ControlFinalResponseExpectations to the analysis data structures, which
will enable us to use this information for Web Connectivity LTE.

Part of ooni/probe#2640.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(minipipeline): add DNS and HTTP unexplained failures (ooni#1453)
2dc5c00 
We need to know about all possible unexplained failures. With this
information and information about ControlFinalResponseExpectations
introduced in ooni#1451 and ooni#1452, we are able to transform unexplained
failures (i.e., failures occurring during redirects) into explained
failures when we have a control-based expectation to compare to.

Part of ooni/probe#2640
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(webconnectivitylte): classic computes XBlockingFlags (ooni#1446)
140d18f 
This diff modifies the "classic" analysis engine to compute
XBlockingFlags. While there, make sure that the flags we compute make
sense thus fixing analysis bugs in the previously used "orig" engine.

Part of ooni/probe#2640.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(webconnectivitylte): classic supports XNullNullFlags (ooni#1454)
18b694c 
This diff extends webconnectivitylte's classic analysis engine and
minipipeline to generate `XNullNullFlags`.

With this change implemented, we have successfully replaced the "orig"
engine with "classic".

This work is part of ooni/probe#2640.

While this diff is large, most of the changes are in the generated files
for the minipipeline test suite.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
cleanup(webconnectivitylte): remove the orig engine (ooni#1455)
4c81abe 
This commit removes the "orig" data analysis engine of the
webconnectivitylte experiment now that the "classic" engine is able to
generate the same or better results.

Part of ooni/probe#2640
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
cleanup(webconnectivitytle): avoid code duplication (ooni#1456)
0cb3a45 
Use a single algorithm for determining whether there's HTTP diff. We can
do this by migrating the HTTP diff flags to use `optional.Value` and
then we can always use code written for the "ext" sub-engine to do that.

Part of ooni/probe#2640
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
funder/drl2022-2024 priority/high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bassosimone @jbonisteel