Clean up query cache code #59722

Zoxc · 2019-04-05T11:14:36Z

There are a couple of queries for which we do not promote saved results, which have been removed from caching here. This was likely the cause of the regression in #57293 (comment).

r? @michaelwoerister

src/librustc/query/mod.rs

Zoxc · 2019-04-05T21:05:21Z

@bors try

bors · 2019-04-05T21:05:32Z

⌛ Trying commit a3493a2492373f97fb47597ad6f2911a3b82f4c7 with merge db126a1e5b055e968f3661c12b92ab2ba40987e3...

bors · 2019-04-05T23:18:51Z

☀️ Try build successful - checks-travis
Build commit: db126a1e5b055e968f3661c12b92ab2ba40987e3

Zoxc · 2019-04-05T23:38:55Z

@rust-build timer db126a1e5b055e968f3661c12b92ab2ba40987e3

Zoxc · 2019-04-07T10:46:04Z

@rust-timer build db126a1e5b055e968f3661c12b92ab2ba40987e3

rust-timer · 2019-04-07T10:46:05Z

Success: Queued db126a1e5b055e968f3661c12b92ab2ba40987e3 with parent acd8dd6, comparison URL.

rust-timer · 2019-04-07T12:31:32Z

Finished benchmarking try commit db126a1e5b055e968f3661c12b92ab2ba40987e3

rust-highfive · 2019-04-14T17:13:37Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:005ccbb4:start=1555261047624443916,finish=1555261048358277880,duration=733833964
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:04:03] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:04] tidy error: /checkout/src/librustc/query/mod.rs:225: line longer than 100 chars
[00:04:05] some tidy checks failed
[00:04:05] 
[00:04:05] 
[00:04:05] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:05] 
[00:04:05] 
[00:04:05] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:05] Build completed unsuccessfully in 0:00:47
[00:04:05] Build completed unsuccessfully in 0:00:47
[00:04:05] make: *** [tidy] Error 1
[00:04:05] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:116c22c6
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Sun Apr 14 17:01:44 UTC 2019
---
travis_time:end:23dc4d9a:start=1555261305653397594,finish=1555261305658518868,duration=5121274
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:01d02cbc
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0822c09c
travis_time:start:0822c09c
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:02770044
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Zoxc · 2019-04-15T15:38:42Z

@bors try

bors · 2019-04-15T15:39:39Z

⌛ Trying commit 475186d8e9976d32df95c00dac01efa96b3a467e with merge 59c0f07bdd675044703a15d56dd70a72543a0532...

bors · 2019-04-15T18:02:31Z

☀️ Try build successful - checks-travis
Build commit: 59c0f07bdd675044703a15d56dd70a72543a0532

Zoxc · 2019-04-15T18:18:48Z

@rust-timer build 59c0f07bdd675044703a15d56dd70a72543a0532

rust-timer · 2019-04-15T18:18:50Z

Success: Queued 59c0f07bdd675044703a15d56dd70a72543a0532 with parent fcf850f, comparison URL.

rust-timer · 2019-04-16T05:55:48Z

Finished benchmarking try commit 59c0f07bdd675044703a15d56dd70a72543a0532

michaelwoerister · 2019-04-26T09:40:35Z

Any idea why performance is affected so badly?

Zoxc · 2019-04-26T16:36:02Z

@michaelwoerister No, but I must have messed something up.

Mark-Simulacrum · 2019-05-18T14:17:09Z

Visiting from triage; marking as waiting on author rather than review; I suspect this is unlikely to make progress until @Zoxc has a chance to investigate the slowdown and fix it.

Zoxc · 2019-06-26T10:41:31Z

@rust-timer build 61796f24147d054025b3ff3c29179733ca54a1b4

rust-timer · 2019-06-26T10:41:32Z

Success: Queued 61796f24147d054025b3ff3c29179733ca54a1b4 with parent d3e2cec, comparison URL.

eddyb · 2019-06-26T10:43:58Z

src/librustc/dep_graph/graph.rs

-                        None
-                    }
+        let data = self.data.as_ref().unwrap();
+        data.colors.values.indices().for_each(|prev_index| {


Why not use a for loop?

Idk, to not make for_each seem useless? =P

eddyb

r=me if perf is good

rust-timer · 2019-06-26T13:14:22Z

Finished benchmarking try commit 61796f24147d054025b3ff3c29179733ca54a1b4, comparison URL.

Zoxc · 2019-06-26T13:33:28Z

@bors r=eddyb

@oli-obk

Store const_eval_raw results to disk Based on #59722. r? @oli-obk

Zoxc · 2019-06-30T09:25:05Z

@bors r=eddyb

bors · 2019-06-30T09:25:07Z

📌 Commit ede41ab has been approved by eddyb

bors · 2019-06-30T12:15:20Z

⌛ Testing commit ede41ab with merge 0af8e87...

@michaelwoerister

Clean up query cache code There are a couple of queries for which we do not promote saved results, which have been removed from caching here. This was likely the cause of the regression in #57293 (comment). r? @michaelwoerister

bors · 2019-06-30T15:39:19Z

☀️ Test successful - checks-azure, checks-travis, status-appveyor
Approved by: eddyb
Pushing 0af8e87 to master...

rust-highfive assigned michaelwoerister Apr 5, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 5, 2019

michaelwoerister reviewed Apr 5, 2019

View reviewed changes

src/librustc/query/mod.rs Outdated Show resolved Hide resolved

Zoxc force-pushed the query-cache branch 3 times, most recently from e3cdfff to a31b2af Compare April 14, 2019 16:44

Zoxc force-pushed the query-cache branch from a31b2af to 475186d Compare April 14, 2019 17:33

Zoxc changed the title ~~[WIP] Clean up query cache code~~ Clean up query cache code Apr 15, 2019

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 18, 2019

Zoxc force-pushed the query-cache branch from 475186d to 26ff212 Compare June 15, 2019 21:12

rust-highfive assigned eddyb and unassigned michaelwoerister Jun 26, 2019

eddyb reviewed Jun 26, 2019

View reviewed changes

eddyb approved these changes Jun 26, 2019

View reviewed changes

Zoxc force-pushed the query-cache branch from 02f751b to fbfee70 Compare June 26, 2019 10:56

Zoxc mentioned this pull request Jun 27, 2019

Store const_eval_raw results to disk #62166

Closed

bors added a commit that referenced this pull request Jun 27, 2019

Auto merge of #62166 - Zoxc:const_eval_raw, r=<try>

c385f72

Store const_eval_raw results to disk Based on #59722. r? @oli-obk

Zoxc closed this Jun 30, 2019

Zoxc reopened this Jun 30, 2019

Zoxc added 2 commits June 30, 2019 11:18

Clean up query cache code

930ad86

Keep caching for non-promoted queries

ede41ab

Zoxc force-pushed the query-cache branch from fbfee70 to ede41ab Compare June 30, 2019 09:23

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 30, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 30, 2019

bors merged commit ede41ab into rust-lang:master Jun 30, 2019

This was referenced Jun 30, 2019

Remove needless lifetimes (rustc) #62039

Merged

Use sharded maps for queries #62108

Merged

Zoxc deleted the query-cache branch June 30, 2019 16:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up query cache code #59722

Clean up query cache code #59722

Zoxc commented Apr 5, 2019

Zoxc commented Apr 5, 2019

bors commented Apr 5, 2019

bors commented Apr 5, 2019

Zoxc commented Apr 5, 2019

Zoxc commented Apr 7, 2019

rust-timer commented Apr 7, 2019

rust-timer commented Apr 7, 2019

rust-highfive commented Apr 14, 2019

Zoxc commented Apr 15, 2019

bors commented Apr 15, 2019

bors commented Apr 15, 2019

Zoxc commented Apr 15, 2019

rust-timer commented Apr 15, 2019

rust-timer commented Apr 16, 2019

michaelwoerister commented Apr 26, 2019

Zoxc commented Apr 26, 2019

Mark-Simulacrum commented May 18, 2019

Zoxc commented Jun 26, 2019

rust-timer commented Jun 26, 2019

eddyb Jun 26, 2019

Zoxc Jun 26, 2019

eddyb left a comment

rust-timer commented Jun 26, 2019

Zoxc commented Jun 26, 2019

Zoxc commented Jun 30, 2019

bors commented Jun 30, 2019

bors commented Jun 30, 2019

bors commented Jun 30, 2019

Clean up query cache code #59722

Clean up query cache code #59722

Conversation

Zoxc commented Apr 5, 2019

Zoxc commented Apr 5, 2019

bors commented Apr 5, 2019

bors commented Apr 5, 2019

Zoxc commented Apr 5, 2019

Zoxc commented Apr 7, 2019

rust-timer commented Apr 7, 2019

rust-timer commented Apr 7, 2019

rust-highfive commented Apr 14, 2019

Zoxc commented Apr 15, 2019

bors commented Apr 15, 2019

bors commented Apr 15, 2019

Zoxc commented Apr 15, 2019

rust-timer commented Apr 15, 2019

rust-timer commented Apr 16, 2019

michaelwoerister commented Apr 26, 2019

Zoxc commented Apr 26, 2019

Mark-Simulacrum commented May 18, 2019

Zoxc commented Jun 26, 2019

rust-timer commented Jun 26, 2019

eddyb Jun 26, 2019

Choose a reason for hiding this comment

Zoxc Jun 26, 2019

Choose a reason for hiding this comment

eddyb left a comment

Choose a reason for hiding this comment

rust-timer commented Jun 26, 2019

Zoxc commented Jun 26, 2019

Zoxc commented Jun 30, 2019

bors commented Jun 30, 2019

bors commented Jun 30, 2019

bors commented Jun 30, 2019