Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor translation unit partitioning/collection as a query #44529

Merged
merged 11 commits into from
Sep 18, 2017
Merged

Refactor translation unit partitioning/collection as a query #44529

merged 11 commits into from
Sep 18, 2017

Commits on Sep 17, 2017

  1. rustc_trans: Refactor collection to use tcx 

    This commit refactors the `collect_crate_translation_items` function to only
require the `TyCtxt` instead of a `SharedCrateContext` in preparation for
query-ifying this portion of trans.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    c72240a View commit details
    Browse the repository at this point in the history

  2. rustc: Refactor trans paritioning to use tcx 

    This commit refactors the the `partitioning::partition` function to operate with
a `TyCtxt` instead of a `SharedCrateContext` in preparation for making it a
query.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    a97ad6a View commit details
    Browse the repository at this point in the history

  3. rustc: Use reachablility through a query 

    Turns out this was already set up as a query, just wasn't using it yet!
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    baca9a6 View commit details
    Browse the repository at this point in the history

  4. rustc: Calculate ExportedSymbols in a query 

    This commit moves the definition of the `ExportedSymbols` structure to the
`rustc` crate and then creates a query that'll be used to construct the
`ExportedSymbols` set. This in turn uses the reachablity query exposed in the
previous commit.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    dba3ddd View commit details
    Browse the repository at this point in the history

  5. rustc: Make trans collect/partition a query 

    This commit moves the `collect_and_partition_translation_items` function into a
query on `TyCtxt` instead of a free function in trans, allowing us to track
dependencies and such of the function.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    132bde7 View commit details
    Browse the repository at this point in the history

  6. rustc: Move some attr methods to queries 

    Otherwise we may emit double errors related to the `#[export_name]` attribute,
for example, and using a query should ensure that it's only emitted at most
once.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    8821aff View commit details
    Browse the repository at this point in the history

  7. rustc: Mostly remove ExportedSymbols 

    This is a big map that ends up inside of a `CrateContext` during translation for
all codegen units. This means that any change to the map may end up causing an
incremental recompilation of a codegen unit! In order to reduce the amount of
dependencies here between codegen units and the actual input crate this commit
refactors dealing with exported symbols and such into various queries.

The new queries are largely based on existing queries with filled out
implementations for the local crate in addition to external crates, but the main
idea is that while translating codegen untis no unit needs the entire set of
exported symbols, instead they only need queries about particulare `DefId`
instances every now and then.

The linking stage, however, still generates a full list of all exported symbols
from all crates, but that's going to always happen unconditionally anyway, so no
news there!
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    afb85cf View commit details
    Browse the repository at this point in the history

  8. rustc: Move a comment to the right spot in trans 

    I believe this comment here is mostly talking about the `ptrcast` function call
below, so move the comment down to that block.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    19727c8 View commit details
    Browse the repository at this point in the history

  9. rustc: Remove another global map from trans 

    This commit removes the `crate_trans_items` field from the `CrateContext` of
trans. This field, a big map, was calculated during partioning and was a set of
all translation items. This isn't quite incremental-friendly because the map may
change a lot but not have much effect on downstream consumers.

Instead a new query was added for the one location this map was needed, along
with a new comment explaining what the location is doing!
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    2eada58 View commit details
    Browse the repository at this point in the history

  10. rustc: Attach an mpsc channel to TyCtxt 

    This commit attaches a channel to the LLVM workers to the `TyCtxt` which will
later be used during the codegen query to actually send work to LLVM workers.
Otherwise this commit is just plumbing this channel throughout the compiler to
ensure it reaches the right consumers.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    3021c1d View commit details
    Browse the repository at this point in the history

  11. rustc: Move codegen to a query 

    This commit moves the actual code generation in the compiler behind a query
keyed by a codegen unit's name. This ended up entailing quite a few internal
refactorings to enable this, along with a few cut corners:

* The `OutputFilenames` structure is now tracked in the `TyCtxt` as it affects a
  whole bunch of trans and such. This is now behind a query and threaded into
  the construction of the `TyCtxt`.

* The `TyCtxt` now has a channel "out the back" intended to send data to worker
  threads in rustc_trans. This is used as a sort of side effect of the codegen
  query but morally what's happening here is the return value of the query
  (currently unit but morally a path) is only valid once the background threads
  have all finished.

* Dispatching work items to the codegen threads was refactored to only rely on
  data in `TyCtxt`, which mostly just involved refactoring where data was
  stored, moving it from the translation thread to the controller thread's
  `CodegenContext` or the like.

* A new thread locals was introduced in trans to work around the query
  system. This is used in the implementation of `assert_module_sources` which
  looks like an artifact of the old query system and will presumably go away
  once red/green is up and running.
    @alexcrichton
    alexcrichton committed Sep 17, 2017
    Configuration menu
    Copy the full SHA
    6d614dd View commit details
    Browse the repository at this point in the history