Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor type memory layouts and ABIs, to be more general and easier to optimize. #45225

Merged
merged 69 commits into from
Nov 20, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
fab2532
rustc_trans: move const & lvalue access helpers from adt.
eddyb Apr 25, 2017
9deea47
rustc_mir: always downcast enums, even if univariant.
eddyb Nov 18, 2017
f44b099
rustc_trans: avoid working with sizes/offsets and alignments as integ…
eddyb Jun 1, 2017
386d59d
rustc_trans: use a predictable layout for constant ADTs.
eddyb Jun 18, 2017
b8671be
rustc_trans: remove obsolete Type methods.
eddyb Jun 14, 2017
260c41b
rustc_trans: do not introspect LLVM aggregate field types.
eddyb Jun 18, 2017
5b1fdae
rustc_trans: use more of the trans::mir and ty::layout APIs throughout.
eddyb Jun 25, 2017
84b5a3d
rustc_trans: remove the in_memory_type_of distinction.
eddyb Jun 26, 2017
0a1fcc3
rustc_trans: use *[T; 0] for slice data pointers instead of *T.
eddyb Jun 26, 2017
8afa3a0
rustc_trans: always insert alignment padding, even before the first f…
eddyb Jun 25, 2017
44eef7c
rustc: do not inject discriminant fields into Layout::General's varia…
eddyb Sep 10, 2017
50a3fd0
rustc: remove useless 0 prefix from Layout::StructWrappedNullablePoin…
eddyb Sep 10, 2017
bc8e1f7
rustc: use an offset instead of a field path in Layout::StructWrapped…
eddyb Sep 10, 2017
aa811d7
rustc: remove source field path from Layout::StructWrappedNullablePoi…
eddyb Sep 11, 2017
8864668
rustc: re-complicate the TyLayout API and use better names.
eddyb Sep 12, 2017
8c4d5af
rustc: remove Ty::layout and move everything to layout_of.
eddyb Sep 12, 2017
9a0efea
rustc: pre-compute field placements out of Layout.
eddyb Sep 13, 2017
3071060
rustc_trans: treat General enums like unions.
eddyb Sep 13, 2017
1dc572b
rustc: represent the discriminant as a field for Layout::{Raw,StructW…
eddyb Sep 13, 2017
caef91d
rustc: introduce layout::Abi for reduced general ABI "passing style".
eddyb Sep 14, 2017
02276e9
rustc: collapse Layout::{Raw,StructWrapped}NullablePointer into one v…
eddyb Sep 15, 2017
335bd8e
rustc: do not track `non_zero` in Layout.
eddyb Sep 16, 2017
61c2bd9
rustc: use Primitive instead of Integer for CEnum and General discrim…
eddyb Sep 16, 2017
d318b9c
rustc: move CEnum's signedness into Primitive::Int.
eddyb Sep 16, 2017
658ebfc
rustc: give Layout::CEnum a discriminant field like Layout::General.
eddyb Sep 16, 2017
33a205b
rustc: collapse Layout::CEnum into Layout::General.
eddyb Sep 16, 2017
bd86f37
rustc: make Layout::NullablePointer a lot more like Layout::General.
eddyb Sep 16, 2017
bd51a2b
rustc: move size/alignment from Layout into layout::Abi.
eddyb Sep 17, 2017
ed788a6
rustc: store CachedLayout for each variant of enum Layout's instead o…
eddyb Sep 17, 2017
08f9f13
rustc: hide details in Layout in favor of Abi or FieldPlacement.
eddyb Sep 17, 2017
18d54aa
rustc: move layout::Struct into FieldPlacement/Abi.
eddyb Sep 19, 2017
fad9954
rustc: split layout::FieldPlacement::Linear back into Union and Array.
eddyb Sep 19, 2017
d0ab6e8
rustc_trans: compute LLVM types from type layouts, not Rust types.
eddyb Sep 19, 2017
b2d52d2
rustc: do not pub use Layout::* in layout.
eddyb Sep 19, 2017
f2e7e17
rustc_trans: pass OperandRef arguments to trans_intrinsic_call.
eddyb Sep 19, 2017
88f7032
rustc_trans: nest abi::ArgType's for fat pointers instead of eagerly …
eddyb Sep 20, 2017
1477119
rustc_trans: keep a layout instead of a type in {Lvalue,Operand}Ref.
eddyb Sep 20, 2017
3fd6b00
rustc_trans: query LLVM types from a layout instead of a Ty.
eddyb Sep 20, 2017
026214c
rustc: collapse Layout::FatPointer into Layout::Univariant.
eddyb Sep 20, 2017
b723af2
rustc_trans: go through layouts uniformly for fat pointers and variants.
eddyb Sep 21, 2017
b28f668
rustc: move size, align & primitive_align from Abi::Aggregate to layout.
eddyb Sep 22, 2017
018323f
rustc: collapse the remains of Layout into Variants (enums vs everyth…
eddyb Sep 22, 2017
de3e581
rustc: support u128 discriminant ranges.
eddyb Sep 23, 2017
abbc1dd
rustc: make TyLayout::field(NonZero<*T>, 0) return &T.
eddyb Sep 24, 2017
0190f27
rustc_trans: check for layout::I1 instead of TyBool.
eddyb Sep 24, 2017
b203a26
rustc: generalize layout::Variants::NicheFilling to niches other than 0.
eddyb Sep 24, 2017
5df25c4
rustc: remove redundant/unused fields from layout::Abi::Vector.
eddyb Sep 26, 2017
f62e43d
rustc: track validity ranges for layout::Abi::Scalar values.
eddyb Sep 26, 2017
ced5e04
rustc: optimize out uninhabited types and variants.
eddyb Sep 26, 2017
f8d5d0c
rustc_trans: compute better align/dereferenceable attributes from poi…
eddyb Oct 3, 2017
ac60872
rustc_trans: generate LLVM pointee types based on alignment.
eddyb Oct 4, 2017
f1b7cd9
rustc_trans: restrict "immediate pairs" to pairs of scalars.
eddyb Oct 5, 2017
cdeb4b0
rustc: encode scalar pairs in layout ABI.
eddyb Oct 6, 2017
c4d9ada
rustc: place ZSTs first during struct field reordering.
eddyb Oct 8, 2017
0b86972
rustc_trans: be more relaxed with non-lvalue consumes, especially ZSTs.
eddyb Oct 8, 2017
37a7521
rustc: unpack scalar newtype layout ABIs.
eddyb Oct 8, 2017
7a36141
rustc: unpack scalar pair newtype layout ABIs.
eddyb Oct 9, 2017
18ecc56
rustc_trans: support scalar pairs directly in the Rust ABI.
eddyb Oct 10, 2017
fa67abd
rustc: don't special-case Box<T> as having a pointer layout.
eddyb Oct 10, 2017
801a1a0
rustc_trans: remove type_is_fat_ptr and its uses.
eddyb Oct 10, 2017
d893285
rustc: use layout::Abi::ScalarPair for structs in more cases.
eddyb Oct 10, 2017
8437d7c
rustc: extend the niche-filling enum optimization past 2 variants.
eddyb Oct 12, 2017
753d582
rustc: rename CachedLayout to LayoutDetails.
eddyb Oct 28, 2017
95687bf
rustc_trans: (hack) use preferred alignment for atomic loads/stores.
eddyb Nov 19, 2017
fb83283
Don't glob-import overlapping variant names in test/codegen/match-opt…
eddyb Nov 18, 2017
b0812de
cargotest: temporarily use eddyb/servo to include servo/servo#19285.
eddyb Nov 19, 2017
88e4d2c
rustc_trans: work around i686-pc-windows-msvc byval align LLVM bug.
eddyb Nov 19, 2017
89e4373
rustc_trans: remove primitive_align optimization.
eddyb Nov 19, 2017
f9f5ab9
Revert "tests: Update run-make/issue-25581 to reflect how fat pointer…
eddyb Nov 19, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions src/liballoc/boxed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ impl<T> Place<T> for IntermediateBox<T> {
unsafe fn finalize<T>(b: IntermediateBox<T>) -> Box<T> {
let p = b.ptr as *mut T;
mem::forget(b);
mem::transmute(p)
Box::from_raw(p)
}

fn make_place<T>() -> IntermediateBox<T> {
Expand Down Expand Up @@ -300,7 +300,10 @@ impl<T: ?Sized> Box<T> {
issue = "27730")]
#[inline]
pub unsafe fn from_unique(u: Unique<T>) -> Self {
mem::transmute(u)
#[cfg(stage0)]
return mem::transmute(u);
#[cfg(not(stage0))]
return Box(u);
}

/// Consumes the `Box`, returning the wrapped raw pointer.
Expand Down Expand Up @@ -362,7 +365,14 @@ impl<T: ?Sized> Box<T> {
issue = "27730")]
#[inline]
pub fn into_unique(b: Box<T>) -> Unique<T> {
unsafe { mem::transmute(b) }
#[cfg(stage0)]
return unsafe { mem::transmute(b) };
#[cfg(not(stage0))]
return {
let unique = b.0;
mem::forget(b);
unique
};
}
}

Expand Down Expand Up @@ -627,7 +637,7 @@ impl Box<Any + Send> {
pub fn downcast<T: Any>(self) -> Result<Box<T>, Box<Any + Send>> {
<Box<Any>>::downcast(self).map_err(|s| unsafe {
// reapply the Send marker
mem::transmute::<Box<Any>, Box<Any + Send>>(s)
Box::from_raw(Box::into_raw(s) as *mut (Any + Send))
})
}
}
Expand Down
4 changes: 3 additions & 1 deletion src/librustc/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,13 @@
#![feature(const_fn)]
#![feature(core_intrinsics)]
#![feature(drain_filter)]
#![feature(i128)]
#![feature(i128_type)]
#![feature(match_default_bindings)]
#![feature(inclusive_range)]
#![feature(inclusive_range_syntax)]
#![cfg_attr(windows, feature(libc))]
#![feature(macro_vis_matcher)]
#![feature(match_default_bindings)]
#![feature(never_type)]
#![feature(nonzero)]
#![feature(quote)]
Expand Down
11 changes: 10 additions & 1 deletion src/librustc/lint/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ use middle::privacy::AccessLevels;
use rustc_serialize::{Decoder, Decodable, Encoder, Encodable};
use session::{config, early_error, Session};
use traits::Reveal;
use ty::{self, TyCtxt};
use ty::{self, TyCtxt, Ty};
use ty::layout::{LayoutError, LayoutOf, TyLayout};
use util::nodemap::FxHashMap;

use std::default::Default as StdDefault;
Expand Down Expand Up @@ -626,6 +627,14 @@ impl<'a, 'tcx> LateContext<'a, 'tcx> {
}
}

impl<'a, 'tcx> LayoutOf<Ty<'tcx>> for &'a LateContext<'a, 'tcx> {
type TyLayout = Result<TyLayout<'tcx>, LayoutError<'tcx>>;

fn layout_of(self, ty: Ty<'tcx>) -> Self::TyLayout {
(self.tcx, self.param_env.reveal_all()).layout_of(ty)
}
}

impl<'a, 'tcx> hir_visit::Visitor<'tcx> for LateContext<'a, 'tcx> {
/// Because lints are scoped lexically, we want to walk nested
/// items in the context of the outer item, so enable
Expand Down
4 changes: 2 additions & 2 deletions src/librustc/middle/mem_categorization.rs
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ impl<'tcx> cmt_<'tcx> {
adt_def.variant_with_id(variant_did)
}
_ => {
assert!(adt_def.is_univariant());
assert_eq!(adt_def.variants.len(), 1);
&adt_def.variants[0]
}
};
Expand Down Expand Up @@ -1096,7 +1096,7 @@ impl<'a, 'gcx, 'tcx> MemCategorizationContext<'a, 'gcx, 'tcx> {
-> cmt<'tcx> {
// univariant enums do not need downcasts
let base_did = self.tcx.parent_def_id(variant_did).unwrap();
if !self.tcx.adt_def(base_did).is_univariant() {
if self.tcx.adt_def(base_did).variants.len() != 1 {
let base_ty = base_cmt.ty;
let ret = Rc::new(cmt_ {
id: node.id(),
Expand Down
8 changes: 4 additions & 4 deletions src/librustc/ty/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ use ty::{PolyFnSig, InferTy, ParamTy, ProjectionTy, ExistentialPredicate, Predic
use ty::RegionKind;
use ty::{TyVar, TyVid, IntVar, IntVid, FloatVar, FloatVid};
use ty::TypeVariants::*;
use ty::layout::{Layout, TargetDataLayout};
use ty::layout::{LayoutDetails, TargetDataLayout};
use ty::maps;
use ty::steal::Steal;
use ty::BindingMode;
Expand Down Expand Up @@ -78,7 +78,7 @@ use hir;
/// Internal storage
pub struct GlobalArenas<'tcx> {
// internings
layout: TypedArena<Layout>,
layout: TypedArena<LayoutDetails>,

// references
generics: TypedArena<ty::Generics>,
Expand Down Expand Up @@ -918,7 +918,7 @@ pub struct GlobalCtxt<'tcx> {

stability_interner: RefCell<FxHashSet<&'tcx attr::Stability>>,

layout_interner: RefCell<FxHashSet<&'tcx Layout>>,
layout_interner: RefCell<FxHashSet<&'tcx LayoutDetails>>,

/// A vector of every trait accessible in the whole crate
/// (i.e. including those from subcrates). This is used only for
Expand Down Expand Up @@ -1016,7 +1016,7 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
interned
}

pub fn intern_layout(self, layout: Layout) -> &'gcx Layout {
pub fn intern_layout(self, layout: LayoutDetails) -> &'gcx LayoutDetails {
if let Some(layout) = self.layout_interner.borrow().get(&layout) {
return layout;
}
Expand Down
Loading