Implemented a smarter TokenStream concatenation system #35539
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
r? @sfackler (rust_highfive has picked a reviewer for you, use r? to override) |
|
r? @nrc |
| fn mk_spanned_empty(sp: Span) -> TokenStream { | ||
| TokenStream { ts: InternalTS::Empty(sp) } | ||
| } | ||
|
|
||
| // Construct an leaf node with a 0 offset and length equivalent to the input. |
|
Some pretty minor comments inline, otherwise looks good. |
| @@ -340,6 +340,8 @@ pub struct TokenStream { | |||
| ts: InternalTS, | |||
| } | |||
|
|
|||
| static LEAF_SIZE : usize = 32; | |||
There was a problem hiding this comment.
should be const, not static, also no space before the colon
There was a problem hiding this comment.
and could you add a comment please?
|
r=me with those last two things addressed |
|
Addressed. |
…e more efficient tokenstream usages.
|
@bors: r+ |
|
📌 Commit 16cc8a7 has been approved by |
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Aug 13, 2016
Implemented a smarter TokenStream concatenation system The new algorithm performs 'aggressive compacting' during concatenation as follows: - If the nodes' combined total total length is less than 32, we copy both of them into a new vector and build a new leaf node. - If one node is an internal node and the other is a 'small' leaf (length<32), we recur down the internal node on the appropriate side. - Otherwise, we construct a new internal node that points to them as left and right. This should produce notably better behavior than the current concatenation implementation.
|
@bors rollup |
eddyb
added a commit
to eddyb/rust
that referenced
this pull request
Aug 14, 2016
Implemented a smarter TokenStream concatenation system The new algorithm performs 'aggressive compacting' during concatenation as follows: - If the nodes' combined total total length is less than 32, we copy both of them into a new vector and build a new leaf node. - If one node is an internal node and the other is a 'small' leaf (length<32), we recur down the internal node on the appropriate side. - Otherwise, we construct a new internal node that points to them as left and right. This should produce notably better behavior than the current concatenation implementation.
eddyb
added a commit
to eddyb/rust
that referenced
this pull request
Aug 14, 2016
Implemented a smarter TokenStream concatenation system The new algorithm performs 'aggressive compacting' during concatenation as follows: - If the nodes' combined total total length is less than 32, we copy both of them into a new vector and build a new leaf node. - If one node is an internal node and the other is a 'small' leaf (length<32), we recur down the internal node on the appropriate side. - Otherwise, we construct a new internal node that points to them as left and right. This should produce notably better behavior than the current concatenation implementation.
bors
added a commit
that referenced
this pull request
Aug 14, 2016
Rollup of 30 pull requests - Successful merges: #34941, #35392, #35444, #35447, #35491, #35533, #35539, #35558, #35573, #35574, #35577, #35586, #35588, #35594, #35596, #35597, #35598, #35606, #35611, #35615, #35616, #35620, #35622, #35640, #35643, #35644, #35646, #35647, #35648, #35661 - Failed merges: #35395, #35415
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new algorithm performs 'aggressive compacting' during concatenation as follows:
them into a new vector and build a new leaf node.
we recur down the internal node on the appropriate side.
right.
This should produce notably better behavior than the current concatenation implementation.