Skip to content
This repository has been archived by the owner on Dec 15, 2022. It is now read-only.

New variables structure #53

Merged
merged 2 commits into from
Jun 12, 2016
Merged

New variables structure #53

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6991cbc
New variables structure
noseglid Jun 10, 2016
70f9a2b
Code style and review fixes
noseglid Jun 11, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 62 additions & 56 deletions grammars/java.cson
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,9 +343,6 @@
{
'include': '#enums'
}
{
'include': '#variables'
}
{
'include': '#generics'
}
Expand All @@ -355,13 +352,30 @@
{
'include': '#annotations'
}
{
'include': '#member-variables'
}
{
'include': '#storage-modifiers'
}
{
'include': '#code'
}
]
'anonymous-block-and-instance-initializer':
'begin': '{'
'beginCaptures':
'0':
'name': 'punctuation.section.block.begin.bracket.curly.java'
'end': '}'
'endCaptures':
'0':
'name': 'punctuation.section.block.end.bracket.curly.java'
'patterns': [
{
'include': '#code'
}
]
'code':
'patterns': [
{
Expand All @@ -377,19 +391,7 @@
'include': '#class'
}
{
'begin': '{'
'beginCaptures':
'0':
'name': 'punctuation.section.block.begin.bracket.curly.java'
'end': '}'
'endCaptures':
'0':
'name': 'punctuation.section.block.end.bracket.curly.java'
'patterns': [
{
'include': '#code'
}
]
'include': '#anonymous-block-and-instance-initializer'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actual changes, just some splitting up code to make it more readable.

}
{
'include': '#assertions'
Expand Down Expand Up @@ -424,6 +426,9 @@
{
'include': '#properties'
}
{
'include': '#variables'
}
{
'include': '#strings'
}
Expand Down Expand Up @@ -744,9 +749,6 @@
'end': '(?=})'
'contentName': 'meta.method.body.java'
'patterns': [
{
'include': '#variables'
}
{
'include': '#code'
}
Expand Down Expand Up @@ -999,50 +1001,54 @@
}
]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this. To be frank, I don't know how It worked with it before, since ; is matched by #code which was included before too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surprisingly first-mate actually recognizes this. The super-sparse documentation seems to suggest that this only matters when the end patterns of a subpattern and the overarching end are exactly the same, but if everything already works, then no need to have it!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. In my manual tests it seems to work well, and the specs agrees.

'variables':
'applyEndPatternLast': 1
'begin': '''
(?x:(?=
(?:
Copy link
Contributor

@winstliu winstliu Jun 10, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're in a lookahead all the non-capturing groups aren't needed. We're not capturing anything anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right you are. Copied this from the previous definition, but might as well simplify!

(?:void|boolean|byte|char|short|int|float|long|double)
|
(?:(?:[a-z]\\w*\\s*\\.\\s*)*[A-Z]+\\w*) # e.g. javax.ws.rs.Response
)
\\s*
[\\w\\d_<>\\[\\],\\?][\\w\\d_<>\\[\\],\\?\\s]*
\\s*(?:=|;)
))
'''
'patterns': [
{
'begin': '''
(?x:(?=
(?:
(?:private|protected|public|native|synchronized|abstract|threadsafe|transient|static|final) # visibility/modifier
|
(?:def)
|
(?:void|boolean|byte|char|short|int|float|long|double)
|
(?:(?:[a-z]\\w*\\s*\\.\\s*)*[A-Z]+\\w*) # object type
)
\\s+
(?!private|protected|public|native|synchronized|abstract|threadsafe|transient|static|final)
[\\w\\d_<>\\[\\],\\?][\\w\\d_<>\\[\\],\\?\\s]*
\\s*(=|;)
))
'''
'match': '([A-Za-z$_][\\w$]*)(?=(\\[\\])?\\s*(?=;|=|,))'
Copy link
Contributor

@winstliu winstliu Jun 10, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can be spaces before the array brackets, eg int test [] = new int[3];.

Also, the variable declaration doesn't strictly have to be on the same line...but I don't think we can capture all the scenarios with single-line regex. /shrug

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

'captures':
'1':
'name': 'variable.definition.java'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized after I merged the PR adding this that the scope should probably be variable.other.definition.java instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that seems to be the consesus in textmate. Changing.

}
{
'include': '#all-types'
}
{
'begin': '='
'beginCaptures':
'0':
'name': 'keyword.operator.assignment.java'
'end': '(?=;)'
'name': 'meta.definition.variable.java'
'patterns': [
{
'match': '([A-Za-z$_][\\w$]*)(?=\\s*(;|=|,))'
'captures':
'1':
'name': 'variable.definition.java'
}
{
'begin': '='
'beginCaptures':
'0':
'name': 'keyword.operator.assignment.java'
'end': '(?=;)'
'patterns': [
{
'include': '#code'
}
]
}
{
'include': '#code'
}
]
}
{
'include': '#code'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To get e.g. punctuation in int a, b, c;

}
]
'end': '(?=;)'
'name': 'meta.definition.variable.java'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in the other PR, end and name before patterns.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

'member-variables':
'begin': '(?=private|protected|public|native|synchronized|abstract|threadsafe|transient|static|final)'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your indentation is off here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch. Seems to work all the same. I hate CoffeeScript/cson for this reason. "Indentation is dead important! sometimes..."

'patterns': [
{
'include': '#storage-modifiers'
}
{
'include': '#variables'
}
]
'end': '(?=;)'
50 changes: 26 additions & 24 deletions spec/java-spec.coffee
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,10 @@ describe 'Java grammar', ->
it 'tokenizes punctuation', ->
{tokens} = grammar.tokenizeLine 'int a, b, c;'

expect(tokens[2]).toEqual value: ',', scopes: ['source.java', 'punctuation.separator.delimiter.java']
expect(tokens[4]).toEqual value: ',', scopes: ['source.java', 'punctuation.separator.delimiter.java']
expect(tokens[6]).toEqual value: ';', scopes: ['source.java', 'punctuation.terminator.java']
console.log tokens
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch. sloppy :( I blame late night coding

expect(tokens[3]).toEqual value: ',', scopes: ['source.java', 'meta.definition.variable.java', 'punctuation.separator.delimiter.java']
expect(tokens[6]).toEqual value: ',', scopes: ['source.java', 'meta.definition.variable.java', 'punctuation.separator.delimiter.java']
expect(tokens[9]).toEqual value: ';', scopes: ['source.java', 'punctuation.terminator.java']

{tokens} = grammar.tokenizeLine 'a.b(1, 2, c);'

Expand Down Expand Up @@ -379,24 +380,25 @@ describe 'Java grammar', ->
expect(lines[0][17]).toEqual value: ' ', scopes: ['source.java', 'meta.class.java']
expect(lines[0][18]).toEqual value: 'Integer', scopes: ['source.java', 'meta.class.java', 'storage.type.generic.java']
expect(lines[0][19]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'punctuation.bracket.angle.java']
expect(lines[2][1]).toEqual value: 'HashMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.java']
expect(lines[2][2]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[2][3]).toEqual value: 'Integer', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.generic.java']
expect(lines[2][4]).toEqual value: ',', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.separator.delimiter.java']
expect(lines[2][6]).toEqual value: 'String', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.generic.java']
expect(lines[2][7]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[2][13]).toEqual value: 'HashMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.java']
expect(lines[2][14]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[2][15]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[3][1]).toEqual value: 'CodeMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.java']
expect(lines[3][2]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[3][3]).toEqual value: 'String', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.generic.java']
expect(lines[3][4]).toEqual value: ',', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.separator.delimiter.java']
expect(lines[3][6]).toEqual value: '?', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.generic.wildcard.java']
expect(lines[3][8]).toEqual value: 'extends', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.modifier.extends.java']
expect(lines[3][10]).toEqual value: 'ArrayList', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.type.generic.java']
expect(lines[3][11]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.bracket.angle.java']
expect(lines[3][12]).toEqual value: ' codemap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java']
expect(lines[2][1]).toEqual value: 'HashMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.java']
expect(lines[2][2]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[2][3]).toEqual value: 'Integer', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.generic.java']
expect(lines[2][4]).toEqual value: ',', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.separator.delimiter.java']
expect(lines[2][6]).toEqual value: 'String', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.generic.java']
expect(lines[2][7]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[2][9]).toEqual value: 'map', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'variable.definition.java']
expect(lines[2][15]).toEqual value: 'HashMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.java']
expect(lines[2][16]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[2][17]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[3][1]).toEqual value: 'CodeMap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.java']
expect(lines[3][2]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[3][3]).toEqual value: 'String', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.generic.java']
expect(lines[3][4]).toEqual value: ',', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.separator.delimiter.java']
expect(lines[3][6]).toEqual value: '?', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.generic.wildcard.java']
expect(lines[3][8]).toEqual value: 'extends', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.modifier.extends.java']
expect(lines[3][10]).toEqual value: 'ArrayList', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.generic.java']
expect(lines[3][11]).toEqual value: '>', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'punctuation.bracket.angle.java']
expect(lines[3][13]).toEqual value: 'codemap', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'variable.definition.java']
expect(lines[4][1]).toEqual value: 'C', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.method.java', 'meta.method.identifier.java', 'entity.name.function.java']
expect(lines[4][3]).toEqual value: 'Map', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.method.java', 'meta.method.identifier.java', 'storage.type.java']
expect(lines[4][4]).toEqual value: '<', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.method.java', 'meta.method.identifier.java', 'punctuation.bracket.angle.java']
Expand Down Expand Up @@ -448,14 +450,14 @@ describe 'Java grammar', ->
}
'''

expect(lines[2][1]).toEqual value: 'private', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.modifier.java']
expect(lines[2][2]).toEqual value: ' ', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java']
expect(lines[2][1]).toEqual value: 'private', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.modifier.java']
expect(lines[2][2]).toEqual value: ' ', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java']
expect(lines[2][3]).toEqual value: 'int', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.primitive.java']
expect(lines[2][4]).toEqual value: ' ', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java']
expect(lines[2][5]).toEqual value: 'variable', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'variable.definition.java']
expect(lines[2][6]).toEqual value: ';', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'punctuation.terminator.java']

expect(lines[3][1]).toEqual value: 'public', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.modifier.java']
expect(lines[3][1]).toEqual value: 'public', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'storage.modifier.java']
expect(lines[3][3]).toEqual value: 'Object', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.object.array.java']
expect(lines[3][4]).toEqual value: '[', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.object.array.java', 'punctuation.bracket.square.java']
expect(lines[3][5]).toEqual value: ']', scopes: ['source.java', 'meta.class.java', 'meta.class.body.java', 'meta.definition.variable.java', 'storage.type.object.array.java', 'punctuation.bracket.square.java']
Expand Down