Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use QuoteNode and LineNumberNode consistently #23885

Merged
merged 1 commit into from
Sep 28, 2017
Merged

use QuoteNode and LineNumberNode consistently #23885

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ Language changes
* Prefix `&` for by-reference arguments to `ccall` has been deprecated in favor of
`Ref` argument types ([#6080]).

* All line numbers in ASTs are represented by `LineNumberNode`s; the `:line` expression
head is no longer used. `QuoteNode`s are also consistently used for quoted symbols instead
of the `:quote` expression head (though `:quote` `Expr`s are still used for quoted
expressions) ([#23885]).

Breaking changes
----------------

Expand Down
1 change: 0 additions & 1 deletion base/codevalidation.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ const VALID_EXPR_HEADS = ObjectIdDict(
:call => 1:typemax(Int),
:invoke => 2:typemax(Int),
:static_parameter => 1:1,
:line => 1:3,
:gotoifnot => 2:2,
:(&) => 1:1,
:(=) => 2:2,
Expand Down
32 changes: 24 additions & 8 deletions doc/src/devdocs/ast.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,6 @@ These symbols appear in the `head` field of `Expr`s in lowered form.

Reference a static parameter by index.

* `line`

Line number and file name metadata. Unlike a `LineNumberNode`, can also contain a file name.

* `gotoifnot`

Conditional branch. If `args[1]` is false, goes to label identified in `args[2]`.
Expand Down Expand Up @@ -337,10 +333,11 @@ Boolean properties:

## Surface syntax AST

Front end ASTs consist entirely of `Expr`s and atoms (e.g. symbols, numbers). There is generally
a different expression head for each visually distinct syntactic form. Examples will be given
in s-expression syntax. Each parenthesized list corresponds to an Expr, where the first element
is the head. For example `(call f x)` corresponds to `Expr(:call, :f, :x)` in Julia.
Front end ASTs consist almost entirely of `Expr`s and atoms (e.g. symbols, numbers).
There is generally a different expression head for each visually distinct syntactic form.
Examples will be given in s-expression syntax.
Each parenthesized list corresponds to an Expr, where the first element is the head.
For example `(call f x)` corresponds to `Expr(:call, :f, :x)` in Julia.

### Calls

Expand Down Expand Up @@ -526,3 +523,22 @@ The first argument is a boolean telling whether the type is mutable.
`try` blocks parse as `(try try_block var catch_block finally_block)`. If no variable is present
after `catch`, `var` is `#f`. If there is no `finally` clause, then the last argument is not present.

### Quote expressions

Julia source syntax forms for code quoting (`quote` and `:( )`) support interpolation with `$`.
In Lisp terminology, this means they are actually "backquote" or "quasiquote" forms.
Internally, there is also a need for code quoting without interpolation.
In Julia's scheme code, non-interpolating quote is represented with the expression head `inert`.

`inert` expressions are converted to Julia `QuoteNode` objects.
These objects wrap a single value of any type, and when evaluated simply return that value.

A `quote` expression whose argument is an atom also gets converted to a `QuoteNode`.

### Line numbers

Source location information is represented as `(line line_num file_name)` where the third
component is optional (and omitted when the current line number, but not file name,
changes).

These expressions are represented as `LineNumberNode`s in Julia.
173 changes: 64 additions & 109 deletions src/ast.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ struct macroctx_stack {
#define JL_AST_PRESERVE_POP(ctx, old) \
ctx->module = (old)

static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, int expronly, jl_module_t *mod);
static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, jl_module_t *mod);
static value_t julia_to_scm(fl_context_t *fl_ctx, jl_value_t *v);
static jl_value_t *jl_expand_macros(jl_value_t *expr, jl_module_t *inmodule, struct macroctx_stack *macroctx, int onelevel);

Expand All @@ -132,7 +132,7 @@ value_t fl_defined_julia_global(fl_context_t *fl_ctx, value_t *args, uint32_t na
// tells whether a var is defined in and *by* the current module
argcount(fl_ctx, "defined-julia-global", nargs, 1);
jl_module_t *mod = ctx->module;
jl_sym_t *sym = (jl_sym_t*)scm_to_julia(fl_ctx, args[0], 0, mod);
jl_sym_t *sym = (jl_sym_t*)scm_to_julia(fl_ctx, args[0], mod);
if (jl_is_globalref(sym)) {
mod = jl_globalref_mod(sym);
sym = jl_globalref_name(sym);
Expand Down Expand Up @@ -397,14 +397,14 @@ static jl_sym_t *scmsym_to_julia(fl_context_t *fl_ctx, value_t s)
return jl_symbol(symbol_name(fl_ctx, s));
}

static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, int expronly, jl_module_t *mod);
static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, jl_module_t *mod);

static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, int expronly, jl_module_t *mod)
static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, jl_module_t *mod)
{
jl_value_t *v = NULL;
JL_GC_PUSH1(&v);
JL_TRY {
v = scm_to_julia_(fl_ctx, e, expronly, mod);
v = scm_to_julia_(fl_ctx, e, mod);
}
JL_CATCH {
// if expression cannot be converted, replace with error expr
Expand All @@ -418,7 +418,7 @@ static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, int expronly, j

extern int64_t conv_to_int64(void *data, numerictype_t tag);

static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, int eo, jl_module_t *mod)
static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, jl_module_t *mod)
{
if (fl_isnumber(fl_ctx, e)) {
int64_t i64;
Expand Down Expand Up @@ -489,83 +489,62 @@ static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, int eo, jl_mod
e = cdr_(e);
else
n++;
if (!eo) {
if (sym == line_sym && (n == 1 || n == 2)) {
jl_value_t *linenum = scm_to_julia_(fl_ctx, car_(e), 0, mod);
jl_value_t *file = jl_nothing;
JL_GC_PUSH2(&linenum, &file);
if (n == 2)
file = scm_to_julia_(fl_ctx, car_(cdr_(e)), 0, mod);
jl_value_t *temp = jl_new_struct(jl_linenumbernode_type, linenum, file);
JL_GC_POP();
return temp;
}
jl_value_t *scmv = NULL, *temp = NULL;
JL_GC_PUSH1(&scmv);
if (sym == label_sym) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
temp = jl_new_struct(jl_labelnode_type, scmv);
JL_GC_POP();
return temp;
}
if (sym == goto_sym) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
temp = jl_new_struct(jl_gotonode_type, scmv);
JL_GC_POP();
return temp;
}
if (sym == inert_sym || (sym == quote_sym && (!iscons(car_(e))))) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
temp = jl_new_struct(jl_quotenode_type, scmv);
JL_GC_POP();
return temp;
}
if (sym == top_sym) {
assert(mod && "top should not be generated by the parser");
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
assert(jl_is_symbol(scmv));
temp = jl_module_globalref(jl_base_relative_to(mod), (jl_sym_t*)scmv);
JL_GC_POP();
return temp;
}
if (sym == core_sym) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
assert(jl_is_symbol(scmv));
temp = jl_module_globalref(jl_core_module, (jl_sym_t*)scmv);
JL_GC_POP();
return temp;
}
if (sym == globalref_sym) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
temp = scm_to_julia_(fl_ctx, car_(cdr_(e)), 0, mod);
assert(jl_is_module(scmv));
assert(jl_is_symbol(temp));
temp = jl_module_globalref((jl_module_t*)scmv, (jl_sym_t*)temp);
JL_GC_POP();
return temp;
}
if (sym == newvar_sym) {
scmv = scm_to_julia_(fl_ctx, car_(e), 0, mod);
temp = jl_new_struct(jl_newvarnode_type, scmv);
JL_GC_POP();
return temp;
}
// nodes with special representations
jl_value_t *ex = NULL, *temp = NULL;
if (sym == line_sym && (n == 1 || n == 2)) {
jl_value_t *linenum = scm_to_julia_(fl_ctx, car_(e), mod);
jl_value_t *file = jl_nothing;
JL_GC_PUSH2(&linenum, &file);
if (n == 2)
file = scm_to_julia_(fl_ctx, car_(cdr_(e)), mod);
temp = jl_new_struct(jl_linenumbernode_type, linenum, file);
JL_GC_POP();
return temp;
}
else if (sym == inert_sym && !iscons(car_(e))) {
sym = quote_sym;
}
jl_value_t *ex = (jl_value_t*)jl_exprn(sym, n);
JL_GC_PUSH1(&ex);
// allocate a fresh args array for empty exprs passed to macros
if (eo && n == 0) {
((jl_expr_t*)ex)->args = jl_alloc_vec_any(0);
jl_gc_wb(ex, ((jl_expr_t*)ex)->args);
if (sym == label_sym) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
temp = jl_new_struct(jl_labelnode_type, ex);
}
else if (sym == goto_sym) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
temp = jl_new_struct(jl_gotonode_type, ex);
}
else if (sym == newvar_sym) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
temp = jl_new_struct(jl_newvarnode_type, ex);
}
else if (sym == globalref_sym) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
temp = scm_to_julia_(fl_ctx, car_(cdr_(e)), mod);
assert(jl_is_module(ex));
assert(jl_is_symbol(temp));
temp = jl_module_globalref((jl_module_t*)ex, (jl_sym_t*)temp);
}
else if (sym == top_sym) {
assert(mod && "top should not be generated by the parser");
ex = scm_to_julia_(fl_ctx, car_(e), mod);
assert(jl_is_symbol(ex));
temp = jl_module_globalref(jl_base_relative_to(mod), (jl_sym_t*)ex);
}
else if (sym == core_sym) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
assert(jl_is_symbol(ex));
temp = jl_module_globalref(jl_core_module, (jl_sym_t*)ex);
}
else if (sym == inert_sym || (sym == quote_sym && (!iscons(car_(e))))) {
ex = scm_to_julia_(fl_ctx, car_(e), mod);
temp = jl_new_struct(jl_quotenode_type, ex);
}
if (temp) {
JL_GC_POP();
return temp;
}
ex = (jl_value_t*)jl_exprn(sym, n);
size_t i;
for (i = 0; i < n; i++) {
assert(iscons(e));
jl_array_ptr_set(((jl_expr_t*)ex)->args, i, scm_to_julia_(fl_ctx, car_(e), eo, mod));
jl_array_ptr_set(((jl_expr_t*)ex)->args, i, scm_to_julia_(fl_ctx, car_(e), mod));
e = cdr_(e);
}
if (sym == lambda_sym)
Expand Down Expand Up @@ -705,7 +684,7 @@ JL_DLLEXPORT jl_value_t *jl_parse_input_line(const char *str, size_t len, const
value_t s = cvalue_static_cstrn(fl_ctx, str, len);
value_t files = cvalue_static_cstrn(fl_ctx, filename, filename_len);
value_t e = fl_applyn(fl_ctx, 2, symbol_value(symbol(fl_ctx, "jl-parse-string")), s, files);
jl_value_t *res = e == fl_ctx->FL_EOF ? jl_nothing : scm_to_julia(fl_ctx, e, 0, NULL);
jl_value_t *res = e == fl_ctx->FL_EOF ? jl_nothing : scm_to_julia(fl_ctx, e, NULL);
jl_ast_ctx_leave(ctx);
return res;
}
Expand Down Expand Up @@ -734,7 +713,7 @@ JL_DLLEXPORT jl_value_t *jl_parse_string(const char *str, size_t len,
if (e == fl_ctx->FL_EOF)
expr = jl_nothing;
else
expr = scm_to_julia(fl_ctx, e, 0, NULL);
expr = scm_to_julia(fl_ctx, e, NULL);

pos1 = jl_box_long(tosize(fl_ctx, cdr_(p), "parse"));
jl_ast_ctx_leave(ctx);
Expand Down Expand Up @@ -796,14 +775,14 @@ jl_value_t *jl_parse_eval_all(const char *fname,
{
JL_TIMING(LOWERING);
if (fl_ctx->T == fl_applyn(fl_ctx, 1, symbol_value(symbol(fl_ctx, "contains-macrocall")), expression)) {
form = scm_to_julia(fl_ctx, expression, 0, inmodule);
form = scm_to_julia(fl_ctx, expression, inmodule);
form = jl_expand_macros(form, inmodule, NULL, 0);
expression = julia_to_scm(fl_ctx, form);
}
expression = fl_applyn(fl_ctx, 1, symbol_value(symbol(fl_ctx, "jl-expand-to-thunk")), expression);
}
jl_get_ptls_states()->world_age = jl_world_counter;
form = scm_to_julia(fl_ctx, expression, 0, inmodule);
form = scm_to_julia(fl_ctx, expression, inmodule);
jl_sym_t *head = NULL;
if (jl_is_expr(form))
head = ((jl_expr_t*)form)->head;
Expand All @@ -813,8 +792,6 @@ jl_value_t *jl_parse_eval_all(const char *fname,
jl_errorf("syntax: %s", jl_string_data(jl_exprarg(form, 0)));
else if (head == error_sym)
jl_interpret_toplevel_expr_in(inmodule, form, NULL, NULL);
else if (head == line_sym)
jl_lineno = jl_unbox_long(jl_exprarg(form, 0));
else if (jl_is_linenode(form))
jl_lineno = jl_linenode_line(form);
else
Expand Down Expand Up @@ -881,7 +858,7 @@ jl_value_t *jl_call_scm_on_ast(const char *funcname, jl_value_t *expr, jl_module
JL_AST_PRESERVE_PUSH(ctx, old_roots, inmodule);
value_t arg = julia_to_scm(fl_ctx, expr);
value_t e = fl_applyn(fl_ctx, 1, symbol_value(symbol(fl_ctx, funcname)), arg);
jl_value_t *result = scm_to_julia(fl_ctx, e, 0, inmodule);
jl_value_t *result = scm_to_julia(fl_ctx, e, inmodule);
JL_AST_PRESERVE_POP(ctx, old_roots);
jl_ast_ctx_leave(ctx);
return result;
Expand Down Expand Up @@ -925,15 +902,9 @@ JL_DLLEXPORT jl_value_t *jl_copy_ast(jl_value_t *expr)
size_t i, l = jl_array_len(e->args);
jl_expr_t *ne = jl_exprn(e->head, l);
JL_GC_PUSH2(&ne, &expr);
if (l == 0) {
ne->args = jl_alloc_vec_any(0);
jl_gc_wb(ne, ne->args);
}
else {
for (i = 0; i < l; i++) {
jl_value_t *a = jl_exprarg(e, i);
jl_exprargset(ne, i, jl_copy_ast(a));
}
for (i = 0; i < l; i++) {
jl_value_t *a = jl_exprarg(e, i);
jl_exprargset(ne, i, jl_copy_ast(a));
}
JL_GC_POP();
return (jl_value_t*)ne;
Expand Down Expand Up @@ -1005,23 +976,7 @@ static jl_value_t *jl_invoke_julia_macro(jl_array_t *args, jl_module_t *inmodule
// __source__ argument
jl_value_t *lno = jl_array_ptr_ref(args, 1);
margs[1] = lno;
if (jl_is_expr(lno) && ((jl_expr_t*)lno)->head == line_sym) {
jl_value_t *file = jl_nothing;
jl_value_t *line = NULL;
switch (jl_expr_nargs(lno)) {
case 2:
file = jl_exprarg(lno, 1); // file
JL_FALLTHROUGH;
case 1:
line = jl_exprarg(lno, 0); // line
JL_FALLTHROUGH;
default: ;
}
if (line == NULL)
line = jl_box_long(0);
margs[1] = jl_new_struct(jl_linenumbernode_type, line, file);
}
else if (!jl_typeis(lno, jl_linenumbernode_type)) {
if (!jl_typeis(lno, jl_linenumbernode_type)) {
margs[1] = jl_new_struct(jl_linenumbernode_type, jl_box_long(0), jl_nothing);
}
margs[2] = (jl_value_t*)inmodule;
Expand Down
2 changes: 1 addition & 1 deletion src/builtins.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -999,7 +999,7 @@ JL_CALLABLE(jl_f_invoke_kwsorter)
jl_expr_t *jl_exprn(jl_sym_t *head, size_t n)
{
jl_ptls_t ptls = jl_get_ptls_states();
jl_array_t *ar = n==0 ? (jl_array_t*)jl_an_empty_vec_any : jl_alloc_vec_any(n);
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix (simplify) jl_copy_ast also now

jl_array_t *ar = jl_alloc_vec_any(n);
JL_GC_PUSH1(&ar);
jl_expr_t *ex = (jl_expr_t*)jl_gc_alloc(ptls, sizeof(jl_expr_t),
jl_expr_type);
Expand Down
13 changes: 4 additions & 9 deletions src/codegen.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1982,7 +1982,7 @@ static void simple_use_analysis(jl_codectx_t &ctx, jl_value_t *expr)
// don't consider assignment LHS as a variable "use"
simple_use_analysis(ctx, jl_exprarg(e, 1));
}
else if (e->head != line_sym) {
else {
size_t elen = jl_array_dim0(e->args);
for (i = 0; i < elen; i++) {
simple_use_analysis(ctx, jl_exprarg(e, i));
Expand Down Expand Up @@ -3664,7 +3664,7 @@ static void emit_stmtpos(jl_codectx_t &ctx, jl_value_t *expr)
jl_expr_t *ex = (jl_expr_t*)expr;
jl_value_t **args = (jl_value_t**)jl_array_data(ex->args);
jl_sym_t *head = ex->head;
if (head == line_sym || head == meta_sym || head == inbounds_sym) {
if (head == meta_sym || head == inbounds_sym) {
// some expression types are metadata and can be ignored
// in statement position
return;
Expand Down Expand Up @@ -5425,14 +5425,9 @@ static std::unique_ptr<Module> emit_function(
}
}
#endif
if (jl_is_linenode(stmt) || (expr && expr->head == line_sym)) {
if (jl_is_linenode(stmt)) {
ssize_t lno = -1;
if (jl_is_linenode(stmt)) {
lno = jl_linenode_line(stmt);
}
else {
lno = jl_unbox_long(jl_exprarg(stmt,0));
}
lno = jl_linenode_line(stmt);
MDNode *inlinedAt = NULL;
if (DI_stack.size() > 0) {
inlinedAt = DI_stack.back().loc;
Expand Down
Loading