New line as whitespace and terminator
I am parsing javascript ES6, however I can't get it to work without semicolons. I used Tin's grammar from ES5 as base , ( https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/lang/javascript/saner/Syntax.rsc ) and already got most of the new features from ES6, but I still can't remove the need for semicolons
Take for example:
lexical Whitespace
= [t-nr ];
lexical LAYOUT
= Whitespace
| Comment
;
layout LAYOUTLIST
= LAYOUT*
!>> [t n]
!>> "/*"
!>> "//" ;
syntax Variable
= VariableIdentifier {VariableDeclaration ","}+ declarations ";"
syntax Statement
= varDecl: Variable varDecl;
I get parsing error replacing ";" to "n" on Syntax Variable , or even creating a new rule for end of statement:
syntax EOS
= ";" | "n";
The parsing error goes to the next line after the newline.
Removing from whitespaces or from the layoutlist gives me parsing error on comments at the start of the file.
And removing on the layoutlist gives me ambiguity.
javascript grammar rascal
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
add a comment |
I am parsing javascript ES6, however I can't get it to work without semicolons. I used Tin's grammar from ES5 as base , ( https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/lang/javascript/saner/Syntax.rsc ) and already got most of the new features from ES6, but I still can't remove the need for semicolons
Take for example:
lexical Whitespace
= [t-nr ];
lexical LAYOUT
= Whitespace
| Comment
;
layout LAYOUTLIST
= LAYOUT*
!>> [t n]
!>> "/*"
!>> "//" ;
syntax Variable
= VariableIdentifier {VariableDeclaration ","}+ declarations ";"
syntax Statement
= varDecl: Variable varDecl;
I get parsing error replacing ";" to "n" on Syntax Variable , or even creating a new rule for end of statement:
syntax EOS
= ";" | "n";
The parsing error goes to the next line after the newline.
Removing from whitespaces or from the layoutlist gives me parsing error on comments at the start of the file.
And removing on the layoutlist gives me ambiguity.
javascript grammar rascal
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01
add a comment |
I am parsing javascript ES6, however I can't get it to work without semicolons. I used Tin's grammar from ES5 as base , ( https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/lang/javascript/saner/Syntax.rsc ) and already got most of the new features from ES6, but I still can't remove the need for semicolons
Take for example:
lexical Whitespace
= [t-nr ];
lexical LAYOUT
= Whitespace
| Comment
;
layout LAYOUTLIST
= LAYOUT*
!>> [t n]
!>> "/*"
!>> "//" ;
syntax Variable
= VariableIdentifier {VariableDeclaration ","}+ declarations ";"
syntax Statement
= varDecl: Variable varDecl;
I get parsing error replacing ";" to "n" on Syntax Variable , or even creating a new rule for end of statement:
syntax EOS
= ";" | "n";
The parsing error goes to the next line after the newline.
Removing from whitespaces or from the layoutlist gives me parsing error on comments at the start of the file.
And removing on the layoutlist gives me ambiguity.
javascript grammar rascal
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
I am parsing javascript ES6, however I can't get it to work without semicolons. I used Tin's grammar from ES5 as base , ( https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/lang/javascript/saner/Syntax.rsc ) and already got most of the new features from ES6, but I still can't remove the need for semicolons
Take for example:
lexical Whitespace
= [t-nr ];
lexical LAYOUT
= Whitespace
| Comment
;
layout LAYOUTLIST
= LAYOUT*
!>> [t n]
!>> "/*"
!>> "//" ;
syntax Variable
= VariableIdentifier {VariableDeclaration ","}+ declarations ";"
syntax Statement
= varDecl: Variable varDecl;
I get parsing error replacing ";" to "n" on Syntax Variable , or even creating a new rule for end of statement:
syntax EOS
= ";" | "n";
The parsing error goes to the next line after the newline.
Removing from whitespaces or from the layoutlist gives me parsing error on comments at the start of the file.
And removing on the layoutlist gives me ambiguity.
javascript grammar rascal
javascript grammar rascal
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
asked Nov 21 '18 at 10:39
Adriano TorresAdriano Torres
1313
1313
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01
add a comment |
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01
add a comment |
1 Answer
1
active
oldest
votes
considering a smaller language, I am dealing with this problem (end of statement might be a ";", a "n", or an end of file) using:
syntax EOS = ";" // handle ";" as end of statement
| Epsilon $ // handle "n" and EOF as end of statement
| Epsilon >> "}" // it might be also necessary (from ANTLR grammar)
;
syntax Epsilon = ;
//and a test case
test bool testParseEOS() {
try {
parse(#BlockStmt, "{x := x + 1; y := 10 n y := z + 1; x := z }");
return true;
}
catch ParseError(loc l): {
println("I found a parse error at line <l.begin.line>, column <l.begin.column>");
return false;
}
}
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53410259%2fnew-line-as-whitespace-and-terminator%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
considering a smaller language, I am dealing with this problem (end of statement might be a ";", a "n", or an end of file) using:
syntax EOS = ";" // handle ";" as end of statement
| Epsilon $ // handle "n" and EOF as end of statement
| Epsilon >> "}" // it might be also necessary (from ANTLR grammar)
;
syntax Epsilon = ;
//and a test case
test bool testParseEOS() {
try {
parse(#BlockStmt, "{x := x + 1; y := 10 n y := z + 1; x := z }");
return true;
}
catch ParseError(loc l): {
println("I found a parse error at line <l.begin.line>, column <l.begin.column>");
return false;
}
}
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
add a comment |
considering a smaller language, I am dealing with this problem (end of statement might be a ";", a "n", or an end of file) using:
syntax EOS = ";" // handle ";" as end of statement
| Epsilon $ // handle "n" and EOF as end of statement
| Epsilon >> "}" // it might be also necessary (from ANTLR grammar)
;
syntax Epsilon = ;
//and a test case
test bool testParseEOS() {
try {
parse(#BlockStmt, "{x := x + 1; y := 10 n y := z + 1; x := z }");
return true;
}
catch ParseError(loc l): {
println("I found a parse error at line <l.begin.line>, column <l.begin.column>");
return false;
}
}
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
add a comment |
considering a smaller language, I am dealing with this problem (end of statement might be a ";", a "n", or an end of file) using:
syntax EOS = ";" // handle ";" as end of statement
| Epsilon $ // handle "n" and EOF as end of statement
| Epsilon >> "}" // it might be also necessary (from ANTLR grammar)
;
syntax Epsilon = ;
//and a test case
test bool testParseEOS() {
try {
parse(#BlockStmt, "{x := x + 1; y := 10 n y := z + 1; x := z }");
return true;
}
catch ParseError(loc l): {
println("I found a parse error at line <l.begin.line>, column <l.begin.column>");
return false;
}
}
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
considering a smaller language, I am dealing with this problem (end of statement might be a ";", a "n", or an end of file) using:
syntax EOS = ";" // handle ";" as end of statement
| Epsilon $ // handle "n" and EOF as end of statement
| Epsilon >> "}" // it might be also necessary (from ANTLR grammar)
;
syntax Epsilon = ;
//and a test case
test bool testParseEOS() {
try {
parse(#BlockStmt, "{x := x + 1; y := 10 n y := z + 1; x := z }");
return true;
}
catch ParseError(loc l): {
println("I found a parse error at line <l.begin.line>, column <l.begin.column>");
return false;
}
}
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
edited Dec 29 '18 at 9:13
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
answered Dec 28 '18 at 15:26
Rodrigo BonifacioRodrigo Bonifacio
1478
1478
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53410259%2fnew-line-as-whitespace-and-terminator%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
This is tricky stuff! You need a number of advanced features and a lot of thinking and debugging you get this right. I'll try and list the useful features in an answer tomorrow if I feel well enough. For now it's best to go for an ambiguous grammar first, which exactly describes js syntax input but is an over-approximation (too many) for its syntax trees. Using step-by-step grammar changes, and adding disambiguation constructs you can filter the superfluous trees. Better make a regression test set using rascals test function feature as well, to help in trusting the changes.
– Jurgen Vinju
Nov 21 '18 at 11:09
Very tricky indeed! Still on in, but have not managed to make any progress. Anyway, thank you for the suggestions. I will keep working on that
– Adriano Torres
Dec 4 '18 at 12:01