Bug 949 – Wrong spec/compiler behaviour for Strings, Integers and Floats

Decimal: 0 NonZeroDigit NonZeroDigit DecimalDigits DecimalFloat: DecimalDigits . DecimalDigits . DecimalDigits DecimalDigits . DecimalDigits DecimalExponent . Decimal . Decimal DecimalExponent DecimalDigits DecimalExponent Current DecimalFloat spec does allow floats like: writefln(.0); // 0 writefln(.10); // NonZeroDigit DecimalDigits but not: writefln(.01); // 0 DecimalDigits (Zero followed by digits) There's no rule in the spec for this, but it's lexed by the compiler. Underscores in exponents aren't handled correct, too. Change DecimalFloat to somthing similar: DecimalFloat: DecimalDigits2 . DecimalDigits2 DecimalDigits . DecimalDigits2 . DecimalDigits DecimalDigits2 DecimalDigits . DecimalDigits DecimalDigits2 . DecimalDigits DecimalExponent DecimalDigits2 DecimalDigits . DecimalDigits DecimalExponent . DecimalDigits2 . DecimalDigits2 DecimalDigits . DecimalDigits2 DecimalExponent . DecimalDigits2 DecimalDigits DecimalExponent DecimalDigits2 DecimalExponent DecimalDigits2 DecimalDigits DecimalExponent DecimalDigits2 0 NonZeroDigits DecimalExponentStart e E e+ E+ e- E- DecimalExponent DecimalExponentStart DecimalDigits2 DecimalExponentStart DecimalDigits2 DecimalDigits DecimalExponentStart DecimalDigits DecimalDigits2 Not sure if it's correct, but the above shouldn't allow: writefln(._); writefln(_.); but: writefln(.01); HexFloat: HexPrefix HexDigits . HexDigits HexExponent HexPrefix . HexDigits HexExponent HexPrefix HexDigits HexExponent Against the spec, this generates no errors by the compiler: writefln(0x.p1); // HexPrefix . HexExponent writefln(0x1.p1); // HexPrefix HexDigits . HexExponent The first rule should give an error, while the second should be added to the spec.

Now, i had a closer look at the grammar for strings, integers and floats. - I think escape sequence \<eof> should give an error instead of a string "\\". If no error is desired a string "\0" seems to match better. - Octals in escape sequences are lexed incorrect: spec says that \0_7 should be the same as \07 but it's lexed as \0 and _7. I belive the spec is wrong. - Same thing with hexadecimal escape sequences \x \u \U: spec says with underscores, but lexed without. - Hex strings can contain underscores according to spec, but lexed without. I rewrote the grammar a little -g- bit to fix all issues with these underscores: - US in hex escape sequences are not allowed, lexer fits. - No single . is allowed in HexFloat, lexer fails. - More than one 0 between . and nonzeros is allowed, lexer fits. - No single US are allowed, lexer fails sometimes: - DecimalFloats starting with US are disallowed, lexer fits. - ._1 is disallowed, lexer fits. - 1._1 is allowed, lexer fits. - 1._ is disallowed, lexer fails. - exponents with only US are not allowed, lexer failes on exp starting with US. Found nothing more, yet. Hope this is not too much. Comments? ##### Integer literals ##### IntegerLiteral: Integer Integer IntegerSuffix Integer: Decimal Binary Octal Hexadecimal Integer IntegerSuffix: L u U Lu LU uL UL Decimal: 0 NonZeroDigit - NonZeroDigit DecimalDigits + NonZeroDigit DecimalDigitsUS Binary: - 0b BinaryDigits - 0B BinaryDigits + BinPrefix BinaryDigitsNoSingleUS +BinPrefix: + 0b + 0B Octal: - 0 OctalDigits + 0 OctalDigitsUS Hexadecimal: - 0x HexDigits - 0X HexDigits + HexPrefix HexDigitsNoSingleUS NonZeroDigit: 1 2 3 4 5 6 7 8 9 DecimalDigits: DecimalDigit DecimalDigit DecimalDigits +DecimalDigitsUS: + DecimalDigitUS + DecimalDigitUS DecimalDigitsUS +DecimalDigitsNoSingleUS: + DecimalDigit + DecimalDigit DecimalDigitsUS + DecimalDigitsUS DecimalDigit +DecimalDigitsNoStartingUS: + DecimalDigit + DecimalDigit DecimalDigitsUS DecimalDigit: 0 NonZeroDigit - _ +DecimalDigitUS: + DecimalDigit + _ -BinaryDigits: - BinaryDigit - BinaryDigit BinaryDigits +BinaryDigitsUS: + BinaryDigitUS + BinaryDigitUS BinaryDigitsUS +BinaryDigitsNoSingleUS: + BinaryDigit + BinaryDigit BinaryDigitsUS + BinaryDigitsUS BinaryDigit BinaryDigit: 0 1 - _ +BinaryDigitUS: + BinaryDigit + _ OctalDigits: OctalDigit OctalDigit OctalDigits +OctalDigitsUS: + OctalDigitUS + OctalDigitUS OctalDigitsUS OctalDigit: 0 1 2 3 4 5 6 7 - _ +OctalDigitUS: + OctalDigit + _ HexDigits: HexDigit HexDigit HexDigits +HexDigitsUS: + HexDigitUS + HexDigitUS HexDigitsUS +HexDigitsNoSingleUS: + HexDigit + HexDigit HexDigitsUS + HexDigitsUS HexDigit HexDigit: DecimalDigit - a - b - c - d - e - f - A - B - C - D - E - F + HexLetter +HexDigitUS: + DecimalDigitUS + HexLetter +HexLetter: + a + b + c + d + e + f + A + B + C + D + E + F ##### Floating literals ##### FloatLiteral: Float Float FloatSuffix Float ImaginarySuffix Float FloatSuffix ImaginarySuffix Float: DecimalFloat HexFloat DecimalFloat: - DecimalDigits . - DecimalDigits . DecimalDigits - DecimalDigits . DecimalDigits DecimalExponent - . Decimal - . Decimal DecimalExponent - DecimalDigits DecimalExponent + DecimalDigitsNoStartingUS . + DecimalDigitsNoStartingUS . DecimalDigitsNoSingleUS + DecimalDigitsNoStartingUS . DecimalDigitsNoSingleUS DecimalExponent + . DecimalDigitsNoStartingUS + . DecimalDigitsNoStartingUS DecimalExponent + DecimalDigitsNoStartingUS DecimalExponent DecimalExponent: - e DecimalDigits - E DecimalDigits - e+ DecimalDigits - E+ DecimalDigits - e- DecimalDigits - E- DecimalDigits + DecimalExponentStart DecimalDigitsNoSingleUS +DecimalExponentStart: + e + E + e+ + E+ + e- + E- HexFloat: - HexPrefix HexDigits . HexDigits HexExponent - HexPrefix . HexDigits HexExponent - HexPrefix HexDigits HexExponent + HexPrefix HexDigitsNoSingleUS . HexDigitsNoSingleUS HexExponent + HexPrefix . HexDigitsNoSingleUS HexExponent + HexPrefix HexDigitsNoSingleUS . HexExponent + HexPrefix HexDigitsNoSingleUS HexExponent HexPrefix: 0x 0X HexExponent: - p DecimalDigits - P DecimalDigits - p+ DecimalDigits - P+ DecimalDigits - p- DecimalDigits - P- DecimalDigits + HexExponentStart DecimalDigitsNoSingleUS +HexExponentStart: + p + P + p+ + P+ + p- + P- FloatSuffix: f F L ImaginarySuffix: i

Commit pushed to https://github.com/D-Programming-Language/d-programming-language.org https://github.com/D-Programming-Language/d-programming-language.org/commit/4e60a4f1d62ff8acd0784c341dbfb203f4127494 fix Issue 949 - Wrong spec/compiler behaviour for Strings, Integers and Floats

Fixed spec.