You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Translation phases 2 and 3 assume that lines are terminated by “new-line characters”. However, the current specification of phase 1 does not guarantee that to be true. In particular, for a UTF-8 file the verbatim sequence of source file characters forms the input for phase 2, even on systems where the line terminator is a carriage return. The non-UTF-8 specification is also defective in that it speaks of “introducing” new-line characters, even for encodings like Latin-1 where new-lines might already be present and no “introduction” is needed or appropriate.
Proposed change:
If an input file is determined to be a UTF-8 file, then it shall be a well-formed UTF-8 code unit sequence and it is decoded to produce a sequence of UCS scalar values that constitutes the sequence of elements of the translation character set, representing each line-termination character or character sequence as a new-line character.
For any other kind of input file supported by the implementation, characters are mapped, in an implementation-defined manner, to a sequence of translation character set elements (5.3) (introducing new-line characters for representing end-of-line indicators as new-line characters).
The text was updated successfully, but these errors were encountered:
wg21bot
changed the title
US 5.2p1 [lex.phases]
US 5.2p1 [lex.phases] new-lines after phase 1
Oct 30, 2022
Translation phases 2 and 3 assume that lines are terminated by “new-line characters”. However, the current specification of phase 1 does not guarantee that to be true. In particular, for a UTF-8 file the verbatim sequence of source file characters forms the input for phase 2, even on systems where the line terminator is a carriage return. The non-UTF-8 specification is also defective in that it speaks of “introducing” new-line characters, even for encodings like Latin-1 where new-lines might already be present and no “introduction” is needed or appropriate.
Proposed change:
If an input file is determined to be a UTF-8 file, then it shall be a well-formed UTF-8 code unit sequence and it is decoded to produce a sequence of UCS scalar values that constitutes the sequence of elements of the translation character set, representing each line-termination character or character sequence as a new-line character.
For any other kind of input file supported by the implementation, characters are mapped, in an implementation-defined manner, to a sequence of translation character set elements (5.3) (
introducing new-line characters forrepresenting end-of-line indicators as new-line characters).The text was updated successfully, but these errors were encountered: