proposal: encoding/xml: Return a SyntaxError for unmatched start elements at EOF

[rsto]

I propose to make the xml.Decoder's Token routine fail with a syntax error, when the XML input contains unmatched start elements and the end of the stream.

Background

The xml.Decoder's Token routine returns successfully for input even if the XML content includes not properly closed start elements. For example, the following input

<root><bar></bar>

causes Token to return nil, io.EOF at the end of the stream, despite the unmatched "root" start element.

In my understanding Token should reject syntactically incorrect XML like this with an error.

This is also indicated by the following statement of the godoc of Token (emphasis mine)

Token guarantees that the StartElement and EndElement tokens it returns are properly 
nested and *matched*: if Token encounters an unexpected end element, it will return an 
error.

Proposal

I propose that a xml.Decoder's Token routine should return xml.SyntaxError for dangling start elements at the end of the stream. To do so, I have a CL ready for submission. Should this go through the proposed proposal process? Should this wait until Go 1.6?

Working example: http://play.golang.org/p/Fs6NPvDLu0
Verified on Go version: 1.4.2 and 1.5 development version (go version devel +751eef8).

[edited for clarity]
[edited to serve as a proposal for the proposal process]

[rsto]

@nigeltao Who could I ping for a comment? I can send a CL over for discussion but would prefer to first get feedback if this should even be fixed (and if so, what's the release target).

[nigeltao]

The Decoder.Token doc comment, which you linked to, does make your proposal sound reasonable. CC'ing @rsc.

As for the proposal process, TBH I'm as new to it as you are, but submitting a proposal seems fine.

In any case, this should wait until 1.6.

[rsc]

I think we should try this. However, it changes the behavior of existing programs, so we will need to call this out very explicitly in the release notes.

[rsto]

Great! I plan to send a CL over the weekend. Please let me know if you would like to see a design doc first.

[rsto]

The CL lives at https://go-review.googlesource.com/#/c/14315/

[gopherbot]

CL https://golang.org/cl/14315 mentions this issue.