Check that TCGETS is indeed the way to go

[szakharchenko]

Please check if TCGETS is appropriate here:

go-isatty/isatty_tcgets.go

Line 11 in 13e91bf

_, err := unix.IoctlGetTermios(int(fd), unix.TCGETS)

See https://git.musl-libc.org/cgit/musl/patch/?id=2de85a985654d2c944931267645d9a0686242dfe for details.

[dolmen]

Source: a patch in the musl libc.

HTML version: https://git.musl-libc.org/cgit/musl/commit/?id=2de85a985654d2c944931267645d9a0686242dfe

on most archs, the TCGETS ioctl command shares a value with SNDCTL_TMR_TIMEBASE, part of the OSS sound API which was apparently used with certain MIDI and timer devices. for file descriptors referring to such a device, TCGETS will not fail with ENOTTY as expected; it may produce a different error, or may succeed, and if it succeeds it changes the mode of the device. while it's unlikely that such devices are in use, this is in principle very harmful behavior for an operation which is supposed to do nothing but query whether the fd refers to a tty.

TIOCGWINSZ, used to query logical window size for a terminal, was chosen as an alternate ioctl to perform the isatty check. it does not share a value with any other ioctl commands, and it succeeds on any tty device.

Here is the current implementation of isatty in musl:

int isatty(int fd)
{
	struct winsize wsz;
	unsigned long r = syscall(SYS_ioctl, fd, TIOCGWINSZ, &wsz);
	if (r == 0) return 1;
	if (errno != EBADF) errno = ENOTTY;
	return 0;
}