Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zssh: re-run autoreconf #66397

Closed
wants to merge 1 commit into from

Conversation

mitchblank
Copy link
Contributor

@mitchblank mitchblank commented Dec 7, 2020

The ./configure file included in the 1.5c tarball is too old to work with Xcode 12.

@mitchblank mitchblank marked this pull request as draft December 7, 2020 20:19
@mitchblank
Copy link
Contributor Author

OK my new make tests failed:

==> Testing zssh
/usr/bin/sandbox-exec -f /private/tmp/homebrew20201207-26825-sxp8ad.sb ruby -W0 -I $LOAD_PATH -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/zssh.rb --verbose
==> /usr/local/Cellar/zssh/1.5c/bin/zssh --version
tcgetattr: Inappropriate ioctl for device
Error: zssh: failed

I guess they don't like running without a pty (even to print --version!) or maybe some sandboxing thing. It works when I run brew test manually, of course.

Formula/zssh.rb Outdated
Comment on lines 42 to 45
test do
system "#{bin}/zssh", "--version"
system "#{bin}/ztelnet", "--version"
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a test that exercises the some of the functionality of the app. Version checks or usage checks (foo --version or foo --help) are not sufficient, as explained in the formula cookbook.

In most cases, a good test would involve running a simple test case: run #{bin}/foo input.txt.

  • Then you can check that the output is as expected (with assert_equal or assert_match on the output of shell_output)
  • You can also check that an output file was created, if that is expected: assert_predicate testpath/"output.txt", :exist?

Some advice for specific cases:

  • If the formula is a library, compile and run some simple code that links against it. It could be taken from upstream's documentation / source examples.
  • If the formula is for a GUI program, try to find some function that runs as command-line only, like a format conversion, reading or displaying a config file, etc.
  • If the software cannot function without credentials, a test could be to try to connect with invalid credentials (or without credentials) and confirm that it fails as expected.
  • Same if the software requires a virtual machine, docker instance, etc. to be running.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am aware that just tests running --version are not great, although they are very common in Homebrew. At least they are verify that the compile generated a runnable binary.

As I've said elsewhere, I am just trying to help out fixing bottling issues on Big Sur. I've never used this particular package and have no idea how to test it more in-depth; I am just trying to make it build again.

Anyway I'll remove this test and we'll just leave this formula test-less as it was before. It was just an attempt at a slight improvement while I was fixing something else.

@carlocab
Copy link
Member

carlocab commented Dec 8, 2020

Try passing your test through shell_output or pipe_output; that might persuade zssh not to look for a pty.

Given that the package is designed for "interactive file transfers over SSH", not sure how much more substantial a test one can write. Of course, maybe I'm just not familiar enough with Homebrew's test infrastructure.

Anyway, this might be helpful: https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula

@gromgit
Copy link
Member

gromgit commented Dec 8, 2020

An MVT (Minimum Viable Test) might be to connect to a free port on localhost and match the output against an expected error message.

@mitchblank
Copy link
Contributor Author

Try passing your test through shell_output or pipe_output; that might persuade zssh not to look for a pty.

Doubt that works. It seems that --version breaks any time zssh doesn't have stdin attached to a terminal. i.e.

$ zssh --version < /dev/null
tcgetattr: Operation not supported by device

so unless there is a way to make the test infra allocate a pty, it probably isn't work persuing further.

Anyway, like I said I'm just going to remove this attempt to add a basic test to this PR and just concentrate on the changes needed to make Big Sur build OK. If there's anything this experience has taught me is that adding a testcase in a PRjust invites nitpicking it to death. I'm just going to concentrate purely on the Big Sur stuff and leave testing be.

@mitchblank mitchblank marked this pull request as ready for review December 8, 2020 15:11
@SMillerDev
Copy link
Member

adding a testcase in a PRjust invites nitpicking it to death.

You might call it nitpicking, but adding a real test that works helps make the next OS update less of a pain.

@BrewTestBot
Copy link
Member

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@BrewTestBot BrewTestBot added the stale No recent activity label Jan 1, 2021
The ./configure file included in the 1.5c tarball is too old to work
with Xcode 12.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale No recent activity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants