Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bind spark listening and server to localhost in tests #296

Closed
StrongestNumber9 opened this issue Aug 29, 2024 · 2 comments · Fixed by #299
Closed

Bind spark listening and server to localhost in tests #296

StrongestNumber9 opened this issue Aug 29, 2024 · 2 comments · Fixed by #299
Assignees
@eemhu
Labels
enhancement New feature or request

Comments

@StrongestNumber9
Copy link
Contributor

Description

Bind spark listening and server to localhost in tests.

Use case or motivation behind the feature request

Frewall rules can cause tests to timeout in certain environments if it decides to access the spark server via public ip and firewall drops such traffic

Related issues

Additional context

https://spark.apache.org/docs/latest/configuration.html#networking
@StrongestNumber9 StrongestNumber9 added the enhancement New feature or request label Aug 29, 2024
@StrongestNumber9
Copy link
Contributor Author

Maybe lowering spark.network.timeout to something like 5 seconds is useful as something has gone catastrophically wrong if we stall for even 5 seconds with networking when testing locally

@eemhu
Copy link
Contributor

eemhu commented Aug 30, 2024

internal PR submitted

StrongestNumber9 added a commit to StrongestNumber9/pth_10 that referenced this issue Sep 2, 2024
@StrongestNumber9
fix GH#292 and teragrep#296: add config items to SparkSession and fix…
013dcfa 
… old version

Ref internal pr-631
@StrongestNumber9 StrongestNumber9 mentioned this issue Sep 2, 2024
Merged
kortemik pushed a commit that referenced this issue Sep 2, 2024
@StrongestNumber9
fix GH#292 and #296: add config items to SparkSession and fix old ver…
ae2da24 
…sion (#299)

Ref internal pr-631
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eemhu eemhu

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix GH#292 and #296: add config items to SparkSession and fix old ver… StrongestNumber9/pth_10
2 participants
@StrongestNumber9 @eemhu