Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(aws-glue-alpha): S3 path specified in --spark-event-logs-path needs to end with slash #29356

Closed
moomindani opened this issue Mar 4, 2024 · 2 comments · Fixed by #29357 · May be fixed by NOUIY/aws-solutions-constructs#98, NOUIY/aws-solutions-constructs#99 or NOUIY/aws-solutions-constructs#101
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. effort/medium Medium work item – several days of effort p2

Comments

@moomindani
Copy link
Contributor

Describe the bug

When only bucket is provided for spark event log path in job, the s3 path does not end with slash.
It is accepted in job API, but it does not work on actual Spark UI.

Expected Behavior

S3 path specified in --spark-event-logs-path ends with slash

Current Behavior

S3 path specified in --spark-event-logs-path does not end with slash in case only bucket is provided but prefix is not provided

Reproduction Steps

      sparkUI: {
        enabled: true,
      },

Possible Solution

Add trailing slash when it does not end with slash

Additional Information/Context

No response

CDK CLI Version

2.131.0

Framework Version

No response

Node.js Version

20.11.1

OS

macOS Ventura 13.6.4

Language

TypeScript

Language Version

No response

Other information

No response

@moomindani moomindani added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 4, 2024
@github-actions github-actions bot added the @aws-cdk/aws-glue Related to AWS Glue label Mar 4, 2024
@tim-finnigan tim-finnigan self-assigned this Mar 6, 2024
@tim-finnigan tim-finnigan added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Mar 6, 2024
@tim-finnigan
Copy link

Thanks for reaching out and creating the PR - it looks like there is already some feedback that is awaiting a response.

@tim-finnigan tim-finnigan added p2 effort/medium Medium work item – several days of effort labels Mar 6, 2024
@tim-finnigan tim-finnigan removed the needs-triage This issue or PR still needs to be triaged. label Mar 6, 2024
@tim-finnigan tim-finnigan removed their assignment Mar 6, 2024
@mergify mergify bot closed this as completed in #29357 Mar 19, 2024
mergify bot pushed a commit that referenced this issue Mar 19, 2024
…with slash (#29357)

### Issue # (if applicable)

Closes #29356.

### Reason for this change

Currently S3 path specified in `--spark-event-logs-path` does not end with slash in case only bucket is provided but prefix is not provided. This parameter causes errors when viewing the event log through Spark UI / Spark history server.

### Description of changes

Add trailing slash when it does not end with slash.

### Description of how you validated changes

Completed unit test and integ test.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

ahammond pushed a commit to ahammond/aws-cdk that referenced this issue Mar 26, 2024
…with slash (aws#29357)

### Issue # (if applicable)

Closes aws#29356.

### Reason for this change

Currently S3 path specified in `--spark-event-logs-path` does not end with slash in case only bucket is provided but prefix is not provided. This parameter causes errors when viewing the event log through Spark UI / Spark history server.

### Description of changes

Add trailing slash when it does not end with slash.

### Description of how you validated changes

Completed unit test and integ test.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. effort/medium Medium work item – several days of effort p2
Projects
None yet
2 participants