Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throw error on empty file #334

Merged
merged 10 commits into from
Dec 18, 2024
Merged

Throw error on empty file #334

merged 10 commits into from
Dec 18, 2024

Conversation

AsafMah
Copy link
Collaborator

@AsafMah AsafMah commented Dec 4, 2024

(This change is for the current 6.0.2 version)

(This change is for the current 6.0.2 version)
@AsafMah AsafMah requested a review from ohadbitt December 4, 2024 11:30
@AsafMah AsafMah changed the base branch from master to 6.x.x December 4, 2024 11:30
Copy link

github-actions bot commented Dec 4, 2024

Unit Test Results

    1 files  ±0    16 suites  ±0   6m 49s ⏱️ +2s
275 tests  - 1  267 ✔️  - 1  8 💤 ±0  0 ±0 
281 runs   - 1  273 ✔️  - 1  8 💤 ±0  0 ±0 

Results for commit ce5beda. ± Comparison against base commit 5913838.

♻️ This comment has been updated with latest results.

@@ -77,7 +77,11 @@ export class FileDescriptor extends AbstractDescriptor implements FileDescriptor
private async _calculateSize(modifier: number = 1): Promise<void> {
if (this.size == null || this.size <= 0) {
const asyncStat = promisify(fs.stat);
this.size = (await asyncStat(this.file as string)).size * modifier;
const fileSize = await asyncStat(this.file as string);
if (fileSize.size <= 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use the same comparison to all

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one time you have <= 0 and above you have === 0

}

if (uploadResponse?.errorCode != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesnt this stop retries ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the only way we signal we want a retry is to throw an error. All errors retry, we don't support permanent errors it seems

Copy link
Contributor

@ohadbitt ohadbitt Dec 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right. There is no such concept in our non-.net SDKs - maybe a little bit in Java ..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a big thing - when does it happen that response doesnt throw but has an error ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should throw on any 400 or 500 code

if (fileSize.size <= 0) {
throw Error("Empty file.");
}
this.size = fileSize.size * modifier;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont remember we had this modifier thing - did someone add this here and not to browser code ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this was the previous version of the method:

    private async _calculateSize(modifier: number = 1): Promise<void> {
        if (this.size == null || this.size <= 0) {
            const asyncStat = promisify(fs.stat);
            this.size = (await asyncStat(this.file as string)).size * modifier;
        }
    }
    ```
    
    According to git blame, you added this last year. Not sure why it's not in browser

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you want to add this to browser code ? should i?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOne

@@ -50,4 +56,15 @@ export class FileDescriptor extends AbstractDescriptor implements FileDescriptor
getCompressionSuffix() {
return this.compressionType ? `.${this.compressionType}` : ".gz";
}

private _calculateSize(modifier: number = 1): void {
if (this.size == null || this.size <= 0) {
Copy link
Contributor

@ohadbitt ohadbitt Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i dont think this works - i dont think size here is ever null .. as we take the file size as the size by default

@@ -50,4 +56,14 @@ export class FileDescriptor extends AbstractDescriptor implements FileDescriptor
getCompressionSuffix() {
return this.compressionType ? `.${this.compressionType}` : ".gz";
}

private _calculateSize(modifier: number = 1): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a fileDescriptor thing to do ? its relevant to all other descriptors ..
Its easier when we align all this king of logic to our other SDKs - this way we can make sure we dont have loose ends

@AsafMah AsafMah merged commit aef5656 into 6.x.x Dec 18, 2024
3 checks passed
AsafMah added a commit that referenced this pull request Jan 23, 2025
Co-authored-by: Asaf Mahlev <asafmahlev@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ariel Yehezkely <aryehezk@microsoft.com>
Co-authored-by: alonadam <96700455+alonadam@users.noreply.github.com>
Fix streaming ingestion (#341)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants