MultipleFileUploader JAVA SDA bytes reported must be non-negative Exception

[studioam]

I also have an odd exception when using a MultipleFileUploader.uploadDir ; the dir is 158Gb sized and includes 30Gb files. It randomly appears with the listener to monitor multiplefilesupload. 5-6 times during total upload process, and some files upload fail. The result is an uncomplete directory in the bucket.

  gen 30, 2017 11:16:39 AM com.amazonaws.event.ProgressListenerChain progressChanged
  AVVERTENZA: Couldn't update progress listener
    java.lang.IllegalArgumentException: bytes reported must be non-negative
at com.amazonaws.event.ProgressEvent.<init>(ProgressEvent.java:68)
at com.amazonaws.event.ProgressEvent.<init>(ProgressEvent.java:43)
at com.amazonaws.services.s3.model.ProgressEvent.<init>(ProgressEvent.java:30)
at com.amazonaws.services.s3.model.LegacyS3ProgressListener.adaptToLegacyEvent(LegacyS3ProgressListener.java:53)
at com.amazonaws.services.s3.model.LegacyS3ProgressListener.progressChanged(LegacyS3ProgressListener.java:47)
at com.amazonaws.event.ProgressListenerChain.progressChanged(ProgressListenerChain.java:98)
at com.amazonaws.services.s3.transfer.MultipleFileTransferProgressUpdatingListener.progressChanged(MultipleFileTransferProgressUpdatingListener.java:41)
at com.amazonaws.event.ProgressListenerChain.progressChanged(ProgressListenerChain.java:98)
at com.amazonaws.event.SDKProgressPublisher.quietlyCallListener(SDKProgressPublisher.java:86)
at com.amazonaws.event.SDKProgressPublisher.deliverEvent(SDKProgressPublisher.java:68)
at com.amazonaws.event.SDKProgressPublisher.publishResetEvent(SDKProgressPublisher.java:213)
at com.amazonaws.event.SDKProgressPublisher.publishRequestReset(SDKProgressPublisher.java:176)
at com.amazonaws.event.RequestProgressInputStream.onReset(RequestProgressInputStream.java:33)
at com.amazonaws.event.ProgressInputStream.reset(ProgressInputStream.java:169)
at com.amazonaws.internal.SdkFilterInputStream.reset(SdkFilterInputStream.java:102)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.resetRequestInputStream(AmazonHttpClient.java:1303)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1129)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
 at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4185)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4132)
at com.amazonaws.services.s3.AmazonS3Client.doUploadPart(AmazonS3Client.java:3172)
at com.amazonaws.services.s3.AmazonS3Client.uploadPart(AmazonS3Client.java:3157)
at com.amazonaws.services.s3.transfer.internal.UploadPartCallable.call(UploadPartCallable.java:33)
at com.amazonaws.services.s3.transfer.internal.UploadPartCallable.call(UploadPartCallable.java:23)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

The listener in the upload tool is set as follows:

 public void progressChanged(ProgressEvent progressEvent) {
        
    if (upload == null) return;
    
    jProgressBar1.setValue((int)upload.getProgress().getPercentTransferred());    
    
            
    if (progressEvent.getEventType()== ProgressEventType.TRANSFER_COMPLETED_EVENT) {
        
                           
           if (upload.isDone()){
             tx.shutdownNow();
       
        
        
        
        
        
        JOptionPane.showMessageDialog(frame,"Upload DCP Completato.");
            
        }
           
           
            	
           
       
    }
    if (progressEvent.getEventType() == ProgressEventType.TRANSFER_FAILED_EVENT){
        try {
            AmazonClientException e = upload.waitForException();
            JOptionPane.showMessageDialog(frame,
                    "Unable to Download file from Amazon S3: " + e.getMessage(),
                    "Error Downloading File", JOptionPane.ERROR_MESSAGE);
        } catch (InterruptedException e) {}
    }
    }`

and in the main prog:

   TransferManager  tx = new TransferManager(s3);
    MultipleFileUpload  upload = tx.uploadDirectory(bucketName, fileChooser.getSelectedFile().getName(), UploadFolder ,true);
    upload.addProgressListener(progressListener);

[dagnir]

Hi @studioam, thanks for reporting this issue. Looks like a case of integer overflow to me: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/LegacyS3ProgressListener.java#L53. Will verify and get a fix out if so.

[dagnir]

Hi @studioam, I've pushed a fix for this internally and it should released next week (probably Monday 2/6). Also, please ensure that you're using com.amazonaws.event.ProgressListener in your code and not com.amazonaws.services.s3.model.ProgressListener which is deprecated.

[studioam]

Hi, thanks for the jint, i was using the deprecated one , com.amazonaws.services.s3.model.ProgressListener so maybe the fix is not necessary. Will try with com.amazonaws.event.ProgressListener and report

[dagnir]

It's still a bug in the code so it's good that it was reported :). It's likely to trip up future users as well so we also added better documentation to avoid using the deprecated classes.

[studioam]

You're right, even with non-deprecated class the issue still stands.

[dagnir]

Hi @studioam, the fix was pushed as part of 1.11.87