Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize I/O code using Okio #4366

Merged
merged 6 commits into from
Apr 10, 2024
Merged

Conversation

cbeyls
Copy link
Contributor

@cbeyls cbeyls commented Apr 9, 2024

This pull request takes advantage of the Okio library to simplify, fix or improve performance of some I/O related code in Tusky.

  • Return early or throw FileNotFoundException early in case contentResolver.openInputStream() returns null instead of throwing NullPointerException later. Change the signature of Closeable.closeQuietly() to only accept a non-null Closeable.
  • Reimplement Uri.copyToFile() using Okio. This takes advantage of the built-in high-performance buffers of the library so a buffer doesn't need to be allocated or managed manually. The new implementation also makes sure that the input and output streams are always closed, as the original code could in some cases return without properly closing a stream.
  • Reimplement ProgressRequestBody as Uri.asRequestBody() (adding to the existing extension functions available in the Okio library to create a RequestBody). The new implementation uses Okio's Buffer instead of a manually managed byte array, which allows to avoid copying bytes from one buffer to the next. The max number of bytes read at once was increased from 2K to 8K to improve performance. Avoid division by zero in case contentLength is 0. Finally, this implementation now takes a Uri as input instead of an InputStream, because a RequestBody must be replayable in case Okio retries the request, and an InputStream can only be used once.

cbeyls added 6 commits April 8, 2024 14:20
…e buffer segments and properly close input and output in all cases
…rs, increasing the chunk size and correctly handle Okio retries by allowing the source stream to be reopened multiple times.
…ntentProvider is unavailable during Uri RequestBody read
Copy link
Contributor

@Goooler Goooler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we can replace java.nio.ByteBuffer and java.nio.charset.Charset.

messageDigest.update(ByteBuffer.allocate(4).putInt(backgroundColor.hashCode()).array())

private val ID_BYTES = ID.toByteArray(Charset.forName("UTF-8"))

fun onProgressUpdate(percentage: Int)
}

fun Uri.asRequestBody(contentResolver: ContentResolver, contentType: MediaType? = null, contentLength: Long = -1L, uploadListener: UploadCallback? = null): RequestBody {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fun Uri.asRequestBody(contentResolver: ContentResolver, contentType: MediaType? = null, contentLength: Long = -1L, uploadListener: UploadCallback? = null): RequestBody {
fun Uri.asRequestBody(
contentResolver: ContentResolver,
contentType: MediaType? = null,
contentLength: Long = -1L,
uploadListener: UploadCallback? = null
): RequestBody {

@connyduck connyduck merged commit 65af269 into tuskyapp:develop Apr 10, 2024
3 checks passed
@cbeyls
Copy link
Contributor Author

cbeyls commented Apr 10, 2024

And we can replace java.nio.ByteBuffer and java.nio.charset.Charset.

messageDigest.update(ByteBuffer.allocate(4).putInt(backgroundColor.hashCode()).array())

private val ID_BYTES = ID.toByteArray(Charset.forName("UTF-8"))

Sorry I didn't have time to apply your suggested changes today.

Do you mean replacing ByteBuffer with Okio Buffer and toByteArray() with Okio Buffer or ByteString? I'm not sure it would be worth the change for these cases.

@Goooler
Copy link
Contributor

Goooler commented Apr 11, 2024

Yeah, just wanna replace all nio usages.

@cbeyls cbeyls deleted the refactor/optimize_io branch April 17, 2024 12:33
nikclayton added a commit to pachli/pachli-android that referenced this pull request Jun 20, 2024
`ImageDownsizer.downsizeImage()`:
- Remove the return value, it was ignored
- Throw `FileNotFoundException` when `openInputStream` returns null

`ImageDownsizer.getImageOrientation()`:
- Throw `FileNotFoundException` when `openInputStream` returns null

`MediaUploader.prepareMedia()`:
- Copy URI contents using Okio buffers / source / sink

`UriExtensions`:
- Rename from `IOUtils`
- Implement `Uri.copyToFile()` using Okio buffers / source / sink
- Replace `ProgressRequestBody()` with `Uri.asRequestBody()` using
  Okio buffers / source / sink

`DraftHelper.copyToFolder()`
- Use Okio buffers / source / sink

`CompositeWithOpaqueBackground`
- Use constants `SIZE_BYTES` and `CHARSET` instead of magic values
- Use `Objects.hash` when hashing multiple objects

Based on work by Christophe Beyls in
- tuskyapp/Tusky#4366
- tuskyapp/Tusky#4372
nikclayton added a commit to pachli/pachli-android that referenced this pull request Jun 20, 2024
`ImageDownsizer.downsizeImage()`:
- Remove the return value, it was ignored
- Throw `FileNotFoundException` when `openInputStream` returns null

`ImageDownsizer.getImageOrientation()`:
- Throw `FileNotFoundException` when `openInputStream` returns null

`MediaUploader.prepareMedia()`:
- Copy URI contents using Okio buffers / source / sink

`UriExtensions`:
- Rename from `IOUtils`
- Implement `Uri.copyToFile()` using Okio buffers / source / sink
- Replace `ProgressRequestBody()` with `Uri.asRequestBody()` using Okio
buffers / source / sink

`DraftHelper.copyToFolder()`
- Use Okio buffers / source / sink

`CompositeWithOpaqueBackground`
- Use constants `SIZE_BYTES` and `CHARSET` instead of magic values
- Use `Objects.hash` when hashing multiple objects

Based on work by Christophe Beyls in
- tuskyapp/Tusky#4366
- tuskyapp/Tusky#4372
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants