Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How to serialise/parse large xml file with KorIO #1167

Closed
multiport opened this issue Dec 27, 2022 · 1 comment
Closed

Question: How to serialise/parse large xml file with KorIO #1167

multiport opened this issue Dec 27, 2022 · 1 comment

Comments

@multiport
Copy link

multiport commented Dec 27, 2022

Hello,

I'm using KMM with Okio, and i'm triying to serialise/parse a large xml file, I see on https://docs.korge.org/korio/serialization/ how to parse a String but since my file is too large, im triying to parse the stream :

  val fileSystem = FileSystem.SYSTEM_TEMPORARY_DIRECTORY
  val path = fileSystem / "myfile.xml"
  // open a source stream
  val sourceFileRead= fileSystem2.source(path)
  // wrap stream with a buffer
  val fileopened = sourceFileRead.buffer()
  // read until end of file
  var line: String?
  var numLigne :Int =0
  while (fileopened.readUtf8Line().also { line = it } != null) {
      println(line)
      var xmlLine = line?.reader(line!!,numLigne)
      Xml.Stream.parse(xmlLine!!)  // <-- Not gonna work but i dont know how to do ?
  }

Thanks for your help if you have a solution

@soywiz
Copy link
Member

soywiz commented Dec 27, 2022

It is not posible right now. Xml streaming uses StrReader that was designed to use a String in memory. It was like that to be as fast as possible (no virtual methods, no suspension points, all in-memory).
The thing is that K/N have evolved a lot and now maybe the performance is much better, so might make sense now to create a String Reader that support streams without having the whole file in memory and make Xml to use it.
I have created a PR with a baby step towards it

@korlibs korlibs locked and limited conversation to collaborators Dec 27, 2022
@soywiz soywiz converted this issue into discussion #1172 Dec 27, 2022
@github-project-automation github-project-automation bot moved this from Backlog to Done in Korlibs Dec 27, 2022

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants