Include script to write to GEXF. (#103)

[greebie]

---

This script creates a GEXF output for link structures similar to WriteGDF.

---

GitHub issue(s):

#103

What does this Pull Request do?

Adds WriteGEXF( RDD[((String, String, String), Int)] , path) to output a link structure RDD to a gexf file.

How should this be tested?

The following test script can be used to produce the file:

import io.archivesunleashed.spark.matchbox.{ExtractDomain, ExtractLinks, RecordLoader, WriteGEXF}
import io.archivesunleashed.spark.rdd.RecordRDD._

val links = RecordLoader.loadArchives("src/test/resources/arc/example.arc.gz", sc)
  .keepValidPages()
  .map(r => (r.getCrawlDate, ExtractLinks(r.getUrl, r.getContentString)))
  .flatMap(r => r._2.map(f => (r._1, ExtractDomain(f._1).replaceAll("^\\s*www\\.", ""), ExtractDomain(f._2).replaceAll("^\\s*www\\.", ""))))
  .filter(r => r._2 != "" && r._3 != "")
  .countItems()
  .filter(r => r._2 > 5)

WriteGEXF(links, "linksTest.gexf")

And then linksTest.gexf should be opened in a network file reader such as Sigma.js or Gephi.

Additional Notes:

Unittest will accompany the script over December (as I make another unit test push)

Interested parties

Tag (@ianmilligan1)

Thanks in advance for your help with the Archives Unleashed Toolkit!

[codecov]

Codecov Report

Merging #137 into master will decrease coverage by 1.91%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #137      +/-   ##
==========================================
- Coverage   65.66%   63.74%   -1.92%     
==========================================
  Files          36       37       +1     
  Lines         731      753      +22     
  Branches      142      143       +1     
==========================================
  Hits          480      480              
- Misses        201      223      +22     
  Partials       50       50

Impacted Files	Coverage Δ
...o/archivesunleashed/spark/matchbox/WriteGEXF.scala	0% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3eb093a...a5724e7. Read the comment docs.

[greebie]

Tested with example.arc.gz, and the two UFT files from aut-resources (one .arc, one .warc). Both files work swimmingly in Gephi.

[ianmilligan1]

Excellent! I'll test this afternoon when I have a cycle.

[ianmilligan1]

OK. Tested in Gephi and it looks good:

And compared with the other export formats and it all lines up!

[ianmilligan1]

As noted in the ticket, tested and it all works well. Fantastic work, Ryan!

[greebie]

Great!