Add binary extraction DataFrames to PySpark.

[ruebot]

GitHub issue(s):

• DataFrame discussion: open thread #190
• Adding Apache Zeppelin to docker and documentation #259
• PDF binary object extraction #302
• Spreadsheet binary object extraction #303
• Doc binary object extraction #304
• Powerpoint binary object extraction #305
• Video binary object extraction #306
• Audio binary object extraction #307

What does this Pull Request do?

Add binary extraction DataFrames to PySpark.

How should this be tested?

• TravisCI
• Fire up a Jupyter Notebook:

$ PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS=notebook ~/bin/spark-2.4.3-bin-hadoop2.7/bin/pyspark --jars /home/nruest/Projects/au/aut/target/aut-0.17.1-SNAPSHOT-fatjar.jar --driver-class-path /home/nruest/Projects/au/aut/target/aut-0.17.1-SNAPSHOT-fatjar.jar --py-files /home/nruest/Projects/au/aut/target/aut.zip

Then do this for each of the additions, and make sure it works:

from aut import *

archive = WebArchive(sc, sqlContext, "/home/nruest/Projects/au/aut/src/test/resources/warc/")

df = archive.spreadsheets()
df.printSchema()

Additional Notes:

Let's see what happens with the test coverage. I assume I'm going to have to add some things to src/test/scala/io/archivesunleashed/df/DataFrameLoaderTest.scala

[codecov]

Codecov Report

Merging #350 into master will decrease coverage by 0.76%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #350      +/-   ##
==========================================
- Coverage   75.52%   74.76%   -0.77%     
==========================================
  Files          39       39              
  Lines        1373     1387      +14     
  Branches      265      265              
==========================================
  Hits         1037     1037              
- Misses        220      234      +14     
  Partials      116      116

[ruebot]

Here's an even better test: https://github.com/archivesunleashed/aut/wiki/Using-AUT-with-PySpark

(Just swap out 0.18.0 with the path to aut-0.17.1-SNAPSHOT-fatjar.jar and aut.zip)

[lintool]

Comments about naming df's if it's not too late...

[ianmilligan1]

Ran through the PySpark documentation with the Jupyter command in this PR. All worked perfectly! 🎉