Tutorial #101

sameeraxiomine · 2015-09-30T13:43:54Z

Created the following set of files

tutorial/README.md
SparkRedshiftTutorial.scala
images/loadreadstep.gif
images/loadunloadstep.gif
images/savetoredshift.gif

First Commiting Readme.md, source code and images

Update tutorial/README.md

Update image path

Update images

Fixing the tutorial links

Update Links

Link to the entire program

Fix path

Made more fixes to verbiage

Updates

codecov-io · 2015-09-30T13:48:56Z

Current coverage is `87.16%`

Merging #101 into master will decrease coverage by -7.65% as of df65fde

@@            master    #101   diff @@
======================================
  Files           11      11       
  Stmts          444     444       
  Branches       109     109       
  Methods          0       0       
======================================
- Hit            421     387    -34
  Partial          0       0       
- Missed          23      57    +34

Review entire Coverage Diff as of df65fde

Powered by Codecov. Updated on successful CI builds.

JoshRosen · 2015-09-30T17:04:42Z

tutorial/SparkRedshiftTutorial.scala

+ * 
+ */
+object SparkRedshiftTutorial {
+  /*


The spacing and indentation seems off in this file. Is it indented using a mixture of spaces and tabs? Please re-indent using only spaces.

I have fixed this.

JoshRosen · 2015-09-30T17:18:31Z

Not to nitpick, but I think PNG might be a better file format for images due to its better quality / compression ratios.

JoshRosen · 2015-09-30T17:20:23Z

tutorial/README.md

+We are ready to interact with Redshift using the spark-redshift library. The skeleton of the program we will be using is shown in Listing 1. The entire `SparkRedshiftTutorial.scala` program can be accessed from [here](SparkRedshiftTutorial.scala). You can also use the Spark REPL to run the lines listed in the program below. 
+
+```scala
+package com.databricks.spark.redshift.tutorial


If the full tutorial source is available as a .scala file, do you think we can cut down on some of the skeleton / harness here to make the prose read a bit more smoothly?

Agreed. Cut down the comments as they are included in the .scala program.

Also changed as gif to png

JoshRosen · 2015-09-30T17:26:06Z

tutorial/README.md

+
+Figure 1 : UNLOAD action
+
+First the Spark Driver communicates with the Redshift Leader node to obtain the schema of the table (or query) requested. The attribute `override lazy val schema: StructType` in the class `com.databricks.spark.redshift.RedshiftRelation` will obtain the schema on demand by invoking the method `resolveTable` of the class `com.databricks.spark.redshift.JDBCWrapper`. The `JDBCWrapper` class is responsible for fetching the schema from the Redshift Leader.   


First, comma.

JoshRosen · 2015-09-30T17:56:55Z

I took one editing pass, but may have additional feedback. I would try reading the current draft aloud to find odd phrasing, typos, and misspellings, then take an editing pass to fix the mechanical issues.

Update based on comments received

Update the tutorial contents

Updates

Code fix

sameeraxiomine · 2015-10-01T00:49:00Z

I updated the tutorial and source code based on your comments. I made some more of my own as I did a full pass through it.

JoshRosen · 2015-10-15T19:02:12Z

I have some additional comments that I'd like to address, but I'm going to take care of them myself by submitting a followup PR. Therefore, I'm going to merge this now. Thanks!

This patch is a follow-up to #101 and makes many minor edits in the tutorial text. /cc sameeraxiomine Author: Josh Rosen <joshrosen@databricks.com> Closes #106 from JoshRosen/tutorial-edits.

Upgrade to spark v3.2.0

sameeraxiomine added 11 commits September 29, 2015 20:30

Spark-Redshift Tutorial

ef21821

First Commiting Readme.md, source code and images

Revert original README.md

3d8d1e0

Tutorial

2010c57

Update tutorial/README.md

Tutorial

7ec73c9

Update image path

Tutorial

77bfe59

Update images

Tutorial

ffc21ac

Fixing the tutorial links

Tutorial

0e3963f

Update Links

Tutorial

3974b7d

Link to the entire program

Tutorial

efa1156

Fix path

Tutorial

f7cd5c4

Made more fixes to verbiage

Tutorial

5a15393

Updates

JoshRosen reviewed Sep 30, 2015
View reviewed changes

sameeraxiomine added 4 commits September 30, 2015 20:07

Tutorial

40d1dbd

Update based on comments received

Tutorial

f0f876b

Update the tutorial contents

Tutorial

70eaf56

Updates

Tutorial

eae938d

Code fix

JoshRosen added the documentation label Oct 1, 2015

JoshRosen closed this in c72dc89 Oct 15, 2015

JoshRosen mentioned this pull request Oct 15, 2015

Edits for tutorial #106

Closed

JoshRosen added a commit that referenced this pull request Oct 17, 2015

Edits for tutorial

0a06c28

This patch is a follow-up to #101 and makes many minor edits in the tutorial text. /cc sameeraxiomine Author: Josh Rosen <joshrosen@databricks.com> Closes #106 from JoshRosen/tutorial-edits.

dorisZ017 pushed a commit to ActionIQ/spark-redshift that referenced this pull request May 18, 2023

Merge pull request databricks#101 from jsleight/u/jsleight/spark3.2

1f12470

Upgrade to spark v3.2.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tutorial #101

Tutorial #101

sameeraxiomine commented Sep 30, 2015

codecov-io commented Sep 30, 2015

JoshRosen Sep 30, 2015

sameeraxiomine Sep 30, 2015

JoshRosen commented Sep 30, 2015

JoshRosen Sep 30, 2015

sameeraxiomine Sep 30, 2015

sameeraxiomine Sep 30, 2015

JoshRosen Sep 30, 2015

sameeraxiomine Sep 30, 2015

JoshRosen commented Sep 30, 2015

sameeraxiomine commented Oct 1, 2015

JoshRosen commented Oct 15, 2015


		Figure 1 : UNLOAD action

		First the Spark Driver communicates with the Redshift Leader node to obtain the schema of the table (or query) requested. The attribute `override lazy val schema: StructType` in the class `com.databricks.spark.redshift.RedshiftRelation` will obtain the schema on demand by invoking the method `resolveTable` of the class `com.databricks.spark.redshift.JDBCWrapper`. The `JDBCWrapper` class is responsible for fetching the schema from the Redshift Leader.

Tutorial #101

Tutorial #101

Conversation

sameeraxiomine commented Sep 30, 2015

codecov-io commented Sep 30, 2015

Current coverage is 87.16%

JoshRosen Sep 30, 2015

Choose a reason for hiding this comment

sameeraxiomine Sep 30, 2015

Choose a reason for hiding this comment

JoshRosen commented Sep 30, 2015

JoshRosen Sep 30, 2015

Choose a reason for hiding this comment

sameeraxiomine Sep 30, 2015

Choose a reason for hiding this comment

sameeraxiomine Sep 30, 2015

Choose a reason for hiding this comment

JoshRosen Sep 30, 2015

Choose a reason for hiding this comment

sameeraxiomine Sep 30, 2015

Choose a reason for hiding this comment

JoshRosen commented Sep 30, 2015

sameeraxiomine commented Oct 1, 2015

JoshRosen commented Oct 15, 2015

Current coverage is `87.16%`