Added org.geotools.{gt-api, gt-opengis} dependencies to core/pom.xml

[mbasmanova]

Without these dependencies, I'm getting reference-not-found errors for the following import statements in SpatialRDD.java:

import org.geotools.geometry.jts.JTS;
import org.opengis.referencing.FactoryException;
import org.opengis.referencing.crs.CoordinateReferenceSystem;
import org.opengis.referencing.operation.MathTransform;

[jiayuasu]

@mbasmanova Hi Masha, I can pass "mvn clean install" without using these dependencies. And travis-ci also can pass it. Are there any special settings on your side?

[jiayuasu]

And, adding these dependencies makes GeoSpark incompatible with previous version. I tried to update GeoSparkTemplateProject using SNAPSHOT dependency but failed because of not finding HintKey.

[mbasmanova]

@jiayuasu , the main difference in setups is that I cannot pull dependencies from repos other then Maven Central. In particular, I cannot pull dependencies from osgeo. To allow build to succeed, I manually downloaded gt-* jars to local repo (~/.m2/repository/org/geotools/).

It seems to me that gt-api, gt-opengis dependencies are actually used, but they are not declared explicitly and mvn is able to pull them as transitive dependencies. Adding maven-dependency-plugin [1] to the pom file may help reveal these "used, but undeclared" dependencies.

The problematic classes are the following:

import org.geotools.geometry.jts.JTS;
import org.opengis.referencing.FactoryException;
import org.opengis.referencing.crs.CoordinateReferenceSystem;
import org.opengis.referencing.operation.MathTransform;

I see the first one in gt-api:

$ jar tf ~/.m2/repository/org/geotools/gt-api/17.0/gt-api-17.0.jar | grep org.geotools.geometry.jts.JTS.class
org/geotools/geometry/jts/JTS.class

And the other three in gt-opengis:

$ jar tf ~/.m2/repository/org/geotools/gt-opengis/17.0/gt-opengis-17.0.jar | grep org.opengis.referencing.FactoryException
org/opengis/referencing/FactoryException.class

$ jar tf ~/.m2/repository/org/geotools/gt-opengis/17.0/gt-opengis-17.0.jar | grep org.opengis.referencing.crs.CoordinateReferenceSystem
org/opengis/referencing/crs/CoordinateReferenceSystem.class

$ jar tf ~/.m2/repository/org/geotools/gt-opengis/17.0/gt-opengis-17.0.jar | grep org.opengis.referencing.operation.MathTransform.class
org/opengis/referencing/operation/MathTransform.class

Where can I find GeoSparkTemplateProject ? I'd like to see why it fails to build.

[1] https://maven.apache.org/plugins/maven-dependency-plugin/examples/failing-the-build-on-dependency-analysis-warnings.html

[mbasmanova]

I found GeoSparkTemplateProject and ran 'mvn package' in geospark/java. I didn't get any errors, but there are warnings about duplicate classes defined across geospark, jscience and sernetcdf jars. @jiayuasu , what was the build error you saw?

[INFO] --- maven-shade-plugin:2.1:shade (default) @ GeoSparkJavaTemplate ---
[INFO] Including org.datasyslab:geospark:jar:0.9.0-SNAPSHOT in the shaded jar.
[INFO] Including org.datasyslab:sernetcdf:jar:0.1.0 in the shaded jar.
[INFO] Including org.geotools:gt-metadata:jar:17.0 in the shaded jar.
[INFO] Including org.geotools:gt-data:jar:17.0 in the shaded jar.
[INFO] Including org.jscience:jscience:jar:4.3.1 in the shaded jar.
[INFO] Including org.javolution:javolution:jar:5.2.3 in the shaded jar.
[INFO] Including com.googlecode.efficient-java-matrix-library:core:jar:0.26 in the shaded jar.
[INFO] Including org.hsqldb:hsqldb:jar:2.3.0 in the shaded jar.
[WARNING] geospark-0.9.0-SNAPSHOT.jar, jscience-4.3.1.jar define 22 overlappping classes: 
[WARNING]   - org.opengis.referencing.crs.CoordinateReferenceSystem
[WARNING]   - org.opengis.referencing.cs.VerticalCS
[WARNING]   - org.opengis.metadata.Identifier
[WARNING]   - org.opengis.referencing.cs.SphericalCS
[WARNING]   - org.opengis.util.InternationalString
[WARNING]   - org.opengis.util.Cloneable
[WARNING]   - org.opengis.referencing.cs.LinearCS
[WARNING]   - org.opengis.metadata.extent.Extent
[WARNING]   - org.opengis.referencing.ReferenceSystem
[WARNING]   - org.opengis.referencing.cs.CylindricalCS
[WARNING]   - 12 more...
[WARNING] jscience-4.3.1.jar, javolution-5.2.3.jar define 316 overlappping classes: 
[WARNING]   - javolution.context.PersistentContext
[WARNING]   - javolution.testing.AssertionException
[WARNING]   - javolution.io.Struct$Bool
[WARNING]   - javolution.testing.TestCase
[WARNING]   - javolution.context.HeapContext$1
[WARNING]   - javolution.lang.Reference
[WARNING]   - javolution.ContextTestSuite$Concurrency
[WARNING]   - javolution.context.ArrayFactory$22
[WARNING]   - javolution.xml.stream.AttributesImpl$1
[WARNING]   - javolution.xml.XMLBinding$7
[WARNING]   - 306 more...
[WARNING] geospark-0.9.0-SNAPSHOT.jar, sernetcdf-0.1.0.jar define 714 overlappping classes: 
[WARNING]   - com.fasterxml.jackson.annotation.JsonAutoDetect
[WARNING]   - com.fasterxml.jackson.databind.deser.impl.BeanAsArrayBuilderDeserializer
[WARNING]   - com.fasterxml.jackson.databind.introspect.AnnotatedMethodMap
[WARNING]   - com.fasterxml.jackson.databind.BeanDescription
[WARNING]   - com.fasterxml.jackson.databind.SerializerProvider
[WARNING]   - com.fasterxml.jackson.core.sym.BytesToNameCanonicalizer
[WARNING]   - com.fasterxml.jackson.databind.ser.std.StaticListSerializerBase
[WARNING]   - com.fasterxml.jackson.databind.ser.std.NumberSerializers$ShortSerializer
[WARNING]   - com.fasterxml.jackson.databind.ser.BeanSerializerFactory
[WARNING]   - com.fasterxml.jackson.databind.jsonFormatVisitors.JsonMapFormatVisitor
[WARNING]   - 704 more...
[WARNING] maven-shade-plugin has detected that some .class files
[WARNING] are present in two or more JARs. When this happens, only
[WARNING] one single version of the class is copied in the uberjar.
[WARNING] Usually this is not harmful and you can skeep these
[WARNING] warnings, otherwise try to manually exclude artifacts
[WARNING] based on mvn dependency:tree -Ddetail=true and the above
[WARNING] output
[WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin
[INFO] Replacing original artifact with shaded artifact.

[jiayuasu]

@mbasmanova The current GeoSparkTemplateProject can pass the execution because I manually add the dependencies such gt-api and gt-opengis in build.sbt and POM.xml.

I was expecting the code at this moment ( jiayuasu/GeoSparkTemplateProject@d865c68) should fit in GeoSpark 0.9.0-snapshot (lower case) without any other changes. But it actually failed. In order to pass CRS transformation, I have to repackage gt-api and gt-opengis again in the template project.

GeoSpark 0.9.0-snapshot contains your commit of gt-api and gt-opengis.