Secor v0.28 is released

[HenryCaiHaiying]

It's been a year since v0.27 is released. In v0.28, we upgrade the default build profile to kafka-2.0 and include various bug fixes.

Please take a look at https://github.com/pinterest/secor/blob/master/RELEASE.md to see the list of bug fixes.

[ksingh7]

@HenryCaiHaiying Thanks for releasing v0.28 , just now i tried to build jars using the codebase of v0.28 tag release but getting this

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.pinterest.secor.uploader.UploaderTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.107 s <<< FAILURE! - in com.pinterest.secor.uploader.UploaderTest
[ERROR] initializationError(com.pinterest.secor.uploader.UploaderTest)  Time elapsed: 0.005 s  <<< ERROR!
java.lang.IllegalAccessError: class jdk.internal.reflect.ConstructorAccessorImpl loaded by org.powermock.core.classloader.MockClassLoader @34cd072c cannot access jdk/internal/reflect superclass jdk.internal.reflect.MagicAccessorImpl

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.009 s <<< FAILURE! - in com.pinterest.secor.io.FileReaderWriterFactoryTest
[ERROR] initializationError(com.pinterest.secor.io.FileReaderWriterFactoryTest)  Time elapsed: 0 s  <<< ERROR!
java.lang.IllegalAccessError: class jdk.internal.reflect.ConstructorAccessorImpl loaded by org.powermock.core.classloader.MockClassLoader @5c448ef cannot access jdk/internal/reflect superclass jdk.internal.reflect.MagicAccessorImpl 

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.pinterest.secor.common.ZookeeperConnectorTest
[INFO] Running com.pinterest.secor.common.FileRegistryTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.002 s <<< FAILURE! - in com.pinterest.secor.common.FileRegistryTest
[ERROR] initializationError(com.pinterest.secor.common.FileRegistryTest)  Time elapsed: 0.001 s  <<< ERROR!
java.lang.IllegalAccessError: class jdk.internal.reflect.ConstructorAccessorImpl loaded by org.powermock.core.classloader.MockClassLoader @6b64429e cannot access jdk/internal/reflect superclass jdk.internal.reflect.MagicAccessorImpl

[INFO] Running com.pinterest.secor.monitoring.OstrichMetricCollectorTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.001 s <<< FAILURE! - in com.pinterest.secor.monitoring.OstrichMetricCollectorTest
[ERROR] initializationError(com.pinterest.secor.monitoring.OstrichMetricCollectorTest)  Time elapsed: 0 s  <<< ERROR!
java.lang.IllegalAccessError: class jdk.internal.reflect.ConstructorAccessorImpl loaded by org.powermock.core.classloader.MockClassLoader @64544ee4 cannot access jdk/internal/reflect superclass jdk.internal.reflect.MagicAccessorImpl

[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   FileRegistryTest.initializationError » IllegalAccess class jdk.internal.reflec...
[ERROR]   FileReaderWriterFactoryTest.initializationError » IllegalAccess class jdk.inte...
[ERROR]   OstrichMetricCollectorTest.initializationError » IllegalAccess class jdk.inter...
[ERROR]   UploaderTest.initializationError » IllegalAccess class jdk.internal.reflect.Co...
[INFO]
[ERROR] Tests run: 102, Failures: 0, Errors: 4, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  15.176 s
[INFO] Finished at: 2020-06-20T01:27:37+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project secor: There are test failures.
[ERROR]
[ERROR] Please refer to /Users/karasing/git/secor/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
datacenter:~/git/secor$

[ksingh7]

i am on macOS, with the following versions, if that helps

datacenter:~/git/secor$ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /Users/karasing/temp/apache-maven-3.6.3
Java version: 14.0.1, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/openjdk-14.0.1.jdk/Contents/Home
Default locale: en_DE, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.5", arch: "x86_64", family: "mac"
datacenter:~/git/secor$

[ksingh7]

I am not sure if I am missing something (it could very likely be, as i am new to java and trying to build the jars). Alternately if you have 0.28, jars that you can release or a Dockerfile that i can use.

Any support would be greatly appreciated

[HenryCaiHaiying]

Are you just doing 'mvn clean package"? I have an Linux env, I didn't see problems.

[HenryCaiHaiying]

I synced the code to head (not on v0.28 tag)

[HenryCaiHaiying]

@ksingh7 looks like you problem might be related to your java version (14), looks like this issue with MockClassLoader starts with java 9: https://stackoverflow.com/questions/50456726/mockclassloader-cannot-access-jdk-internal-reflect-superclass-jdk-internal-refle

[ksingh7]

@HenryCaiHaiying So i tried this

git checkout master
git pull origin master
mvn clean package

but this time as well it failed with exact same error

• Here are the java and mvn versions i have

datacenter:~/git/secor$ java --version
openjdk 14.0.1 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7)
OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)
datacenter:~/git/secor$
datacenter:~/git/secor$ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /Users/karasing/temp/apache-maven-3.6.3
Java version: 14.0.1, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/openjdk-14.0.1.jdk/Contents/Home
Default locale: en_DE, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.5", arch: "x86_64", family: "mac"
datacenter:~/git/secor$

What is your java/mvn version on your Linux workstation, may be i can give it a try on a Linux box. Any other suggestion you may have for me ?

[ksingh7]

@HenryCaiHaiying and as per this comment can you please guide me on what test base files i need to add the mentioned annotation ?

[ksingh7]

So there are a lot of test files, where i need to add

@RunWith(PowerMockRunner.class)
@PowerMockIgnore({"javax.management.", "com.sun.org.apache.xerces.", "javax.xml.", "org.xml.", "org.w3c.dom.",
"com.sun.org.apache.xalan.", "javax.activation.*"})
public class PowerMockitoBaseRunner {
}

Is there a base class, where i can change it once and inherited on all test files ?

[HenryCaiHaiying]

java8 definitely works, I am also trying the suggestions from that stack overflow with a java11 build.

[ksingh7]

After trying this comment

• pom.xml

• Test file

[ksingh7]

@HenryCaiHaiying i have tried a lot with no success, thank a TON for your help so far.

I will go and take some rest, will try again tomorrow. If you have any tips for me let me know

[ksingh7]

@HenryCaiHaiying confirming that build works on java8 (Linux) , flawlessly

[HenryCaiHaiying]

Yes, java8 works. I am still working on java11, fixed a few unit tests but still have a few more to go.

[HenryCaiHaiying]

This PR should be able to resolve unit test failures in JDK 11: #1400

[ksingh7]

@HenryCaiHaiying you are awesome, i will give it a try.