Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DJL training test failure #1606

Closed
lanking520 opened this issue Apr 26, 2022 · 0 comments · Fixed by #1623
Closed

DJL training test failure #1606

lanking520 opened this issue Apr 26, 2022 · 0 comments · Fixed by #1623
Labels
bug Something isn't working

Comments

@lanking520
Copy link
Contributor

Description

happening on training with GPU

[ERROR] - Test ai.djl.integration.tests.training.GradientCollectorIntegrationTest.testFreezeParameters FAILED
--
930 | [ERROR] -
931 | java.lang.IllegalStateException: Split can only be called on a batch containing a batchifier
932 | at ai.djl.training.dataset.Batch.split(Batch.java:222) ~[api-0.17.0-SNAPSHOT.jar:?]
933 | at ai.djl.training.dataset.Batch.split(Batch.java:196) ~[api-0.17.0-SNAPSHOT.jar:?]
934 | at ai.djl.training.EasyTrain.trainBatch(EasyTrain.java:89) ~[api-0.17.0-SNAPSHOT.jar:?]
935 | at ai.djl.integration.tests.training.GradientCollectorIntegrationTest.testFreezeParameters(GradientCollectorIntegrationTest.java:110) ~[main/:?]
936 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
937 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
938 | at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
939 | at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
940 | at ai.djl.integration.IntegrationTest$TestClass.runTest(IntegrationTest.java:351) ~[main/:?]
941 | at ai.djl.integration.IntegrationTest.runTests(IntegrationTest.java:111) ~[main/:?]
942 | at ai.djl.integration.IntegrationTest.runTests(IntegrationTest.java:80) ~[main/:?]
943 | at ai.djl.integration.IntegrationTests.runIntegrationTests(IntegrationTests.java:46) ~[test/:?]
944 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
945 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
946 | at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
947 | at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
948 | at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135) ~[testng-7.5.jar:7.5]
949 | at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673) ~[testng-7.5.jar:7.5]
950 | at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220) ~[testng-7.5.jar:7.5]
951 | at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50) ~[testng-7.5.jar:7.5]
952 | at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945) ~[testng-7.5.jar:7.5]
953 | at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193) ~[testng-7.5.jar:7.5]
954 | at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146) ~[testng-7.5.jar:7.5]
955 | at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128) ~[testng-7.5.jar:7.5]
956 | at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
957 | at org.testng.TestRunner.privateRun(TestRunner.java:808) ~[testng-7.5.jar:7.5]
958 | at org.testng.TestRunner.run(TestRunner.java:603) ~[testng-7.5.jar:7.5]
959 | at org.testng.SuiteRunner.runTest(SuiteRunner.java:429) ~[testng-7.5.jar:7.5]
960 | at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:423) ~[testng-7.5.jar:7.5]
961 | at org.testng.SuiteRunner.privateRun(SuiteRunner.java:383) ~[testng-7.5.jar:7.5]
962 | at org.testng.SuiteRunner.run(SuiteRunner.java:326) ~[testng-7.5.jar:7.5]
963 | at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) ~[testng-7.5.jar:7.5]
964 | at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) ~[testng-7.5.jar:7.5]
965 | at org.testng.TestNG.runSuitesSequentially(TestNG.java:1249) ~[testng-7.5.jar:7.5]
966 | at org.testng.TestNG.runSuitesLocally(TestNG.java:1169) ~[testng-7.5.jar:7.5]
967 | at org.testng.TestNG.runSuites(TestNG.java:1092) ~[testng-7.5.jar:7.5]
968 | at org.testng.TestNG.run(TestNG.java:1060) ~[testng-7.5.jar:7.5]
969 | at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:141) ~[?:?]
970 | at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:90) ~[?:?]
971 | at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) ~[?:?]
972 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
973 | at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
974 | at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
975 | at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
976 | at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) ~[?:?]
977 | at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) ~[?:?]
978 | at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) ~[?:?]
979 | at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) ~[?:?]
980 | at com.sun.proxy.$Proxy5.stop(Unknown Source) ~[?:?]
981 | at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193) ~[?:?]
982 | at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) ~[?:?]
983 | at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) ~[?:?]
984 | at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) ~[?:?]
985 | at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) ~[?:?]
986 | at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) ~[?:?]
987 | at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) ~[?:?]
988 | at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) ~[gradle-worker.jar:?]
989 | at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74) ~[gradle-worker.jar:?]
@lanking520 lanking520 added the bug Something isn't working label Apr 26, 2022
zachgk added a commit to zachgk/djl that referenced this issue May 5, 2022
zachgk added a commit that referenced this issue May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant