VRSpace.org

So, i gave it a shot locally on my Fedora 35 workstation. It built. That is, i dnf installed maven, and then cloned the source, and ran the mvn clean install command, and it runs, no errors. I can then run the server as well, it even appears to work! So now the question is why didnt it build in my container.

I'll start with the obvious difference. The container is based on the RHEL 8 UBI, and my workstation is Fedora 35. I also did not install the specialized java on my workstation, i just let maven include openjdk17, something that did not get included on my container, and in fact it wasnt available.

My container lists the following maven packages installed
[root@ea889734a3e2 vrspace]# rpm -qa | grep maven
maven-wagon-http-3.1.0-1.module+el8+2452+b359bfcd.noarch
maven-3.5.4-5.module+el8+2452+b359bfcd.noarch
maven-resolver-api-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-wagon-provider-api-3.1.0-1.module+el8+2452+b359bfcd.noarch
maven-resolver-spi-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-resolver-impl-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-resolver-connector-basic-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-lib-3.5.4-5.module+el8+2452+b359bfcd.noarch
maven-wagon-http-shared-3.1.0-1.module+el8+2452+b359bfcd.noarch
maven-resolver-util-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-shared-utils-3.2.1-0.1.module+el8+2452+b359bfcd.noarch
maven-resolver-transport-wagon-1.1.1-2.module+el8+2452+b359bfcd.noarch
maven-wagon-file-3.1.0-1.module+el8+2452+b359bfcd.noarch

and my fedora workstation has:
[nlager@batou-lan vrspace]$ rpm -qa | grep maven
maven-shared-utils-3.3.4-2.fc35.noarch
maven-wagon-3.4.2-4.fc35.noarch
maven-resolver-1.6.1-5.fc35.noarch
maven-lib-3.6.3-13.fc35.noarch
maven-openjdk17-3.6.3-13.fc35.noarch
maven-3.6.3-13.fc35.noarch


Now, jdk11 is what got installed in the container, and 17 is what came with maven on the workstation. I am going to restart the container from scratch with openjdk17 to see what happens.

I'm stabbing in the dark here, since I am not a java developer, nor a maven, or spring developer. So... shrug

same error with jdk17. The updated Dockerfile:


FROM registry.access.redhat.com/ubi8/ubi
RUN dnf -y install maven git java-17-openjdk
WORKDIR /app
RUN git clone https://github.com/jalmasi/vrspace.git
WORKDIR /app/vrspace
RUN JAVA_HOME=/usr/lib/jvm/jre-17/ mvn clean install
CMD ["java", "-jar", "/app/vrspace/server/server-0.2.2.jar"]


and the error


[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] DBTest.testCountUsers:561 expected: <2> but was: <3>
[ERROR] DBTest.testWorlds:526 expected: <2> but was: <3>
[INFO]
[ERROR] Tests run: 85, Failures: 2, Errors: 0, Skipped: 2


Something about docker and Neo4J, I can't reproduce it on either linux, windows or mac.
Created https://redmine.vrspace.org/issues/109

Workaround:
mvn clean install -DskipTests

The server is reported to work in docker just fine in spite of these failing tests.
Though the way they're failing tells me to expect disconnected users to remain in the space.

what exactly is the test doing? maybe i can help get to the bottom of it?

in any case i’ll try the work around, and i’ll let you knees if i have issues with disconnected users not leaving the space.

thanks!

Well, that worked around the build test problem, but now I cant actually run the server.

2022-02-24 13:14:03.620 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'config' defined in class path resource [org/vrspace/server/config/NeoConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.neo4j.graphdb.GraphDatabaseService]: Factory method 'config' threw exception; nested exception is java.lang.LinkageError: Cannot to link java.nio.DirectByteBuffer

Here is the container definition I am building from:

FROM registry.access.redhat.com/ubi8/ubi
RUN dnf -y install maven git java-17-openjdk
WORKDIR /app
RUN git clone https://github.com/jalmasi/vrspace.git
WORKDIR /app/vrspace
RUN JAVA_HOME=/usr/lib/jvm/jre-17/ mvn clean install -DskipTests
EXPOSE 8080
CMD ["/usr/lib/jvm/jre-17/bin/java", "-jar", "/app/vrspace/server/target/server-0.4.5-SNAPSHOT.jar"]


Is that java 17?

There's a big sausage of --add-opens this and that command line arguments that's required to make anything work with java 17, it's in the readme.
Java 17 is broken, but it's a feature ;)

You can see failing tests here: https://github.com/jalmasi/vrspace/blob/master/server/src/test/java/org/vrspace/server/core/DBTest.java
Scroll all the way down, these are two latest ones.

testWorlds() is trivial, simply adds two users to two worlds and counts worlds and users, testCountWorlds is more of the same, in more detail.
What could possibly go wrong? ;)

Well here's the source repository method tests call: https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/VRObjectRepository.java
These are simple "select" statements, called "match" in Neo4J terminology.
Why they fail, beats me, might be Neo4J bug. Best I can do is find workaround - if I can reproduce it.

I seem to have reproduced the error on a VM (not in a container). That is, the database test error. Using java 11, not 17. Yes that was 17 in my last example, because I thought i needed it.

The VM is also red hat enterprise linux 8.

[ERROR] Failures:
[ERROR] DBTest.testCountUsers:561 expected: <2> but was: <3>
[ERROR] DBTest.testWorlds:526 expected: <2> but was: <3>
[INFO]
[ERROR] Tests run: 85, Failures: 2, Errors: 0, Skipped: 2


Also, switching the container to java 11, fixed the run error. I can now start up vrspace using an image created from the following definition:

FROM registry.access.redhat.com/ubi8/ubi
RUN dnf -y install maven git java-11-openjdk
WORKDIR /app
RUN git clone https://github.com/jalmasi/vrspace.git
WORKDIR /app/vrspace
RUN JAVA_HOME=/usr/lib/jvm/jre-11/ mvn clean install -DskipTests
EXPOSE 8080
CMD ["/usr/lib/jvm/jre-11/bin/java", "-jar", "/app/vrspace/server/target/server-0.4.5-SNAPSHOT.jar"]

What's the RAM and VCPU count of your VM?

Now, I'm reasonably sure RHEL is not causing any issues, every vendor must make sure their stuff works well in it, or bye bye enterprise customers.

Please try to run only these tests, one by one:
mvn test -Dtest=DBTest#testWorlds
mvn test -Dtest=DBTest#testCountUsers
(there may be garbage remaining from previously executed tests)

4gb, 1 cpu. Libvirt guest running RHEL 8.5.

I am running vrspace as a user, not root, if that matters.

Yes, RHEL is a very stable, and tested platform, however it's also very conservative. One of the reasons ive been trying newer releases of some of the components (like when I tried java17) is because RHEL commonly lags behind in some libraries and frameworks for the sake of stability and compatibility.

Now, for the tests you've asked about. I have omitted some of the INFO output, I can provide the rest of the output if you need it.
[gangrif@meta vrspace]$ JAVA_HOME=/usr/lib/jvm/jre-11/ mvn test -Dtest=DBTest#testWorlds
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] VRSpace parent file [pom]
[INFO] server [jar]
[INFO] babylon [pom]
[INFO] content [jar]
[INFO] web [jar]
[INFO]
[INFO] -------------------------< org.vrspace:parent >-------------------------
[INFO] Building VRSpace parent file 0.4.5-SNAPSHOT [1/5]
...
2022-02-25 08:13:02.726 INFO 12264 --- [ionShutdownHook] org.vrspace.server.config.NeoConfig : Deleting database directory /opt/vrspace/server/vrspace-test.db
2022-02-25 08:13:02.781 INFO 12264 --- [ionShutdownHook] org.neo4j.driver.Driver : Closing driver instance 1379222018
2022-02-25 08:13:02.785 INFO 12264 --- [ionShutdownHook] org.neo4j.driver.ConnectionPool : Closing connection pool towards localhost:7688
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] ------------------------< org.vrspace:babylon >-------------------------
[INFO] Building babylon 0.4.5-SNAPSHOT [3/5]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ------------------------< org.vrspace:content >-------------------------
[INFO] Building content 0.4.5-SNAPSHOT [4/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) content ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /opt/vrspace/content/src/main/resources
[INFO] skip non existing resourceDirectory /opt/vrspace/content/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) content ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) content ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /opt/vrspace/content/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) content ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) content ---
[INFO] No tests to run.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] VRSpace parent file 0.4.5-SNAPSHOT ................. SUCCESS [ 0.027 s]
[INFO] server ............................................. SUCCESS [01:46 min]
[INFO] babylon ............................................ SUCCESS [ 0.003 s]
[INFO] content ............................................ FAILURE [ 0.175 s]
[INFO] web 0.4.5-SNAPSHOT ................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:49 min
[INFO] Finished at: 2022-02-25T08:13:03-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project content: No tests were executed! (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :content
[gangrif@meta vrspace]$@

And the CountUsers test
[gangrif@meta vrspace]$ mvn test -Dtest=DBTest#testCountUsers
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] VRSpace parent file [pom]
[INFO] server [jar]
[INFO] babylon [pom]
[INFO] content [jar]
[INFO] web [jar]
[INFO]
[INFO] -------------------------< org.vrspace:parent >-------------------------
[INFO] Building VRSpace parent file 0.4.5-SNAPSHOT [1/5]
...
[INFO] Properties file [/opt/vrspace/server/target/classes/git.properties] is up-to-date (for module server)...
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) server ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) server ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) server ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] VRSpace parent file 0.4.5-SNAPSHOT ................. SUCCESS [ 0.043 s]
[INFO] server ............................................. FAILURE [ 16.572 s]
[INFO] babylon ............................................ SKIPPED
[INFO] content ............................................ SKIPPED
[INFO] web 0.4.5-SNAPSHOT ................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19.538 s
[INFO] Finished at: 2022-02-25T08:18:20-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project server: There are test failures.
[ERROR]
[ERROR] Please refer to /opt/vrspace/server/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /opt/vrspace/server && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre/bin/java --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED --add-opens java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED -jar /opt/vrspace/server/target/surefire/surefirebooter7294610754902669062.jar /opt/vrspace/server/target/surefire 2022-02-25T08-18-19_227-jvmRun1 surefire6225211361224354243tmp surefire_03333812921161510946tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /opt/vrspace/server && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-2.el8_5.x86_64/jre/bin/java --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED --add-opens java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED -jar /opt/vrspace/server/target/surefire/surefirebooter7294610754902669062.jar /opt/vrspace/server/target/surefire 2022-02-25T08-18-19_227-jvmRun1 surefire6225211361224354243tmp surefire_03333812921161510946tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
[ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
[ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:954)
[ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
[ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.lang.reflect.Method.invoke(Method.java:498)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :server@

i gave building as root a try, and that seemed to work, no errors. Does this need to run as root?

Edit: building as root, then giving my user permissions to the directory structure, then running as my user, appears to work. I was able to start up the server, and connect from two browsers to test multiplayer.

I wonder if this has some bearing on why the test fails in a container? If you don't mind ill add some of this detail into the issue you logged yesterday regarding this.

Discussion moved to https://redmine.vrspace.org/issues/109