VRSpace.org

{{toc}}

h1. Welcome!

h2. External resources

Demo site: https://www.vrspace.org/

Github project page: https://github.com/jalmasi/vrspace

OpenSource.com articles: 

https://opensource.com/article/20/12/virtual-reality-server

https://opensource.com/article/22/1/open-source-metaverse

VR Days video (features, technologies): https://vimeo.com/475142708

Open Source Metaverse talk in London, Open Source in Finance conference: https://www.youtube.com/watch?v=lIQd_3935Gk&list=PLbzoR-pLrL6rr72M1w7pTYw-X8FduLz1v&index=10

Youtube channel: https://www.youtube.com/channel/UCLdSg22i9MZ3u7ityj_PBxw

Facebook page: https://fb.com/vrspace.org

Client API jsdoc: https://www.vrspace.org/docs/jsdoc/index.html

Server javadoc: https://www.vrspace.org/docs/javadoc/index.html?overview-summary.html

h2. Playgrounds

Multi-user interaction, shared objects: https://playground.babylonjs.com/#ZBK155

Multi-user world: https://playground.babylonjs.com/#Y6ILJ5

Avatar selection, portals: https://playground.babylonjs.com/#HDV7LA

VR Avatar template world: https://www.babylonjs-playground.com/#VXA0R3

h1. Frequently Asked Questions

h2. Is this open source, free to use and share?

Yes, it's all free and open. Server and client code is published under Apache 2 license, all 3D models published by their respective authors under Creative Commons Attribution license.

h2. Is there any documentation?

This is good place to start, this page links to all available resources.

h2. I want to build my space, where do I start?

This question prompted writing Getting Started section of this wiki, but set up basic development environment first.

You don't need to modify any of the server code, but you need to be able to get it from github and build it.

Once you do, start it up, and everything else is html 5 and javascript ES6.

h2. How do I create my own world(s)?

You can use any authoring tool that exports glTF: Blender, Unity, Unreal, to name a few. You will likely need some glTF exporter plugin.

Export your model in a folder under vrspace/content/worlds, vrspace server automatically picks it up, and creates another portal for it.

Save a screenshot of your world as jpg file in vrspace/content/worlds, and it will be used as portal texture.

h2. How many users can share a “world/space” simultaneously?

By default, number of users per space is not limited. There is a server parameter to limit that (org.vrspace.server.maxSessions) that can be specified either in config file or command line. Users that hit the limit remain in the queue for configurable timeout (org.vrspace.server.sessionStartTimeout, zero by default) until someone leaves. An error is raised on timeout.

h2. How can I implement custom authentication and/or authorization?

Typically you will authenticate users on your web server, before they enter the world. 

If that's not enough, you can implement your own "ClientFactory":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/ClientFactory.java and configure it in "application.properties":https://github.com/jalmasi/vrspace/blob/master/server/src/main/resources/application.properties file.

This allows you to do your own database queries, REST calls or whatever else you may need to identify your users.

h2. How is that different from a game server?

It's not all that different, though the emphasis is different: vrspace.org is not focused on games specifically. It's more about multi-user 3D web, in particular WebXR, open standards and open source.

So while game servers usually don't bother with video/audio streaming, vrspace.org provides it out of the box, using WebRTC. On the other hand, there's nothing like game level in vrspace.org.

Basic difference is that, while game servers focus on sharing the state of the game, vrspace.org server shares the state of any number of objects. No object in particular special in any way, there's no 'game loop' object.

h2. How does inverse kinematics and motion tracking work in vrspace.org characters?

Everything is encapsulated "Avatar class":https://github.com/jalmasi/vrspace/blob/master/babylon/js/ui/avatar.js, but relies on observed glTF avatar structure. Heuristics used to analyze the character structure is explained in depth in "this research paper":https://www.researchgate.net/publication/356987355_TOWARDS_AUTOMATIC_SKELETON_RECOGNITION_OF_HUMANOID_3D_CHARACTER.

h2. Why websockets?

Since death of internet explorer, websockets just work, in any browser. Some reliable multicast protocol would do better job for object state distribution, but no such protocol is supported by web browsers. WebRTC on the other hand is widely supported, but is unreliable, and requires complex development environment setup.

h2. Can I make/buy and use my own unique avatar?

Sure you can, but note that technically you can't prevent anybody else from copying and using it. Using video avatar is the only practical option to have unique appearance.

h2. How to enable https?

The server comes bundled with self-signed certificate, so all you need to do to enable SSL is change *server.ssl.enabled* property in "application.properties":https://github.com/jalmasi/vrspace/blob/master/server/src/main/resources/application.properties to true. You may also want to set *server.port* to 443.

h2. Running in a container (docker, podman, OCI compliant)

h3. Build your own

See step-by-step guide and the discussion here: https://redmine.vrspace.org/boards/2/topics/226

h3. Run the pre-built community maintained container image

Pull the container from Docker Hub.  https://hub.docker.com/r/gangrif/vrspace

h1. Getting started

So, you're new to VRSpace, and you want to make yourself a new virtual world.

You will not need to change any of the server code, but you need to get it from github, build it, and start it.

Once done, familiarize yourself with directory structure, and especially with template world, as this is intended stating point.

h2. Directory structure

h3. server

Contains the server code and configuration. Generally you don't need to change the code, but you may want to change the configuration. Main configuration file is in src/main/resources/application.properties, have a look at what you can change.

The server executable is a jar file in target subdirectory.

h3. babylon

This is reference Babylon.js client implementation. You don't really need any of it, but it is highly recommended to build on top of it, using javascript code provided there.

Html files there implement and/or test different functions available, like avatar loading and movement, video/audio streaming etc.

Most important one is avatar-selection.html, this is main entry point that allows user to select avatar and enter a world.

h3. web

This is the actual web site that's available online on vrspace.org, your final web site will probably look somewhat like this.

h3. content

Actual web content used to build virtual worlds, with self-explanatory directory names. Two of these contain special attention though:

h4. char

Characters, under female, male and other subdirectories. These directories are listed and presented by avatar-selection.html, you will probably to change how it works eventually.

But adding or removing characters is as simple as dropping them here.

h4. worlds

Available worlds, also listed by avatar-selection.html, and displayed as portals. Every subdirectory contains one world, and also note there's an image with the same name - that's thumbnail displayed on the portal.

Every world has some specific features.

Paris is huge, a real-life model.

Persian city and Aladin both feature the same procedural terrain and plants.

Cave is an actual game level, and utilizes floors built with vrspace.org floor editor.

Classrooom allows for sharing screen.

But the template is where you start - copy it over to another directory, and edit to suit your needs.

h2. Your first world

So with the server running, copy the template directory to your own. This world is immediately listed at http://localhost:8080/content/worlds/ and also available in http://localhost:8080/babylon/avatar-selection.html as a portal.

All you need to do to replace that dolphin with your own world is to replace this.file='dolphin.glb' in world.js with your own file name. Or really, delete the entire constructor, and save your world as scene.gltf in the same directory.

Hit reload button/key in your web browser, changes become visible immediately.

There are two files there, a html and a js, but html also contains some javascript.

The idea behind this separation of concerns is that world.js contains all programming logic that relates to the world itself, it should not depend on any server functionality. So build your world first, then add multi-user functions to it. Javascript in the world.html file shows some basic multiuser functions. As these grow, you'll probably want them in another javascript file, but these are minimum to get started.

Hopefully comments world.js and world.html explain enough to get started. As you need more functionality, look at other worlds and copy it over.

h1. Setting up development environment

Here's all you need to start development on Windows.

Linux distributions include all these tools as native packages, so no additional downloads should be required.

h2. Basic setup

h3. Git bash

IDEs can work with github directly, but whatever you ask, you'll get a command line answer.

Command line git is simply a must have. Bash also includes a lot of goodies like ssh.

https://git-scm.com/downloads

h3. Java

Java 8 will work, java 11 recommended. JDK is required to build the server.

Get it either from Oracle or elsewhere, e.g. Zulu OpenJDK: https://www.azul.com/downloads/zulu-community/?package=jdk

h3. Node.js

Node is used by IDE to evaluate javascript. You'll also may need it if you modify any of babylon.js source.

Mind that IDE will complain if you installed unsupported version of Node; should that happen, remove Node, and install latest one supported.

Get it from https://nodejs.org/

h3. Maven

Apache Maven is used to build the server from command prompt.

Get it from https://maven.apache.org/download.cgi

h3. IDE

Eclipse for Java Enterprise Java and Web Developers: https://www.eclipse.org/

Install Lombok plugin https://projectlombok.org/

And then go to Help -> Eclipse Marketplace

Search and install Spring Tools 4.

h2. Import and start the project

h3. git bash

git clone https://github.com/jalmasi/vrspace.git

cd vrspace

mvn clean install

cd server

java -jar target/server-VERSION-executable.jar

(server version is printed after the build, e.g. 0.6.3-SNAPSHOT)

h3. IDE

In Eclipse, you can use either default or new workspace for the project.

Assuming you have cloned the project from the github, 

Go to File -> Open Projects From the Filesystem

Then choose vrspace directory.

This will import vrspace folders and project subfolders, click Finish.

In vrspace project folder, in src/main/java, there's org.vrspace.server.ServerApplication.java.

Open it, then right click on the code.

From the menu, choose either Run as or Debug as -> Spring Boot App.

To run with java 17, you'll need to specify run configuration instead, with VM arguments: --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED --add-opens java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED

Open http://localhost:8080/babylon/connect.html with two browsers, and navigate around.

That's all, you're all set!

h2. Advanced setup

h3. SSL

HTTPS is required for pretty much everything - WebXR, camera, mic access.

By default, the server runs on 8080 port with plain HTTP. To enable HTTPS, edit application.properties (found in src/main/resources), and change following properties:

<pre>

server.ssl.enabled=false

# default port 8080

#server.port=8443

</pre>

h3. Apache

Apache reverse proxy setup, linux, windows, TBD

h3. Docker and OpenVidu

OpenVidu voice/video chat server runs as docker image. This is only required for development of voice chat functions.

Modify openvidu.publicurl and openvidu.secret in application.properties, or run server.jar with -Dopenvidu.publicurl=YOUR_URL and -Dopenvidu.secret=YOUR_SECRET

h4. Local execution (development)

docker run -p 4443:4443 --rm -e FORCE_PLAIN_HTTP=false -e SERVER_SSL_ENABLED=true -e OPENVIDU_SECRET=YOUR_SECRET -e DOMAIN_OR_PUBLIC_IP=YOUR_IP openvidu/openvidu-server-kms:2.30.1

Enter a space. You may get

WebSocket connection to 'wss://YOUR_IP:4443/openvidu?sessionId=cave' failed: Error in connection establishment: net::ERR_CERT_AUTHORITY_INVALID

in the console. To get rid of it, open https://YOUR_IP:4443/ and accept the cert.

(using localhost as YOUR_IP may not work)

h4. Running on server

https://docs.openvidu.io/en/2.30.0/deployment/deploying-on-premises/

h1. Software Architecture

!https://redmine.vrspace.org/attachments/download/15/vrspace-diagram.png!

h2. Client-Server Communication

Clients communicate with server by sending JSON messages over "WebSockets":https://en.wikipedia.org/wiki/WebSocket. Reference javascript implementation of client communication layer is in "VRSpace.js":https://www.vrspace.org/docs/jsdoc/VRSpace.html.

General approach to communication is rather obscure Half-Object pattern: server-side and client-side object have same properties, but different implementations.

Whenever an object's property changes in (any) client's address space, it's transmitted to the server, that broadcasts it to all clients currently 'watching' the object.

Whenever a client wants to perform any change to any object in the space, it has to go through the VRSpace server.

Clients may or may not communicate directly, but this is out of the scope of VRSpace server.

h2. Server Responsibilities

Sole responsibility of VRSpace server is management of 3D space: persisting space objects, tracking their properties, processing and distributing events from/to objects.

Whenever we talk about objects, that includes clients, i.e. users - a client is a special case of an object.

The server does not even handle the authentication - it is assumed to be responsibility of web app serving the space.

3D geometry is also not in server's scope, it's just another property of an object (mesh).

h2. Server design

Key concepts here are Active Objects, Actor model, and Live distributed object.

"VRObject":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/VRObject.java is a basic shared object, with some basic properties like position and rotation, and of course, mesh. It essentially a "Business object":https://en.wikipedia.org/wiki/Business_object acting as a "Live distributed object":https://en.wikipedia.org/wiki/Live_distributed_object.

"Client":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/Client.java extends VRObject, and adds capabilities to communicate over web sockets, and listen to changes to other objects. Typically, a client represents a remote user, but it can also represent a robot connected over a web socket, or be a base class for a server-side robot.

A Client can be thought of as "Active Object":https://en.wikipedia.org/wiki/Active_object or an "Actor":https://en.wikipedia.org/wiki/Actor_model, 

Server relies on Spring Boot and embedded Tomcat to handle all I/O and threads.

Design does not require websockets in particular, in fact some reliable multicast protocol would surely fit better. Websockets work in web browsers.

h3. Client

Each "Client":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/Client.java has it's own "Scene":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/Scene.java that tracks all shared objects - including other clients - visible by the client. The scene is initially populated once the client logs in to the server, and starts the session. Scene is refreshed periodically, after a movement, or explicitly.

Scene maintains the event model, by adding the Client as listener to all other active objects (usually other users) in the scene.

Client simply notifies it's own listeners on any changes to any of it's own properties. As a listener, it propagates any observed change on any other object over the network, to the user.

Client has just a couple of persistent properties, like position, rotation and name. The name must be unique.

All other properties are transient.

"ClientFactory":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/ClientFactory.java allows for customization of Client class and instances.

h3. Events and messages

Typically events exchanged are changes to properties of active objects, e.g. user moves around, changing own position and rotation. However, an object may emit any event, and the event gets propagated to all listeners.

This is to simplify client development: simply emit any custom event you want.

All distributed events are encapsulated in "VREvent":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/VREvent.java class.

However, there are other types of messages that are not distributed to other clients: commands and their return values, and errors.

Client will typically execute at least one command during the session - "Session":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/Session.java start. It will not receive any messages before that.

Should any server-side exception occur during the session, errors are sent to the client as simple JSON maps.

"WorldManager":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/WorldManager.java manages clients and event distribution between them.

h3. Worlds

A server can host many worlds, i.e. shared spaces. A world contains users and other objects that are shared only within the world. It can be thought of as a chatroom, but mind the difference: being in the same world does not mean that users can see and talk to each other.

Client can enter another world by issuing "Enter":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/Enter.java command.

h3. Ownership and privacy

A class can be annotated as Owned, and all of instances automatically become owned as they are created. An owned object does not receive events from anyone by owner(s). A Client is owned, and nobody but himself can change his name.

On the other hand, a generic VRObject is public, and receives events from anyone. Like a door, anybody can open and close it. But even public objects can have some properties annotated as Owned, that can't be change by anyone but owner.

Fields of a class can be annotated as Private, and their contents will never be published.

"Dispatcher":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/Dispatcher.java takes care of it before property changes are applied, and before they are published to listeners.

h3. Package structure

Package "org.vrspace.server":https://github.com/jalmasi/vrspace/tree/master/server/src/main/java/org/vrspace/server contains only main application class. Under it,

- config package contains server configuration classes that are executed only on server statup

- core is the core of the server

- dto is misnomer that stands Data Transfer Objects, though this package contains all objects that are not shared nor persisted

- obj contains persistent objects, most importantly Client and VRObject

- types are custom types used elsewhere, be it interfaces or annotations

- web is a primitive admin interface, disabled by default

- api contains REST controllers