Project

General

Profile

Wiki » History » Version 82

Jasna A, 06/18/2024 03:09 PM

1 4 Josip Almasi
{{toc}}
2
3 1 Josip Almasi
h1. Welcome!
4 2 Josip Almasi
5 18 Josip Almasi
h2. External resources
6 4 Josip Almasi
7 14 Josip Almasi
Demo site: https://www.vrspace.org/
8
9 15 Josip Almasi
Github project page: https://github.com/jalmasi/vrspace
10 2 Josip Almasi
11 63 Josip Almasi
OpenSource.com articles: 
12
https://opensource.com/article/20/12/virtual-reality-server
13
https://opensource.com/article/22/1/open-source-metaverse
14 3 Josip Almasi
15 15 Josip Almasi
VR Days video (features, technologies): https://vimeo.com/475142708
16 3 Josip Almasi
17 73 Josip Almasi
Open Source Metaverse talk in London, Open Source in Finance conference: https://www.youtube.com/watch?v=lIQd_3935Gk&list=PLbzoR-pLrL6rr72M1w7pTYw-X8FduLz1v&index=10
18 70 Josip Almasi
19 15 Josip Almasi
Youtube channel: https://www.youtube.com/channel/UCLdSg22i9MZ3u7ityj_PBxw
20 1 Josip Almasi
21 16 Josip Almasi
Facebook page: https://fb.com/vrspace.org
22 14 Josip Almasi
23
Client API jsdoc: https://www.vrspace.org/docs/jsdoc/index.html
24
25
Server javadoc: https://www.vrspace.org/docs/javadoc/index.html?overview-summary.html
26 72 Josip Almasi
27
28 23 Josip Almasi
h2. Playgrounds
29
30
Multi-user interaction, shared objects: https://playground.babylonjs.com/#ZBK155
31
Multi-user world: https://playground.babylonjs.com/#Y6ILJ5
32
Avatar selection, portals: https://playground.babylonjs.com/#HDV7LA
33 24 Josip Almasi
VR Avatar template world: https://www.babylonjs-playground.com/#VXA0R3
34 23 Josip Almasi
35 17 Josip Almasi
h1. Frequently Asked Questions
36
37
h2. Is this open source, free to use and share?
38
39
Yes, it's all free and open. Server and client code is published under Apache 2 license, all 3D models published by their respective authors under Creative Commons Attribution license.
40
41
h2. Is there any documentation?
42
43
This is good place to start, this page links to all available resources.
44
45 45 Josip Almasi
h2. I want to build my space, where do I start?
46
47 46 Josip Almasi
This question prompted writing Getting Started section of this wiki, but set up basic development environment first.
48
You don't need to modify any of the server code, but you need to be able to get it from github and build it.
49
Once you do, start it up, and everything else is html 5 and javascript ES6.
50 45 Josip Almasi
51 66 Josip Almasi
h2. How do I create my own world(s)?
52
53
You can use any authoring tool that exports glTF: Blender, Unity, Unreal, to name a few. You will likely need some glTF exporter plugin.
54 67 Josip Almasi
Export your model in a folder under vrspace/content/worlds, vrspace server automatically picks it up, and creates another portal for it.
55 68 Josip Almasi
Save a screenshot of your world as jpg file in vrspace/content/worlds, and it will be used as portal texture.
56 66 Josip Almasi
57 17 Josip Almasi
h2. How many users can share a “world/space” simultaneously?
58
59
By default, number of users per space is not limited. There is a server parameter to limit that (org.vrspace.server.maxSessions) that can be specified either in config file or command line. Users that hit the limit remain in the queue for configurable timeout (org.vrspace.server.sessionStartTimeout, zero by default) until someone leaves. An error is raised on timeout.
60
61 39 Josip Almasi
h2. How can I implement custom authentication and/or authorization?
62 1 Josip Almasi
63 39 Josip Almasi
Typically you will authenticate users on your web server, before they enter the world. 
64
If that's not enough, you can implement your own "ClientFactory":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/ClientFactory.java and configure it in "application.properties":https://github.com/jalmasi/vrspace/blob/master/server/src/main/resources/application.properties file.
65 1 Josip Almasi
This allows you to do your own database queries, REST calls or whatever else you may need to identify your users.
66
67 39 Josip Almasi
h2. How is that different from a game server?
68 38 Josip Almasi
69 40 Josip Almasi
It's not all that different, though the emphasis is different: vrspace.org is not focused on games specifically. It's more about multi-user 3D web, in particular WebXR, open standards and open source.
70 39 Josip Almasi
So while game servers usually don't bother with video/audio streaming, vrspace.org provides it out of the box, using WebRTC. On the other hand, there's nothing like game level in vrspace.org.
71 59 Josip Almasi
Basic difference is that, while game servers focus on sharing the state of the game, vrspace.org server shares the state of any number of objects. No object in particular special in any way, there's no 'game loop' object.
72 38 Josip Almasi
73 42 Josip Almasi
h2. How does inverse kinematics and motion tracking work in vrspace.org characters?
74 41 Josip Almasi
75 62 Josip Almasi
Everything is encapsulated "Avatar class":https://github.com/jalmasi/vrspace/blob/master/babylon/js/ui/avatar.js, but relies on observed glTF avatar structure. Heuristics used to analyze the character structure is explained in depth in "this research paper":https://www.researchgate.net/publication/356987355_TOWARDS_AUTOMATIC_SKELETON_RECOGNITION_OF_HUMANOID_3D_CHARACTER.
76 41 Josip Almasi
77 56 Josip Almasi
h2. Why websockets?
78
79
Since death of internet explorer, websockets just work, in any browser. Some reliable multicast protocol would do better job for object state distribution, but no such protocol is supported by web browsers. WebRTC on the other hand is widely supported, but is unreliable, and requires complex development environment setup.
80
81 60 Josip Almasi
h2. Can I make/buy and use my own unique avatar?
82
83
Sure you can, but note that technically you can't prevent anybody else from copying and using it. Using video avatar is the only practical option to have unique appearance.
84
85 64 Josip Almasi
h2. How to enable https?
86
87 65 Josip Almasi
The server comes bundled with self-signed certificate, so all you need to do to enable SSL is change *server.ssl.enabled* property in "application.properties":https://github.com/jalmasi/vrspace/blob/master/server/src/main/resources/application.properties to true. You may also want to set *server.port* to 443.
88 64 Josip Almasi
89 75 Nate Lager
h2. Running in a container (docker, podman, OCI compliant)
90 74 Josip Almasi
91 76 Nate Lager
h3. Build your own
92 77 Nate Lager
93 74 Josip Almasi
See step-by-step guide and the discussion here: https://redmine.vrspace.org/boards/2/topics/226
94 1 Josip Almasi
95 76 Nate Lager
h3. Run the pre-built community maintained container image
96 77 Nate Lager
97 76 Nate Lager
Pull the container from Docker Hub.  https://hub.docker.com/r/gangrif/vrspace
98 74 Josip Almasi
99 47 Josip Almasi
h1. Getting started
100
101
So, you're new to VRSpace, and you want to make yourself a new virtual world.
102
You will not need to change any of the server code, but you need to get it from github, build it, and start it.
103
Once done, familiarize yourself with directory structure, and especially with template world, as this is intended stating point.
104
105
h2. Directory structure
106
107
h3. server
108
109
Contains the server code and configuration. Generally you don't need to change the code, but you may want to change the configuration. Main configuration file is in src/main/resources/application.properties, have a look at what you can change.
110 50 Josip Almasi
The server executable is a jar file in target subdirectory.
111 47 Josip Almasi
112
h3. babylon
113
114
This is reference Babylon.js client implementation. You don't really need any of it, but it is highly recommended to build on top of it, using javascript code provided there.
115
Html files there implement and/or test different functions available, like avatar loading and movement, video/audio streaming etc.
116
Most important one is avatar-selection.html, this is main entry point that allows user to select avatar and enter a world.
117
118
h3. web
119
120
This is the actual web site that's available online on vrspace.org, your final web site will probably look somewhat like this.
121
122
h3. content
123
124
Actual web content used to build virtual worlds, with self-explanatory directory names. Two of these contain special attention though:
125
126
h4. char
127
128
Characters, under female, male and other subdirectories. These directories are listed and presented by avatar-selection.html, you will probably to change how it works eventually.
129 48 Josip Almasi
But adding or removing characters is as simple as dropping them here.
130 47 Josip Almasi
131
h4. worlds
132
133 48 Josip Almasi
Available worlds, also listed by avatar-selection.html, and displayed as portals. Every subdirectory contains one world, and also note there's an image with the same name - that's thumbnail displayed on the portal.
134 49 Josip Almasi
Every world has some specific features.
135
Paris is huge, a real-life model.
136
Persian city and Aladin both feature the same procedural terrain and plants.
137 1 Josip Almasi
Cave is an actual game level, and utilizes floors built with vrspace.org floor editor.
138 50 Josip Almasi
Classrooom allows for sharing screen.
139
But the template is where you start - copy it over to another directory, and edit to suit your needs.
140
141
h2. Your first world
142
143 51 Josip Almasi
So with the server running, copy the template directory to your own. This world is immediately listed at http://localhost:8080/content/worlds/ and also available in http://localhost:8080/babylon/avatar-selection.html as a portal.
144 1 Josip Almasi
All you need to do to replace that dolphin with your own world is to replace this.file='dolphin.glb' in world.js with your own file name. Or really, delete the entire constructor, and save your world as scene.gltf in the same directory.
145 51 Josip Almasi
Hit reload button/key in your web browser, changes become visible immediately.
146 50 Josip Almasi
147
There are two files there, a html and a js, but html also contains some javascript.
148 52 Josip Almasi
The idea behind this separation of concerns is that world.js contains all programming logic that relates to the world itself, it should not depend on any server functionality. So build your world first, then add multi-user functions to it. Javascript in the world.html file shows some basic multiuser functions. As these grow, you'll probably want them in another javascript file, but these are minimum to get started.
149 50 Josip Almasi
150 51 Josip Almasi
Hopefully comments world.js and world.html explain enough to get started. As you need more functionality, look at other worlds and copy it over.
151 47 Josip Almasi
152 4 Josip Almasi
h1. Setting up development environment
153
154
Here's all you need to start development on Windows.
155 20 Josip Almasi
Linux distributions include all these tools as native packages, so no additional downloads should be required.
156 4 Josip Almasi
157
h2. Basic setup
158
159
h3. Git bash
160
161
IDEs can work with github directly, but whatever you ask, you'll get a command line answer.
162
Command line git is simply a must have. Bash also includes a lot of goodies like ssh.
163
164
https://git-scm.com/downloads
165
166
h3. Java
167
168 21 Josip Almasi
Java 8 will work, java 11 recommended. JDK is required to build the server.
169
Get it either from Oracle or elsewhere, e.g. Zulu OpenJDK: https://www.azul.com/downloads/zulu-community/?package=jdk
170 4 Josip Almasi
171
h3. Node.js
172
173
Node is used by IDE to evaluate javascript. You'll also may need it if you modify any of babylon.js source.
174
Mind that IDE will complain if you installed unsupported version of Node; should that happen, remove Node, and install latest one supported.
175
176
Get it from https://nodejs.org/
177
178 19 Josip Almasi
h3. Maven
179
180
Apache Maven is used to build the server from command prompt.
181
182
Get it from https://maven.apache.org/download.cgi
183
184 4 Josip Almasi
h3. IDE
185
186 79 Josip Almasi
Eclipse for Java Enterprise Java and Web Developers: https://www.eclipse.org/
187
Install Lombok plugin https://projectlombok.org/
188 1 Josip Almasi
189 78 Josip Almasi
And then go to Help -> Eclipse Marketplace
190 79 Josip Almasi
Search and install Spring Tools 4.
191 6 Josip Almasi
192 81 Jasna A
h2. Import and start the project
193 79 Josip Almasi
194 81 Jasna A
h3. git bash
195 1 Josip Almasi
196 81 Jasna A
git clone https://github.com/jalmasi/vrspace.git
197
cd vrspace
198
mvn clean install
199 82 Jasna A
cd server
200 81 Jasna A
java -jar target/server-0.4.5-SNAPSHOT.jar
201 5 Josip Almasi
202 22 Josip Almasi
h3. IDE
203
204 5 Josip Almasi
In Eclipse, you can use either default or new workspace for the project.
205
Assuming you have cloned the project from the github, 
206
Go to File -> Open Projects From the Filesystem
207
Then choose vrspace directory.
208
209
This will import vrspace folders and project subfolders, click Finish.
210
211
In vrspace project folder, in src/main/java, there's org.vrspace.server.ServerApplication.java.
212
Open it, then right click on the code.
213
From the menu, choose either Run as or Debug as -> Spring Boot App.
214
215 80 Josip Almasi
To run with java 17, you'll need to specify run configuration instead, with VM arguments: --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED --add-opens java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED
216 22 Josip Almasi
217
Open http://localhost:8080/babylon/connect.html with two browsers, and navigate around.
218
219
That's all, you're all set!
220
221 69 Nick Naglich
222 4 Josip Almasi
223
h2. Advanced setup
224
225 12 Josip Almasi
h3. SSL
226 4 Josip Almasi
227 12 Josip Almasi
HTTPS is required for pretty much everything - WebXR, camera, mic access.
228
By default, the server runs on 8080 port with plain HTTP. To enable HTTPS, edit application.properties (found in src/main/resources), and change following properties:
229 1 Josip Almasi
230 12 Josip Almasi
<pre>
231
server.ssl.enabled=false
232
# default port 8080
233
#server.port=8443
234 13 Josip Almasi
</pre>
235 12 Josip Almasi
236
h3. Apache
237
238
Apache reverse proxy setup, linux, windows, TBD
239 4 Josip Almasi
240
h3. Docker and OpenVidu
241
242
OpenVidu voice/video chat server runs as docker image. This is only required for development of voice chat functions.
243
244 71 Josip Almasi
Modify openvidu.publicurl and openvidu.secret in application.properties, or run server.jar with -Dopenvidu.publicurl=YOUR_URL and -Dopenvidu.secret=YOUR_SECRET
245
246
h4. Local execution (development)
247
248 1 Josip Almasi
docker run -p 4443:4443 --rm -e OPENVIDU_SECRET=YOUR_SECRET -e DOMAIN_OR_PUBLIC_IP=YOUR_IP openvidu/openvidu-server-kms:2.17.0
249
250 71 Josip Almasi
Enter a space. You may get
251
WebSocket connection to 'wss://YOUR_IP:4443/openvidu?sessionId=cave' failed: Error in connection establishment: net::ERR_CERT_AUTHORITY_INVALID
252
in the console. To get rid of it, open https://YOUR_IP:4443/ and accept the cert.
253
(using localhost as YOUR_IP may not work)
254
255
h4. Running on server
256
257 11 Josip Almasi
https://docs.openvidu.io/en/2.17.0/deployment/deploying-on-premises/
258
259 25 Josip Almasi
260
h1. Software Architecture
261 26 Josip Almasi
262
!https://redmine.vrspace.org/attachments/download/15/vrspace-diagram.png!
263
264
h2. Client-Server Communication
265
266 31 Josip Almasi
Clients communicate with server by sending JSON messages over "WebSockets":https://en.wikipedia.org/wiki/WebSocket. Reference javascript implementation of client communication layer is in "VRSpace.js":https://www.vrspace.org/docs/jsdoc/VRSpace.html.
267 26 Josip Almasi
268
General approach to communication is rather obscure Half-Object pattern: server-side and client-side object have same properties, but different implementations.
269
Whenever an object's property changes in (any) client's address space, it's transmitted to the server, that broadcasts it to all clients currently 'watching' the object.
270
271
Whenever a client wants to perform any change to any object in the space, it has to go through the VRSpace server.
272
Clients may or may not communicate directly, but this is out of the scope of VRSpace server.
273
274
h2. Server Responsibilities
275
276
Sole responsibility of VRSpace server is management of 3D space: persisting space objects, tracking their properties, processing and distributing events from/to objects.
277 27 Josip Almasi
Whenever we talk about objects, that includes clients, i.e. users - a client is a special case of an object.
278
The server does not even handle the authentication - it is assumed to be responsibility of web app serving the space.
279
3D geometry is also not in server's scope, it's just another property of an object (mesh).
280
281
h2. Server design
282
283
Key concepts here are Active Objects, Actor model, and Live distributed object.
284
285 53 Josip Almasi
"VRObject":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/VRObject.java is a basic shared object, with some basic properties like position and rotation, and of course, mesh. It essentially a "Business object":https://en.wikipedia.org/wiki/Business_object acting as a "Live distributed object":https://en.wikipedia.org/wiki/Live_distributed_object.
286 27 Josip Almasi
287 28 Josip Almasi
"Client":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/Client.java extends VRObject, and adds capabilities to communicate over web sockets, and listen to changes to other objects. Typically, a client represents a remote user, but it can also represent a robot connected over a web socket, or be a base class for a server-side robot.
288
A Client can be thought of as "Active Object":https://en.wikipedia.org/wiki/Active_object or an "Actor":https://en.wikipedia.org/wiki/Actor_model, 
289 27 Josip Almasi
290
Server relies on Spring Boot and embedded Tomcat to handle all I/O and threads.
291 29 Josip Almasi
292 55 Josip Almasi
Design does not require websockets in particular, in fact some reliable multicast protocol would surely fit better. Websockets work in web browsers.
293
294 33 Josip Almasi
h3. Client
295 29 Josip Almasi
296 30 Josip Almasi
Each "Client":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/obj/Client.java has it's own "Scene":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/Scene.java that tracks all shared objects - including other clients - visible by the client. The scene is initially populated once the client logs in to the server, and starts the session. Scene is refreshed periodically, after a movement, or explicitly.
297 29 Josip Almasi
298
Scene maintains the event model, by adding the Client as listener to all other active objects (usually other users) in the scene.
299 1 Josip Almasi
300 30 Josip Almasi
Client simply notifies it's own listeners on any changes to any of it's own properties. As a listener, it propagates any observed change on any other object over the network, to the user.
301
302
Client has just a couple of persistent properties, like position, rotation and name. The name must be unique.
303
All other properties are transient.
304
305 33 Josip Almasi
"ClientFactory":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/ClientFactory.java allows for customization of Client class and instances.
306 1 Josip Almasi
307 33 Josip Almasi
h3. Events and messages
308
309 30 Josip Almasi
Typically events exchanged are changes to properties of active objects, e.g. user moves around, changing own position and rotation. However, an object may emit any event, and the event gets propagated to all listeners.
310 1 Josip Almasi
This is to simplify client development: simply emit any custom event you want.
311 32 Josip Almasi
All distributed events are encapsulated in "VREvent":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/VREvent.java class.
312
313
However, there are other types of messages that are not distributed to other clients: commands and their return values, and errors.
314 1 Josip Almasi
Client will typically execute at least one command during the session - "Session":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/Session.java start. It will not receive any messages before that.
315
Should any server-side exception occur during the session, errors are sent to the client as simple JSON maps.
316 33 Josip Almasi
317
"WorldManager":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/WorldManager.java manages clients and event distribution between them.
318
319
h3. Worlds
320
321
A server can host many worlds, i.e. shared spaces. A world contains users and other objects that are shared only within the world. It can be thought of as a chatroom, but mind the difference: being in the same world does not mean that users can see and talk to each other.
322
Client can enter another world by issuing "Enter":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/dto/Enter.java command.
323
324 35 Josip Almasi
h3. Ownership and privacy
325
326
A class can be annotated as Owned, and all of instances automatically become owned as they are created. An owned object does not receive events from anyone by owner(s). A Client is owned, and nobody but himself can change his name.
327
On the other hand, a generic VRObject is public, and receives events from anyone. Like a door, anybody can open and close it. But even public objects can have some properties annotated as Owned, that can't be change by anyone but owner.
328
Fields of a class can be annotated as Private, and their contents will never be published.
329
330
"Dispatcher":https://github.com/jalmasi/vrspace/blob/master/server/src/main/java/org/vrspace/server/core/Dispatcher.java takes care of it before property changes are applied, and before they are published to listeners.
331
332 33 Josip Almasi
h3. Package structure
333
334
Package "org.vrspace.server":https://github.com/jalmasi/vrspace/tree/master/server/src/main/java/org/vrspace/server contains only main application class. Under it,
335
- config package contains server configuration classes that are executed only on server statup
336
- core is the core of the server
337 34 Josip Almasi
- dto is misnomer that stands Data Transfer Objects, though this package contains all objects that are not shared nor persisted
338 33 Josip Almasi
- obj contains persistent objects, most importantly Client and VRObject
339
- types are custom types used elsewhere, be it interfaces or annotations
340
- web is a primitive admin interface, disabled by default
341 54 Josip Almasi
- api contains REST controllers