Project

General

Profile

Feature #86

universal text input

Added by Josip Almasi over 2 years ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Start date:
11/05/2021
Due date:
% Done:

0%

Estimated time:

Description

PC, VR and mobiles are drastically different devices.
HTML text input element is useful on PC, less so but still useful on mobiles, and does not exist in VR.
Virtual keyboard in VR is barely useful.
Chrome has some speech recognition API.

Build a component that will use any one of these, and provide callback that can be used elsewhere.
Switch input upon entering/existing VR, but also provide function to control over input selection.

History

#1

Updated by Josip Almasi 12 months ago

  • Status changed from New to In Progress
  • Assignee set to Josip Almasi
#2

Updated by Josip Almasi 12 months ago

  • Status changed from In Progress to Resolved

Form class with utility methods, including text input and voice recognition. Extended on SearchForm in world-editor.js.
Virtual keyboard is tricky: requires an additional AdvancedDynamicTexture, or significant resizing on the same texture used for text input. Enabled on SearchForm in XR only.
Voice recognition works in chrome and edge, but works different in edge - microsoft adds punctuation :) Also works on mobile android, but not in oculus browser.
Merged to master.

#3

Updated by Josip Almasi 12 months ago

Known issues with android speech recognition:

1) it just keeps restarting speech recognition, which causes endless beeping every two seconds:

https://bugs.chromium.org/p/chromium/issues/detail?id=679010&can=2&start=0&num=100&q=&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&groupby=&sort=
https://github.com/TalAter/annyang/issues/194

Workaround implemented as starting speech recognition on touch, and restarting it until it ends with silence.

2) WebRTC streaming disables speech recognition: https://bugs.chromium.org/p/chromium/issues/detail?id=427567

Stopping the publisher does not help, it just doesn't work.

So speech recognition on Android can't be used in vrspace.

#4

Updated by Josip Almasi 11 months ago

... but gamepad turns out to be useful for mobile VR. HUD UI integrated with gamepad, including text input.

#5

Updated by Josip Almasi 7 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF