Virtual Reality with Artificial Intelligence-Powered Non-player Characters

The creator of Modbox has combined Windows speech recognition technology, OpenAI's GPT-3 AI and Replica's natural speech synthesis to create a unique demonstration: one of the world's first artificially intelligent virtual characters.

Modbox is a sandbox game where players can create their own multiplayer experiences, and it is supported by SteamVR. The game was officially launched in late 2020 after a few years of public beta testing, although it is still marked as an Early Access title. We first tried it with the HTC Vive back in 2016.

The developer recently tried out two of the most advanced machine learning tools: OpenAI's GPT-3 language model and Replica's natural speech synthesis. The results are truly astonishing - go to around 4 minutes and 25 seconds to see the conversations with two virtual characters.

Microsoft, which has invested $1 billion in OpenAI, holds exclusive rights to the source code and commercial use of GPT-3, so it is unlikely that this feature will be included in Modbox itself. However, this demo video gives us a good glimpse of what the future of interactive characters could look like. Future language models could greatly affect game design and make entirely new genres possible.

There is a rather long time lag between when a question is asked and when a response is received because GPT-3 and Replica are both cloud-based services. Future models that run on-device could potentially eliminate this lag. Google and Amazon already use custom chips in some of their smart home devices to reduce the response time for digital assistants.

Source: https://uploadvr.com/modbox-gpt3-ai-npc-demo/