Reviving Einstein for Vives

Before we go into detail, you have to meet Flax, an AI system that has taken over the Vives experience space. In this particular space, Flax challenges visitors through four games to prove they can’t beat the AI system, and in fact, they need AI. After going through this journey and learning all about the potential of AI, you will be led to the demo center, where we took over the “AI & Education” booth. In this booth, visitors and students are able to meet Einstein’s digital twin.

The idea behind this AI-powered Einstein — also known as Ainstein — is that students should be able to experience AI first-hand. This set-up allows them to converse with a digital version of Einstein in multiple languages. What makes this digital twin stand out is its capability to respond to questions in real-time in Einstein’s characteristic manner. To make Einstein come to life, we combined these key technologies:

1. WhisperAI: To convert speech to text, capturing user speech in (any) natural language
2. GPT: To comprehend natural language and generate responses in Einstein’s persona
3. ElevenLabs: To convert GPT’s textual responses into audio files using Einstein’s voice clone
4. Audio2Face: To animate facial expressions and lip movements in sync with the generated audio
5. Metahuman: To create a hyperrealistic visualization of Albert Einstein, modeled in detail from online photographs, including custom hair styling
6. Unreal Engine: Renders the entire experience, providing a lifelike visual presentation

In addition to the experience, students can actually see the behind the scenes on another screen. On this screen, each step from the pipeline described above is displayed. First, they can see the speech-to-text transcription, then you see a prompt being generated for GPT, which a response is generated to, which is in turn sent to the voice cloning technology, and so on.

We invite you to step into the realm of AI with Flax, the House of AI. Immerse yourself in this unique experience and interact with one of history’s greatest minds, reborn through technology. Book your ticket here: https://www.theflax.be