Users will also get the option to have the latest content from the web incorporated into Yasa-1’s answers.
Reka, the AI startup founded by researchers from DeepMind, Google, Baidu and Meta, has announced Yasa-1, a multimodal AI assistant that goes beyond text to understand images, short videos and audio snippets.
Available in private preview, Yasa-1 can be customised on private datasets of any modality, allowing enterprises to build new experiences for a myriad of use cases. The assistant supports 20 different languages and also brings the ability to provide answers with context from the internet, process long context documents and execute code.
It comes as the direct competitor of OpenAI’s ChatGPT, which recently got its own multimodal upgrade with support for visual and audio prompts.
“I’m proud of what the team has achieved, going from an empty canvas to an actual full-fledged product in under 6 months,” Yi Tay, the Chief Scientist and Co-founder of the company, wrote on X (formerly Twitter).
This, Reka said, included everything, right from pretraining the base models and aligning for multimodality to optimising the training and serving infrastructure and setting up an internal evaluation framework.
However, the company also emphasised that the assistant is still very new and has some limitations – which will be ironed out over the coming months.
Available via APIs and as docker containers for on-premise or VPC deployment, Yasa-1 leverages a single unified model trained by Reka to deliver multimodal understanding, where it understands not only words and phrases but also images, audio and short video clips.
This capability allows users to combine traditional text-based prompts with multimedia files to get more specific answers.
Yasa-1 also brings additional features such as support for 20 different languages, long context document processing and the ability to actively execute code (exclusive to on-premise deployments) to perform arithmetic operations, analyse spreadsheets or create visualisations for specific data points.
In the coming weeks, Reka plans to give more enterprises access to Yasa-1 and work towards improving the capabilities of the assistant while ironing out its limitations.
