Google originally unveiled its generative 3D AI system called Dream Fields in 2021, and now a new and improved version has arrived.
Google’s new next-generation artificial intelligence software designed to convert text into 3D generated images is called DreamFusion. So, how does this work? In a new proof-of-concept paper published to the pre-print server arXiv, researchers outlined that Dream Fusion, much like Dream Fields, uses a neural network called Neural Radiance Field (NeRF) that is designed to general novel views of complex 3D scenes using 2D datasets.
However, DreamFusion has taken a different approach than Dream Fields. Google research scientist Ben Poole wrote on Twitter that the team replaced OpenAI’s CLIP technology that powered Dream Fields with Google’s AI model, Imagen. The 3D models seen above and below aren’t as photo-realistic as what we’ve seen with Mid journey. However, they are certainly still impressive as the 3D models have accurate surface geometry, depth and are even relightable for various lighting conditions.
Furthermore, Poole explains on Twitter that several 3D models generated by DreamFusion can even be thrown into a single scene.
