Nvidia’s Latest AI Technology Transforms Words Into Realistic Images

In context: Nvidia’s GauGAN technology has already demonstrated what it is capable of, turning simple sketches into photorealistic images. We’ve seen it used in Nvidia Canvas since then, but it seems the GPU giant is aiming for higher ground with its AI, releasing a new version capable of turning words into images.

Nvidia presented its GauGAN technology for the first time in 2019, but only recently did we see it featured in a product available to the general public. Called Canvas, this piece of software can be a lot of fun to use as it allows users to create amazing photo-like images with basic sketches.

It has been a few months since the Canvas announcement, but work on GauGAN has continued and is now reaching version 2.0. The technology has become even more impressive, as now able to convert words in photorealistic images, providing a result similar to that obtained when using the draw to image function.

As seen in the video above, type something in the text box and an image will be generated immediately based on your words. Add an adjective or replace a noun in the sentence and the image will change accordingly.

Converting words to images has never seemed so easy

For further customization, users can combine text and picture-to-picture functions. Using written words to generate the base and drawings to detail the image, you can change the shape, size, and textures of any object within the image.

To achieve these results, Nvidia’s GauGAN 2 text-to-image feature uses a network-based adversary-generating artificial intelligence model that “combines segmentation mapping, painting, and text-to-image generation.” This model was trained using 10 million landscape images, so it should be well prepared for whatever you throw at it (or not).

You can give it a try using Nvidia’s interactive AI demo for GauGAN 2 using your web browser. To play with it, you will first need to agree to Nvidia’s terms and conditions (check the box at the bottom of the page).