Exploring Generative Fill in Photoshop: Enhancing Creativity with AI

I spent the day working with the brand new Generative Fill in Photoshop (actually the Photoshop Beta, which you can get here). I'll be recording a tutorial on it, but wanted to share my thoughts here. 

First, the good news! With generative AI, removing objects, extending your image, or reproducing missing parts of your image is going to be easier than ever. The bad news is that generative fill is not going to live up to people's expectations of "AI art." It is definitely NOT a replacement for creative talent or even Photoshop talent. But it is an upgrade to Photoshop as significant as the introduction of smart selections, possibly more so. And with the speed that AI is advancing, I think we'll see better integrations coming soon.

My thoughts below...

What it's really good at:

• Removing objects
• Extending images
• Replacing backgrounds (I'd put this in the "good, but not great" category)

What it's not good at:

• Generating things that don't exist
• Anything out of context
• Compositing

Things to keep in mind

Having used it on a wide variety of images and projects, these are some things you should keep in mind as your experiment with the tool yourself:

• The generative AI captures a snapshot of your layer stack before processing, regardless of its position in the stack. If you've been using Photoshop for a while and are accustomed to its intuitive behavior, you may need some time to adjust to this. For instance, if you generate a bird and later add a color grade curve adjustment to your layer stack, you'll need to disable it before generating more bird variants, even if the generative layer is positioned below the curve layer.
• The generative AI's primary task is to be seamless. While this limitation can be a drawback, it can also be leveraged to your advantage. It means that any elements created within the context of the image will consistently yield better results compared to adding elements that are out of context. Moreover, when it comes to creating visual incongruity, your artistic skills will always surpass those of AI.
• Intricate masking is better done after generation. Don't think of your initial selection as the bounds of your fill. Instead, think of it as the area you want to integrate into. Or better yet, think of what's outside of your selection as the areas you don't want affected.
• Generative AI will always have certain content restrictions that can limit creative freedom. For instance, nudity, which has been a recurring subject in art throughout history and remains a favorite among photographers, painters, and sculptors worldwide, is not allowed. Similarly, depictions of blood or guns are also prohibited. Many clients will need these, irrespective of Adobe's sensibilities. We'll see how this plays out in the actual release.
• When it comes to the primary subject of your artwork, the generative AI's results aren't suitable for professional projects. Unfortunately, there's a substantial gap between something that "looks good in a thumbnail" and something that "works in a professional marketing campaign." While the Generative Fill excels in the former, it falls short in the latter when it comes to adding new elements.
• Generative fill works best when used as an extremely smart version of content-aware fill. It does not work well in bringing new ideas or designs to life. To do that, you have to know how to use the rest of the tools in Photoshop.

The Future of AI in Photoshop

While Midjourney creates artwork that surpasses the skills of 95% of human artists, its purpose was never to facilitate artists. Instead, it serves as a tool to empower producers, art directors, and advertisers to efficiently create artwork without the need of an artist. Over the next few years, Midjourney and its competitors, including Adobe Firefly, are going to replace many human artists in various domains.

Photoshop, on the other hand, was never intended to replace human artists. Its initial purpose was to provide photographers with a digital platform for retouching and enhancing their photos. As time progressed, Photoshop evolved into a versatile tool for compositing, manipulating, and designing photo-based images. Generative Fill represents an exciting upgrade within this context.

Although it's being promoted as a "text prompt art generator built into Photoshop" it's actually a really smart content-aware fill. While it can do some text prompt art generation, it isn't good at it. And it won't give you something you can use for paid projects. It can be likened to the leap from the Magic Wand tool to the Select Subject tool. It significantly accelerates the process of filling in missing parts of an image, enhancing efficiency and even quality.

Looking ahead, I think the future of AI in Photoshop will come in a series of AI-enhanced upgrades like this one. There'll be AI-powered blending, skin smoothing, dodging and burning, lighting adjustments, shadow generation, and much more. Competing software like Luminar already demonstrates some of these capabilities, and Adobe's Firefly page hints at their ongoing development in these areas. And you even see them starting to appear in Neural Filters.

AI art generators have undoubtedly surpassed the artistic abilities of most individuals. However, they cannot replace the inherent creativity of human artists. The more I use AI generators, the more I see their reliance on context and their limited capacity to think outside the confines of the (admittedly vast) box in which they were created. 

So, for at least the foreseeable future, Photoshop isn’t going anywhere and Generative Fill will be an awesome new tool to help artists and photographers to bring their imagination, memories and stories to life.

(For the tutorial, subscribe to my channel below and you'll be the first to see it when it comes out.)