or: How AI failed me, but Unity came shining through.

Coming from a art back ground, I get the odd commission here and there – it helps buy things for my game dev journey.

My mother-in-law came to me with a request: she had a dream about a cabin on the beach, it was very peaceful and relaxing; and she’d very much like it if I painted it for her.

Being eager to try AI art generation, I sat down with her and had her describe as much as she could in exacting detail. And we spent the better part of an afternoon feeding prompts into my local install of Automatic 1111 and seeing what it would spit out.

She ‘liked’ what she saw, but felt it was ‘too resort-like’ and ‘too plastic-y’; not keeping with the humble esthetic she’d dreamed about. After a few more tries with negative prompts, we had to call it quits to feed kiddos and take a break from her growing frustration with the process.

About a week later, she texted me- excited she had found a photo in the paper that was more true to the dream and brought it to the next family gathering:

It was small, and printed in that obnoxious half-tone process newspapers use, which made it low resolution. And very grainy.

So- hello Google Reverse image search. Found a better version and started Photoshopping things with a renewed energy. Found different images that she liked and started editing them together. Added filters and adjustment layers, tweaking things left and right.

And she still wasn’t thrilled with the result:

What it boiled down to was this: the cabin image was just too flat and two dimensional to really get the ‘look’ she was after. Since it was too flat – I felt I must make it 3D.

Taking a cue from Ian Hubert’s Lazy Tutorials – I took the photos I found and began modeling. The tut in reference is:

its only a minute and SO worth the watch!

So after some tinkering, I got the cabin in Unity and sat down with Ma to sculpt out the terrain, adding a few free assets, such as rocks, along the way. She was super impressed with how quickly we could iterate changes, move things around and tinker with the various settings to make it look ‘right’.

Once we got it looking the way she wanted, I started showing off, adding wind effects to the trees, I cribbed some seagull sounds off some video I filmed at the beach last year and looped some surf sounds as well. Once she liked the look of everything, I took a screenshot of it and said, “Ok, this is what I’ll use for the reference for your painting!”

Suffice to say – this was the most involved project I’ve ever attempted for a painting – usually its sitting the subject with some lights and taking a few dozen photos until I get the mood and feeling I’m looking for. Having tried 3 vastly different approaches, I’m thrilled that I have so many tools at my disposal in order to create. At the same time, tis also daunting, far better artists than I have been silenced by the sheer scope and volume of what they can create with…

Post Script:

 

So my initial attempt at painting it didn’t work out quite as planned. In my race to get the final underway, I grabbed a non-permanent ink and it bled like crazy as I was putting down my first washes.

I did go back and try to mitigate some of the bleeding – but it looks like I’ll have to take another run at it.

Just let me go work on my game for a bit, first.