I ask an infamous AI chatbot
Q1: briefly in 100 words how diffusion based art generators work ?
Diffusion-based art generators use mathematical algorithms to simulate the diffusion of particles in a two-dimensional space. These particles are represented by pixels or other visual elements, and their movement is controlled by parameters such as velocity and density. As the particles move, they create patterns and textures that evolve over time, resulting in unique and complex visual compositions. The parameters can be adjusted to achieve different effects, such as the speed and direction of diffusion, the size and shape of the particles, and the color palette. The result is a form of generative art that produces a wide range of organic and dynamic images.
All well and good but what does it all mean? If you didn’t study this how it works before, now is your chance. You’ll read that each source image is progressively degraded back to white noise before being brought back. At each degrading step the surviving features are registered in terms of these parameters of velocity and density. With two quite similar images the similar features will be tracking each other. When recovering and/or generating emerging features will follow pathways similar to those already encountered
In the wide world you can see diffusion at work in a latte coffee as the dark espresso floated on top of the milk spreads into the pale white below.
Let us get a definition of diffusion in this context from our chatBot
Q2: in the context of a diffusion based art generator please give a short explanation of diffusion
Diffusion is a physical process that describes the movement of particles from an area of high concentration to an area of low concentration. In the context of a diffusion-based art generator, this process is simulated using mathematical algorithms to create visual patterns and textures.
so what’s to do ?
technology doesn’t go away and if you owe your living to painting the threat is greatest.
the systems at the moment seem strongest on synthesising stroke, gesture and colour
so make these your USP – unique selling point. In volume terms, square inches of canvas, people buy art terms of the domestic enjoyment of filled square feet of wall space. You cannot blame them for buying a pleasing image, screen printed onto canvas and neatly framed, which has a “impressionist” look about it.
Meet your purchasers and get them to like you as a person. Pictures are not the easy equivalent of a 1000 words; your story in probably well over one thousand words has to be the narrative of their purchase decision.
Sorry to be so negative but that is the arc of capitalism in this age
if I repeat that last command and enter another well known artist’s name — I tried Munch, Chirico, van Gogh — results came highly reminiscent of other well known works by these artists
this is to be expected — an image by Renoir is pulled apart and increasing amounts of noise are inserted while how the image maintains or loses areas of internal consistency which track colour palette choices, brushwork, and composition. how the generator works
When the user then cites Renoir as a target style for the output these consistencies can be used to steer the formation of an image from a match between the user’s input text providing a starting image towards an image with a number of features drawn from images by Renoir.
Let us try this out.
I am a great believer in free and open source software [FOSS] I get to use well developed and exciting products, with great user group support and end up making donations to their costs because I find the gain in productivity is worth it.
I am going to use the Stable Diffusion offering Stable Diffusion, get started for free and I want to start with a short poem by Ezra Pound
The apparition of these faces in the crowd: Petals, on a wet, black bough
In a Station of the Metro. 1913
When I put this text into Stable Diffusion, a few seconds go by and then 4 images are offered, here is one of them
boring even. time to try and understand how to drive this stuff and make it work stable diffusion syntax explained
so i guess that I’ve been too ambitious for in asking for an interpretation of a minimalist poem
I cut the scope of the problem to “wet night, crowded station”
still quite photographic,
try “wet night, crowded station; oil on canvas” and even worse, “wet night, crowded station; oil on canvas;renoir”
confession time: I have worked in IT, programming mostly, and therefore my interaction with computers falls naturally into the verbal. I see computer systems as complexes that have to be approached in a specific manner to gain the results that I want.
I found this explanation of the way in which these systems work how art generators work and though it is a technical read it made plain to me one key fact
images found on the web are analysed in terms of their affect and the text that has become associated with that affect.
so this matches the intention of an artist to generate affect — that is the feelings in the viewer, far beyond the reach of the original artist. The user of one of these systems is striving to achieve affect in themselves by invoking words — in the terminology, prompts — that in the moment of the user’s interaction with the system result in an image that moves towards meeting the user’s intention.
thus just as a painter considers the paint on the palette with the intention of generating a result / affect so does the user of a system like Midjourney consider whether the prompt they enter moves toward the desired affect or needs to be undone.
