News about the DAZ Studio and Poser software

Category Archives: Poser to Stable Diffusion

Perfecting 1:1 watercolour in Flux Kontext – getting better watercolour with a LoRA

Posted on by Posted in Companion software, Natural media emulation, Poser, Poser to Stable Diffusion 1 Comment

Further to my Flux Kontext Dev experiments of the last few days, for filtering Poser renders… here I show how to get better watercolour by using a LoRA. Add the Aurelle v2 LoRA at 0.8. Specifically, this is designed to give an imperfect ‘human-made watercolour’ look for Flux. No garish day-glo colour (as with the default Flux Kontext), and no colour instructions or later Photoshop adjustments needed. And it’s much more subtle. It does want to swish down the hat-brim, and the 1:1 registration is lost there. But that’s easy to fix and otherwise it’s great. We also get rid of the dark 3D shadow under the hat.

The 1024px is slightly fuzzy, fuzzier than the default watercolour output. Possibly that’s because I’m using a regular Flux LoRA, not a Flux Kontext LoRA. But even so, a Firefly line-art render could be layered and blended in Photoshop, to bring back a little harder definition of the shapes and on the edges. That’s not been done here, on this quick demo.

I was however using the real-time character render from Poser that was .PNG and masked with transparency, which seems better than one with a white background. Another test showed 2048px gives clearer and larger results with more detail (e.g. fingernails), but takes far longer and is not so watercoloury. Try working in 1024px to test ideas, with 2x upscale. Then move to pure 2048px for a second pass, which is later blended back into the first in Photoshop?

The ComfyUI workflow shown is the official GGUF demo, adapted for a LoRA and with elements moved around. Note that FluxGuidance (CFG in Stable Diffusion -speak) is at 1.0 rather than 2.5, and apparently this favours a traditional artwork style.

Flux Kontext

Posted on by Posted in Comics, Companion software, Poser to Stable Diffusion, Python scripts, Spotted in the News Leave a comment

I’m downloading the free Flux Context and its various dependencies now, and it looks like it will void all previous local Stable Diffusion attempts at ‘style makeover while keeping the content fixed’. In other words, ‘work like a Photoshop filter’.

But, a super-powered one that also knows how to make precise image edits (e.g ‘work like a Photoshop filter, but add a hat and change nothing else’). It’ll thus be very interesting to see what this new form of local AI can do with some basic real-time Poser renders.

Oh, and I also have a ComfyUI node which outputs a workflow as a Python script, which would seem to offer potential to have Poser grab a just-made render and feed it straight into Flux Kontext. The drawback on that is the render-time (1min+ for Kontext, per 1024px image), which is not so turbo-charged. So, near-instant basic Stable Diffusion style-change methods will likely still have their place.

Watch this space.

Script completed

Posted on by Posted in Automation, Companion software, Poser, Poser to Stable Diffusion Leave a comment

I’ve now completed an automated Python script for the bulk of the Poser to Stable Diffusion workflow…

The one-click automated output is the .PNG files, plus the same .PNGs automatically stacked into a Photoshop .PSD file and saved.

For later work in Photoshop: Firefly AO (Ambient Occlusion pass, adds subtle shadows if needed), a real-time Preview faux ‘clown pass’ (aka ToonID) (easy masking, though the Meshbox HP Lovecraft figure doesn’t have much to mask here), and a Preview colour render (for blending back colours, so they’re consistent from panel-to-panel in a comic, and for dropping in a backdrop). Their folder is datestamped and also has the name of the Poser scene file. Then the Poser scene is reverted to where it started, safe and sound.

The Comic-Book Preview and the Firefly lineart layers are then merged in Photoshop, with an Action, and the result is then dropped to the desktop and from there manually dragged over to be used as the Img2Img source in SD.

The SD result then gets saved and manually opened in Photoshop, and an automated Action takes over and restores the Poser Preview render colours. After that it’s optional to add a holding line, mask areas, add very subtle 3D shading of the ligne claire (clear line) comic style, or cut out the plain backdrop and add a new one.

Had to go back to Poser 11 for this script, because Poser 13 doesn’t appear to support options.BucketSize(256) in Python. Without having Firefly set a big bucket size, the three passes of Firefly rendering are rather slow.

Source DPI, SD controlnet, and the ‘jaggies’

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion Leave a comment

More on my devising of a Poser to SD workflow. I was happy with the look I had got. But I took time to look into the ‘jaggies’ in my Poser to Stable Diffusion success. The relatively small 786px input .PNG, used for both Img2Img and the Controlnet was thought to be the problem. The output is too jaggy on many lines, despite Poser’s real-time Comic Book Preview render being anti-aliased. How to fix it?

First I ran the source through Vector Magic at medium detail level, which includes smoothing along with the vectorisation.

Much better linework from SD with this as the source, and the dotted suit-lapels vanish. Though it’s lost definition on the suit buttons, and there’s a weird long fingernail. Also the sleeve is a little rough. There are other glitches elsewhere.

Though the edges – such as that sleeve – are not a problem — just add a centered ‘holding-line’ outline stroke in Photoshop and the bobbliness is covered up. But things changing shape and glitching is not good for replacing the colour, via a full Poser Preview render used as a blending-layer in Photoshop. So vectorisation of 768px is perhaps not the way to go. Though it did at least seem to confirm that it’s not some inherent limitation in the Img2Img or Controlnet process that causes the problem.

Nor was the free anti-aliasing filter for Photoshop, Japan’s OLM Smoother, found to do anything. It’s quite subtle and there’s very little difference in the end result.

What about the G’MIC Repair / Smooth options? Nope. Though there is a way to use a G’MIC custom filter on the SD output, to get an anti-aliased ‘inked’ layer that might overlay the SD output lines, in Photoshop. Nice, but it’s not consistent across images and moves things further way from replicability across different images on a comic-book page.

A custom variant of FXEngrave.

In the end, after further tests, I appear to have discovered that… source DPI affects style in Stable Diffusion 1.5. Who knew? 72dpi source and Img2Img = roughly drawn comic book (seen at the start of this post). Exactly the same settings with a 300dpi source give a far more refined and also smoother-lined style. I guess the added DPI gives SD even more ‘wiggle room’ to add the style, in an Img2Img process that moves the source from 768px to 1024px?

Next step, then, is to experiment with 120dpi and 150dpi to find a balance point. But even before that, a Python script to automate the testing and save time/gruntwork.

Final Lovecraft

Posted on by Posted in Comics, Companion software, Poser, Poser to Stable Diffusion Leave a comment

I think I’ve hit the final balance in my Poser to Stable Diffusion workflow. The H.P. Lovecraft head, formerly too shaded and photoreal looking, now better matches the rough drawn style of the suit and body. Without getting too sketchy or losing the likeness. The lovely expressive eyes are kept. The result is 1024px and replicable.

Lovecraft hails a New York taxi-cab, or something…

As before, I should state that this was a deliberately bad pose and hands, to test the styling capabilities of SD 1.5 in Img2Img. Thumbs are obviously a problem at this level of denoising, and would ideally be artfully hidden.

The result can have colour consistency from panel to panel. It can be greyscaled, overlaid in Photoshop with a colour Poser render, and then colour-blended. The suit will thus darken down, so the crease-lines and dotted-line lapels won’t be so obvious.

Details, details…

Posted on by Posted in Comics, Companion software, Poser, Poser to Stable Diffusion Leave a comment

More progress with my experiments with style changing from a basic Poser real-time render. Up the CFG to 9, up the Embedding strength to 1.2, up the Img2Img Denoising to 7. Keep Canny at 87%. Specify dotted_line in negative, and change Steps to 42, with together remove some of the granulation of the lines that would otherwise occur.

Apart from the thumbs and a stray line below the buttons, the layer registration with the original is exact. Meaning easy re-colouring in Photoshop, via a normal Poser render and the ‘Color’ layer blending-mode.

The face is now too chiselled, and a H.P. Lovecraft character LoRA may be called for. But worth the extra work, since the eyes now especially benefit. There is a slight painterly smudge on the white, which you may be able to see here on a good monitor. But the original outline should be able to be selected in Photoshop. If not, one could use a mask from Poser. Or just do the scene with a backdrop straight from Poser.

The subtle 3D shading being auto-added could interfere with any light-direction shading you wanted to add later (perhaps by dropping a Smooth Shaded Poser render into the layer stack, in Photoshop).

More success with a Poser to Moebius look

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion Leave a comment

I’ve finally been able to get back to my Poser to Stable Diffusion 1.5 experiments. More immediate success, this time by starting with a Brian Habelin style ‘darklight’ lit Preview render (very low single light, Comic Book lineart in b&w, textures showing through just a bit) and then a Firefly lineart-only all-lines render. Combine these two in Photoshop, and you deal with the dark suit and the lack of definition of the suit detailing, at the same time. Nice. Img2Img as before, with the Poser render also dropped in the Canny Controlnet.

Deliberately awkward M4 pose and hands on the Meshbox Lovecraft figure, and rendered at just 768px, to see how SD handles such awkwardness.

The 1024px result stays nice and light, and doesn’t immediately start veering into the grungy ‘black on black’ territory which bedevils 3D work. And there’s no need to lift the gamma or adjust textures in Poser. The contrast is easily adjusted later in Photoshop. There’s also more of a Eurocomics ‘ligne claire’ (clear line) look than my previous highly detailed Moebius attempt. I like this simpler look. Interestingly, SD is colourising without being asked to do so.

Quite a high CFG for this, so SD 1.5 had more room to wiggle. Which means both the thumbs have gone wonky, and the eyes have been adjusted to look away rather than up. However, everything else is locked and we still have the Photoshop-accurate layer registration otherwise. So one could colourise the result by layering a Poser render on top in Photoshop, and simply setting the layer blending mode to ‘Color’. This would give you a base of consistent colours from panel-to-panel in a comic-book page.

Successful test – the Moebius variant

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion Leave a comment

More testing of Poser to Stable Diffusion 1.5. This time the aim was 1024px output with a Moebius style that wasn’t his later super-slick poster/art-print style and was more like his old Heavy Metal comics linework and dash-shading. Combining the WASMoebius embedding with a LORA, and juggling the various settings for a few hours got me a repeatable workflow — not perfect and a bit chunky (i.e. the main lines are too thick, the dash-and-dot shading is too thin), but it’s as close as I can get while keeping it as Poser-render compatible as possible (for later masking and re-colouring).

Auto-brightness and reduce from 1024px to 768px…

Still not ideal, rather too ‘busy’, but it’s moving towards a Moebius look.

$20 on the Wishlist

Posted on by Posted in Poser, Poser to Stable Diffusion Leave a comment

My Renderosity Store Credit finally nudged above $20. Hurrah. This is the nice haul for Poser that I got with it. A Sixus1 ‘Beorn’ fantasy bear, old but AI can fix that. Caudipteryx, a prehistoric proto-bird, just because it was dead cheap and I didn’t have it. Again, old but AI can fix it. The Mari Lwyd, an unusual and archaic Welsh folk figure with M4 support. Natures Throne, a tree-throne that looks great. And AJ’s Pharaohs Boat, because I want to eventually illustrate an Ancient Egyptian myth when I retire, and it may not still be available by then. The boat also has DAZ Studio MATs.

The $1.58 Caudipteryx. As usual, the prehistoric Poser 6 renders don’t do the product any favours in the Store. Seen here, instead, installed and with a quick Superfly render in Poser 13…

Thankfully, they mostly ate plants…

Still not great, as the textures were for Firefly not Superfly. No short fur on parts of the body (as the real creature would have had), but that could be added in the Poser Hair Room. No space-suits for it either, so you can’t call it a Space Turkey alien.

But here is the Superfly render ‘AI enhanced’ in Stable Diffusion 1.5…

Successful test – adding LORAs

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion, Tutorials Leave a comment

Adding a LORA to the recent Poser to SD 1.5 workflow.

Vintagecomic at 0.63 strength.

Crab Grass at 0.35 strength.

Moebiuscolor at 0.55 strength

All managed to keep the registration with the original render, in terms of re-coloring via Photoshop’s layer blending.

There are many more LORAs to experiment with. But these three suggest the workflow is robust enough not to be thrown too far out by adding a LORA.

Successful test – restore the Poser colour

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion, Tutorials Leave a comment

Another Poser to SD experiment.

1. Take the final from the Poser to SD 1.5 workflow shown earlier…

2. Give it a simple “desaturate” in Photoshop (better b&w conversions are available).

3. Have Photoshop enlarge the crude original source for the Img2Img input, to 1024px. Then paste it over the top of the 1024px final.

4. Set the Poser render’s Photoshop layer blending mode to ‘Colour’…

The colours of the original Poser render are thus restored, which across a comic-book page and pages will give you the Holy Grail of consistent colouring across different SD images. You’re welcome.

5. Flatten layers, tweak Curves and Brightness in Photoshop, to add more graphic ‘pop’ and/or suit the intended lighting for your story.

Obviously the Poser source we used here is a bit crude…

… and you might get better making another finer render to be used only for Colour blending in Photoshop. If sticking with the real-time Preview render, note you’ll get better colouring by rendering Preview using more than the default 512px material textures.

Don’t forget you might also lay on a 30% blending layer with a Smooth Shaded render from Poser, made under a suitable light, which will add back some shadow and volume (for those hankering desperately for ‘the 3D look’). Then there’s Poser’s Sketch renders too, to play with.

Successful test – the final

Posted on by Posted in Comics, Poser, Poser to Stable Diffusion, Tutorials 1 Comment

A follow-on to my earlier tutorials, using two Poser renders and an SD 1.5 model (in InvokeAI) as if it were a Photoshop filter — keeping everything in the Poser renders stable, but having SD change the style to something that regular comics readers wouldn’t laugh and jeer at.

Here the background stays stable because the 786x Firefly lineart-only render, being used in the Controlnet at 87%, now includes the background as well. This stops the background from getting ‘SD gloopy’. The figure outline can be masked in Photoshop, because it stays the same as a figure-only Poser render.

Quite a nice strong graphic style, I think, that would be suited to a four or five-panel comic-book page. Obviously you’d work it over a bit with the dodge and burn tools in Photoshop, and re-ink some bits. And you’d power up the lens choice, camera angles, figure expressions and suchlike. Maybe also experiment with lighting in Poser, since you can’t prompt for it in SD when used like this.

The SD 1.5 model Photon is meant for photography, but it’s an excellent early model that isn’t polluted by manga/anime and does what you tell it to. Unlike ADAM it doesn’t get in the way too much, when you push it towards a graphic illustration. The WASMoebius embedding is not really needed, strangely, but I left it in anyway as some may want to experiment with pushing it further. The problem with doing that, though, is that you’d lose panel-to-panel consistency in the comic, which is the whole point of this workflow.

Enjoy.

Successful test – the background

Posted on by Posted in Comics, Companion software, Poser, Poser to Stable Diffusion, Tutorials 1 Comment

Definitely getting there, in the quest to use Stable Diffusion 1.5 like a Photoshop filter. This is another follow-on from my two previous tutorials for Poser with SD.

The final result

To obtain this final result, for the Img2Img source I started again with exactly the same Poser scene, camera and light as before, but this time I dropped AS’s Hanyma Platform for Poser (now no longer sold) to the scene as a background prop. Poser’s Comic Book inking applied to both.

Raw scene in the Poser UI.

Then I made a quick Preview render of this scene at 768 (remembering to boost the texture sizes from 512), and then in Photoshop used the Glitterato plugin to add a quick starry sky.

In InvokeAI this replaced the previous figure-only Img2Img image, but the Firefly lineart Controlnet image stayed the same, thus giving a fixed figure outline that matches that of the Poser render — possibly important for later consistent colouring.

In this experiment I also added a Moebius LORA, which knows what our hero Lovecraft looks like. No additional prompting is needed to account for the backdrop, since the CFG is so low.

The final result at 1024px

It’s all going a bit ‘black on black’ (arrgh!) for this quick demo, but a non-background SD generation of the figure alone can then be used in Photoshop to mask (Crtl + click on layer, invert) and then fade or lighten the background a touch (as I believe Brian Haberlin does) so as to make the character stand out a little more. And ideally your comic script would try to avoid ‘black cat in a coal cellar’ settings, for this reason.

Simply using the new Img2Img as the Controlnet, replacing the Firefly outlines render? Nope, that doesn’t keep the detailing on the character or keep him consistent across a quad of generations. The Canny Controlnet needs to be focused just on the character, in the same way the comic reader’s eye is. Going from 768px to 1024x in the Img2Img seems to give SD some creative wiggle-room, despite the low CFG. Since it’s a low CFG for the Img2Img, there’s not much shifting in the details of the background. This seems to be the sweet spot: good model, Img2Img with a low CFG but slight upscale, and use the Controlnet with pure Poser Firefly lineart to keep the figure stable and in lockstep with your Poser renders. Presumably all this would also work for two characters interacting.

By having both character and backdrop generated by SD, there’s some SD gloop and later a loss of flexibility when putting the frame together in Photoshop. But one also avoids the need to mask, extract, defringe, colour-balance etc. It may be possible to prompt for lighting and get it, but I haven’t tried that yet.

Obviously for a comic you’d also start breaking free of the stock camera lens and use foreshortening etc for a more dynamic look. Poser has a special camera dial that makes that very easy.

Successful test – the ‘proof of workflow’

Posted on by Posted in Comics, Companion software, Poser, Poser to Stable Diffusion, Tutorials 2 Comments

Here’s a follow on from last night’s Successful test – Poser to Stable Diffusion enhancement, a proof-of-concept (or perhaps more accurately proof-of-workflow for the compositing. It’s crude, and he’s meant to be on an airship which isn’t shown, but it shows it can all work.

1. Rerender the Poser Firefly line-art at 4 x the original 768px (i.e. 3027px square) as a .PNG file.

2. In Photoshop I then run this render through GMIC and a custom filter for line-art (very similar to Dynamic AutoPainter’s Comic filter, but free). This turns the thin lines into chunkier ‘inked’ lines. Takes about 40 seconds, but does the job.

3. Size the GMIC result back to 1024px and place it over our final 1024px outcome from the successful test. Set the new layer to Multiply blending mode, and adjust opacity to suit. Flatten, and then select and cut out the figure from the white background. Defringe. Then have Photoshop add a thick ‘holding-line’ around the figure’s outline (Stroke 5px, black). This latter item helps subtly isolate the figure from the background it’s going to be pasted onto.

4. Reselect the resulting cutout figure and drop over a suitable backdrop. Here for speed I’ve merely selected a SD landscape experiment and given it a very crude tooning effect via a Photoshop filter — a final frame would have a much better lineart background. But it serves for now. Make a white layer behind, and fade the background a little by simply opacity-blending it with the white. This is why, ideally, you want figures and backgrounds as separate elements.

5. Colour balance the figure with the background. Flatten layers, tweak the Curves in Photoshop to add contrast and ‘pop’ (without things getting all ‘black on black’). Slot into the page-layout’s frame and add a text box. You’re done.

The demo is very basic and crude, especially the pose and background. But it proves that one can go from a Poser render to a finished comics panel and have it look somewhat acceptable for storytelling purposes. Especially considering the start point was this…

The workflow may sound fiddly, but once nailed down a Poser Python script could handle the renders in one click. Then a set of custom Photoshop Actions would handle much of the rest. Regrettably Stable Diffusion software, despite being built on Python, omitted to add Python scripting for UI and rendering automation.