Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved Negative Prompt Support for Anime, Paintings and Other Non-Photographic Media #2481 #2492

Merged
merged 7 commits into from
Mar 15, 2024

Conversation

DavidDragonsage
Copy link
Contributor

Reference: Improved Negative Prompt Support for Anime, Paintings and Other Non-Photographic Media #2481

INTRODUCTION:

Currently the Anime preset uses Fooocus Negative in its default_styles list. This is an unfortunate choice in that Fooocus Negative suppresses "anime", as well as "drawing, painting, crayon, sketch, graphite, impressionist, noisy... cartoon, graphic, text, painting, crayon, graphite, abstract" (yes, "painting" is listed twice).

Fooocus Enhance is a somewhat better choice for non-photographic negative prompting in that it does not suppress "painting", but it does have "(airbrushed, cartoon, anime, semi-realistic, cgi, render, blender, digital art, manga, amateur:1.3), (3D ,3D Game, 3D Game Scene, 3D Character:1.1)" in its no-fly zone.

Apart from that, Fooocus Enhance contains a lot of suppression that is really valuable for semi-realistic art, such as "bad anatomy", "watermark", "cropped", etc.

My proposal is to have a version of the Fooocus Enhance style in which the words "airbrushed" through to "3D Character" are removed. This new style could be called "Fooocus Semi Realistic".

PROCEDURE:

Modify or add the following components, as follows:

  1. In sdxl_styles_fooocus.json, insert this text at Line 6:

    {
    "name": "Fooocus Semi Realistic",
    "negative_prompt": "(worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, grayscale, bw, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)"
    },

  2. In anime.json, at Line 36,

replace "Fooocus Negative" with "Fooocus Semi Realistic"

  1. To create the sample image

a) Use run_anime.bat to start Fooocus with the anime preset
b) Select 1024x1024 for a square aspect ratio
c) Deselect all styles except "Fooocus Semi Realistic"
d) Enter "cat" as the positive prompt
e) Generate
f) Select an appropriate image and use IrfranView to resize to 128x128
g) Save as fooocus_semi_realistic.jpg

TESTING:

  1. Start Fooocus with the anime preset. The Style selections correctly display Fooocus V2, Fooocus Semi Realistic and Fooocus Masterpiece as the defaults. See "fooocus style selector - Semi Realistic.png." This image capture was taken after the creation of fooocus_semi_realistic.jpg and correctly displays the popup for the new style.

  2. After deselecting Fooocus V2 and Fooocus Masterpiece, "Semi Realistic Log Data.png" correctly shows the Fooocus Semi Realistic style selection, and "Semi Realistic Metadata.png" shows that the Fooocus Semi Realistic was correctly used as the full_negative prompt.

ATTACHMENTS:

"Improved Support for Anime & Paintings.zip" contains the amended sdxl_styles_fooocus.json, anime.json, and fooocus_semi_realistic.jpg, as well as supporting documentation.

Improved Support for Anime & Paintings.zip

In anime.json, at Line 36,
replace "Fooocus Negative" with "Fooocus Semi Realistic"
In sdxl_styles_fooocus.json, insert this text at Line 6:

    {
        "name": "Fooocus Semi Realistic",
        "negative_prompt": "(worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, grayscale, bw, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)"
    },
Popup image for the new "Fooocus Semi Realistic" style
@DavidDragonsage
Copy link
Contributor Author

Its probably more helpful if I add the testing images as separate uploads rather than being wrapped up in that zip.
fooocus style selector - Semi Realistic
Semi Realistic Log Data
Semi Realistic Metadata

@DavidDragonsage DavidDragonsage marked this pull request as draft March 11, 2024 16:26
Removed "grayscale, bw" from the proposed Fooocus Realistic entry at Line 6 of sdxl_styles_fooocus.json
@DavidDragonsage
Copy link
Contributor Author

After further thought, I removed "grayscale, bw," from the proposed Fooocus Semi Realistic style that I added to sdxl_styles_fooocus.json. This was done in Commit #5.

I think this new negative style should be as generic as possible because it has useful suppression such as "signature", "bad anatomy" and "out of frame" that would benefit sketches, black & white photos and greyscale printing - as well as the anime and paintings that I was focused on earlier.

Many of the genre specific styles such as SAI Anime already have "black and white" in the negative prompt already, and others such as Artstyle Impressionist have "color" in the positive prompt. If the user occasionally has to add "black and white" manually to a negative prompt I would think that would be preferable than trying to list many of the other beneficial negatives that are in found Fooocus Semi Realistic.

I made a mistake while I was trying to figure out how to revise a PR. Commit #4 removes sdxl_styles_fooocus.json completely. This commit should obviously be deleted, but I haven't been able to discover how to do that. My apologies!

@DavidDragonsage
Copy link
Contributor Author

Oh dear, now I think I closed the whole PR! Hopefully this will reopen it.

@DavidDragonsage DavidDragonsage marked this pull request as ready for review March 11, 2024 17:59
@mashb1t mashb1t added the Size S small change, basically no testing needed label Mar 11, 2024
juggernautv8, seed 0, 1024x1024, no LoRAs, only this style, positive prompt "cat"
@mashb1t
Copy link
Collaborator

mashb1t commented Mar 11, 2024

Hey @DavidDragonsage, thank you for your first contribution, which is especially well documented!
Your intention is totally understandable and the benefits of using the new style Fooocus Semi Realistic for semi-realistic use-cases are obvious.
Backwards compatibility is another factor. This change luckily will neither affect users who import data from private log nor those who will import metadata from already generated images, only those who start Fooocus with the prefix and not change anything are affected. As the style only affects negative prompt and is used by default in combination with Fooocus V2 and Fooocus Masterpiece, the new style should not result in unexpected behaviour.

Comparison (metadata in images, compared between same seeds):

Rhino

Negative
image - 2024-03-11T215535 091

Semi Realistic
image - 2024-03-11T215410 360

girl with lizard on her hat

Negative
image - 2024-03-11T214804 084

Semi Realistic
image - 2024-03-11T214730 750

Brown Cat

Negative
2024-03-11_21-38-05_5476

Semi Realistic
2024-03-11_21-37-47_4072

Forest Elf

Negative
image - 2024-03-11T205648 010

Semi Realistic
image - 2024-03-11T205711 466

We can conclude that this PR delivers real benefit to users when generating anime / semi-realistic images.
I'll keep this PR open for a few more days before merging to give others also the chance to review it.
Thanks again for your contribution and the excellent PR, much appreciated.

@mashb1t mashb1t added good first issue Good for newcomers and removed good first issue Good for newcomers labels Mar 11, 2024
@mashb1t mashb1t added this to the 2.3.0 (draft) milestone Mar 11, 2024
@DavidDragonsage
Copy link
Contributor Author

DavidDragonsage commented Mar 11, 2024

Thank you, @mashb1t, I appreciate the encouragement!

Do you need a different cat thumbnail using the criteria mentioned above:
juggernautv8, seed 0, 1024x1024, no LoRAs, only this style, positive prompt "cat"
In fact I was just in the process of doing so when I read your latest message.

I rather liked my first cat thumbnail, but I don't mind making another one if that's not the standard. :-)
The first one was made with a random seed using the only the new style, but with the Anime preset.

@mashb1t
Copy link
Collaborator

mashb1t commented Mar 11, 2024

@DavidDragonsage no need, i've alread created a new one with standard criteria

@DavidDragonsage
Copy link
Contributor Author

Thank you, yes I just found it was already there just now. I was just about to commit the same version of the little beast. :)

I can appreciate why they should all be standardized with the same model and seed, that gives the maximum consistency.

I was intrigued by the study you made with the four anime subjects. It seemed that the new negative style allowed for a little more ornamentation. I'm always fascinated that even negative prompts can create stylistic variations. It makes sense, but it still seems surprising to me.

@mashb1t mashb1t changed the base branch from main to develop March 15, 2024 21:50
@mashb1t mashb1t merged commit 37274c6 into lllyasviel:develop Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Size S small change, basically no testing needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Improved Negative Prompt Support for Anime, Paintings and Other Non-Photographic Media
2 participants