Create AI art with ComfyUI and Krita

ComfyUI provides powerful AI image generation, but to make it truly useful, you need to integrate it into your creative workflow. This guide shows you how to connect ComfyUI running on Olares to Krita on your computer, so you can generate AI artwork directly within your digital painting environment.

Learning objectives

In this guide, you will learn how to:

• Deploy and configure ComfyUI in Olares to maximize performance and resource efficiency.
• Install and configure the Krita AI Diffusion plugin.
• Connect Krita to your Olares-hosted ComfyUI instance.
• Generate AI artwork using text prompts in Krita.

Prerequisites

• A working Olares installation with ComfyUI Shared installed and running.
• Krita installed on your computer.
• Sufficient system resources on your Olares device to download models.

Set up ComfyUI

Open ComfyUI Launcher and click START to make sure the service is running.

Maximize GPU performance

You can set the GPU mode to App exclusive and assign ComfyUI full GPU access in Settings > GPU to ensure maximum performance.

Get the endpoint for ComfyUI

1. On Olares, open Settings, then go to Application > ComfyUI Shared.

2. In Entrances, click ComfyUI.

3. Make sure its Authentication level is set to Internal.

4. Click Set up endpoint, then copy the endpoint URL displayed.

   

Download and enable AI Diffusion plugin

1. Download the Krita AI Diffusion plugin.

2. Launch Krita, and from the toolbar, select Tools > Scripts > Import Python Plugin from File.

   

3. Select the downloaded ZIP package.

4. When prompted, confirm the plugin activation and restart Krita.

   

5. After restarting, verify the installation in Krita > Preferences > Python Plugin Manager.

   

Connect Krita to ComfyUI

The connection steps depend on whether your computer and Olares device are on the same network.

Prepare models and plugins

Install required custom nodes

The Krita AI Diffusion plugin requires the following custom nodes:

• ControlNet preprocessors: https://github.com/Fannovel16/comfyui_controlnet_aux
• IP-Adapter: https://github.com/cubiq/ComfyUI_IPAdapter_plus
• Inpaint nodes: https://github.com/Acly/comfyui-inpaint-nodes
• External tooling nodes: https://github.com/Acly/comfyui-tooling-nodes

To install them:

1. Open ComfyUI Launcher, and go to Plugins > Custom Install.

2. Paste the GitHub URL of the custom node (e.g., https://github.com/Acly/comfyui-tooling-nodes), and click INSTALL PLUGIN.

3. Repeat for each of the remaining custom nodes listed above.

4. Go back to the Home page in ComfyUI Launcher, then click RESTART for the changes to take effect.

5. Optional: If you go back to Krita and click Connect again, you should see an error message indicating that required models are still missing.

   

Install required models

The plugin needs these utility models to work properly. Without them, some features will not function correctly. Pre-installing them ensures a smoother experience.

• NMKD Superscale: https://huggingface.co/gemasai/4x_NMKD-Superscale-SP_178000_G/resolve/main/4x_NMKD-Superscale-SP_178000_G.pth
• OmniSR X2: https://huggingface.co/Acly/Omni-SR/resolve/main/OmniSR_X2_DIV2K.safetensors
• OmniSR X3: https://huggingface.co/Acly/Omni-SR/resolve/main/OmniSR_X3_DIV2K.safetensors
• OmniSR X4: https://huggingface.co/Acly/Omni-SR/resolve/main/OmniSR_X4_DIV2K.safetensors
• MAT Inpaint: https://huggingface.co/Acly/MAT/resolve/main/MAT_Places512_G_fp16.safetensors

To download them via ComfyUI Launcher:

1. Open ComfyUI Launcher, and go to Models > Custom Download.

2. Download upscale models:

   a. Paste the NMKD Superscale URL, set Destination folder to Upscale Models, and click DOWNLOAD MODEL.

   b. Repeat for the three OmniSR models using the same destination folder.

3. Download the inpaint model:

   a. Paste the MAT Inpaint URL.

   b. Set Destination folder to Custom Directory.

   c. Enter inpaint as the Directory Name.

   d. Click DOWNLOAD MODEL.

4. Go back to the Home page in ComfyUI Launcher, then click RESTART for the changes to take effect.

5. Optional: If you go back to Krita and click Connect again, you should see an error message indicating that base models are still missing.

   

Install a base diffusion model

At least one diffusion model (commonly called a “checkpoint”) is required. This guide uses Z-Image Turbo as an example. Z-Image Turbo is a medium-sized model that balances quality and speed, producing realistic images without requiring excessive memory.

1. Open ComfyUI Launcher, and scroll down to the Package installation section.

2. Find Z-Image Turbo Package and click VIEW.

   

3. On the package details page, click GET ALL to start downloading. You can track the progress in the status bar.

   

4. Go back to the Home page in ComfyUI Launcher, then click RESTART for the changes to take effect.

5. In Krita, go to Connection > Server Configuration and click Connect again. A green “Connected” indicator confirms a successful connection. You should see Z-Image marked as “supported” in the base model list.

   

Add a style

Before generating images, you need to create a Style Preset that tells Krita which model to use.

1. Open the Configure Image Diffusion dialog in Krita, and go to the Styles tab.

2. For Style Presets, select Z-Image Turbo from the built-in styles.

   

3. Click the duplicate icon to create a duplicate of the current style.

   

4. For Model Checkpoint, select the Z-Image model. The model name should be public/z_image_turbo_bf16.

   

5. Click the refresh icon to refresh the available styles.

6. Keep default values for other settings, and click Ok to save changes.

   WARNING

   It is recommended to use the default settings if you are not familiar with Krita. Changing the default settings might generate unexpected results.

Create AI art with text prompts

1. In the AI Image Generation panel, confirm that the Z-Image Turbo style is selected.

2. Enter your prompts in the text box. For example:

   A person relaxing on a sandy beach, basking in the warm sunlight, with the calm blue ocean in the background.

3. Click Generate. The generated image appears on the canvas.

4. Click Generate again to generate a new image.

5. Select a preferred result, and click Apply to add it to the layers.

Inpaint

To refine specific areas of a generated image, use inpainting. This lets you modify parts of the image while keeping the rest intact.

1. Select the freehand selection tool and draw around the area you want to modify.

2. Enter a description for what you want in the selected area. For example:

   Seagulls can be seen flying in the distant sky.

3. Click Fill. Several fill candidates will appear in the panel.

4. Click each candidate to preview it on the canvas.

5. When you find a result you like, click Apply to add it to the layers.

Troubleshooting

Cannot connect to ComfyUI from Krita

If Krita shows a connection error:

Check	What to do
Network connectivity	Make sure your computer and Olares are on the same network.
ComfyUI authentication level	In Settings > Application > ComfyUI Shared, confirm it is set to Internal.
LarePass VPN for .com URLs	Enable VPN connection in the LarePass desktop app.
Interfering proxy/VPN	Temporarily disable other VPN or proxy software.
ComfyUI service status	Open ComfyUI Launcher and verify the service is Running.
GPU access	In Settings > GPU, verify ComfyUI is bound to the GPU with enough VRAM allocated.
Required plugins and models	Make sure all custom nodes, utility models, and base diffusion models are downloaded and ComfyUI has been restarted.

Learn more

• ComfyUI quick start guide: Install ComfyUI and generate your first image.
• Manage ComfyUI using ComfyUI Launcher: Control the ComfyUI service, manage models, and configure the environment.
• Krita AI Diffusion documentation: Explore advanced features and workflows.