Fixed Square Brush, Improve Brush Hardness and Smoothing Precision

[brucew4yn3rp]

This PR does the following:

• Improves the scaling and calculation of brush hardness
  • Historically, a radial gradient was being applied to both the circle and square pens with a hardness < 1, which was causing the square pen to revert into a circle. This has been fixed with the use of a helper, drawSoftSquare function.
• Improves the scale of smoothing precision so that there is a meaningful difference in step count based on the smoothing precision.

Fixed.Smoothing.Precision.and.Soft.Square.Brush.mp4

┆Issue is synchronized with this Notion page by Unito

[github-actions]

⚠️ Warnings

⚠️ Warning: E2E Test Coverage Missing

If this PR modifies behavior that can be covered by browser-based E2E tests, those tests are required. PRs lacking applicable test coverage may not be reviewed until added. Please add or update browser tests to ensure code quality and prevent regressions.

[brucew4yn3rp]

@trsommer @webfiltered Quickly following up; this is a much smaller, addendum, PR that improves functionality to 4361

[brucew4yn3rp]

@christian-byrne @webfiltered Can you please review this PR when you have a chance?

[brucew4yn3rp]

Hi @webfiltered @christian-byrne

Following up on the review of this patch as it fixes and improves functionality of the new Image Canvas. Shouldn’t be too much in terms of changes to review.

[christian-byrne]

Great work fixing the square brush hardness issue! The solution correctly addresses the core problem where square brushes were becoming circular when hardness < 1.

Key Improvements:

• Fixes square brush rendering with proper hardness control
• Improves smoothing precision algorithm with better granular control
• Maintains backward compatibility
• Clear separation between RGB and mask brush logic

I've added specific inline comments focusing on performance optimizations and code quality improvements. The most important suggestions are:

1. Performance: Consider optimizing the 10-fill loop in drawSoftSquare
2. Code Quality: Leverage existing colorUtil functions for safer color manipulation

The changes work correctly as-is - these are optimization suggestions for better maintainability and performance.

[brucew4yn3rp]

Thank you for your review @christian-byrne. I've replied to your comments and pushed some updates accordingly.

[christian-byrne]

LGTM

[christian-byrne]

Performance concern with the drawSoftSquare function - is it creating a new canvas and manipulating ImageData for every brush stroke when using soft square brushes?

That seems liek it would be significantly slower than the existing gradient-based rendering for other brush types. Could we consider using cached brush textures or optimizing the ImageData approach to avoid creating new canvases on each stroke?

The fix for square brush hardness is great, but the performance impact could cause lag during drawing for users of soft rectangular brushes.

[christian-byrne]

const tempCanvas = document.createElement('canvas')
const tempCtx = tempCanvas.getContext('2d')
// ... manual pixel manipulation loop

[brucew4yn3rp]

Thanks @christian-byrne. I've fixed brush rendering anti-aliasing and performance issues.

• Added solid fills for max hardness brushes to eliminate gradient anti-aliasing and achieve true binary mask edges.
• Implemented texture caching for soft square brushes to avoid creating new canvases on every stroke. The cache is self-managing in terms of memory to avoid bloat.

Additionally, It seems that @trsommer has reacted with a thumbs up to Kevin's message here in regards to adding me as codeowner. Could you / the team please do so when you have a chance?

[christian-byrne]

Additionally, It seems that @trsommer has reacted with a thumbs up to Kevin's message #4361 (comment) in regards to adding me as codeowner. Could you / the team please do so when you have a chance?

Sure, could you make a PR to add yourself to the CODEOWNERS file?

[brucew4yn3rp]

Done - can you please approve changes if they look good to you?

[christian-byrne]

This is looking good. I tested it and it works nicely. What are your thoughts on adding a unit test 😄? You don't have to.

[christian-byrne]

What about using an LRU cache? We have a library we use in some other places, it can be used like:

import QuickLRU from '@alloc/quick-lru'

private static brushTextureCache = new QuickLRU<string, HTMLCanvasElement>({
  maxSize: 8 // Reasonable limit for brush texture variations?
})

[brucew4yn3rp]

I can leverage QuickLRU but I'm getting the following error when doing pnpm run build. Have you seen this before? I git clone'd the main Comfy repo and the error is happening with that one too:

> vue-tsc --noEmit

error TS2688: Cannot find type definition file for 'dompurify'.
  The file is in the program because:
    Entry point for implicit type library 'dompurify'

[christian-byrne]

Does it still happen when using node 24?

nvm use 24

[brucew4yn3rp]

It does still happen when using v24.4.0

I bypassed the dompurify issue by going to ComfyUI_frontend\src\types and creating a file dompurify.d.ts that contains: declare module "dompurify"

But there are still issues with the packages. This is likely stemming from the switch to pnpm in the latest frontend updates. For example, the primeuix forms, styles, and themes are all giving errors like the following

 ERROR  error during build:
[generate-import-map-plugin] ENOENT: no such file or directory, copyfile 'C:\Users\BATMAN\Desktop\Testing\ComfyUI_frontend\node_modules\@primeuix\forms\index.mjs' -> 'C:\Users\BATMAN\Desktop\Testing\ComfyUI_frontend\dist\assets\lib\@primeuix\forms\index.mjs'
    at Object.copyFileSync (node:fs:3100:11)
    at Object.generateBundle (file:///C:/Users/BATMAN/Desktop/Testing/ComfyUI_frontend/vite.config.mts.timestamp-1756603068573-7b2cee086facf8.mjs:140:14)
    at file:///C:/Users/BATMAN/Desktop/Testing/ComfyUI_frontend/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/es/shared/node-entry.js:22395:40

Are you not seeing the errors on your end?
It might have to do with pnpm’s behavior on Windows

Switching to pnpm breaks things because it doesn’t use a flat node_modules like npm/yarn — instead it creates symlinks into a hidden store. Tools and type resolvers (like TypeScript looking for @types/dompurify or your import-map plugin looking for @primeuix/styles) often assume the old flat layout, so they can’t find files where they expect them and throw ENOENT or “cannot find type definition” errors.

Might be easier to chat on discord to not sidetrack this thread? I sent you a DM and also posted in the frontend channel of the Comfy server.

[christian-byrne]

Are you not seeing the errors on your end?

No, I am not. Have you tried deleeting the node_modules folder and reinstalling?

rm -rf node_modules pnpm-lock.yaml
pnpm install 

Can also try

rm -rf node_modules pnpm-lock.yaml
pnpm install --shamefully-hoist 

[brucew4yn3rp]

@christian-byrne Fixed the issue. The frontend’s switch to pnpm resulted in much longer file paths due to its symlinked node_modules structure. On my system, pnpm install was silently failing because Windows long paths were not enabled in the group policy:

Just pushed an update that uses QuickLRU for the brushTextureCache.

If the changes look good to you, can you approve instead of @trsommer? I'm not sure that he is active on this PR.

[christian-byrne]

LGTM!