Brew sandbox fails for apps using FileManager.SearchPathDirectory.applicationSupportDirectory

[emmby]

Output of brew config

HOMEBREW_VERSION: 4.4.21-42-gcf635bc
ORIGIN: https://github.com/Homebrew/brew
HEAD: cf635bc28a02d696a16df58ec77ea99524d20d7e
Last commit: 2 days ago
Branch: master
Core tap HEAD: b6e3a7c2456ce1ec681f753cf2c3a54fe04c6ce4
Core tap last commit: 35 hours ago
Core tap branch: statesmith
Core tap JSON: 19 Feb 02:54 UTC
Core cask tap HEAD: 1a4d5a258478044d69d341ba8fc290212441fb79
Core cask tap last commit: 6 days ago
Core cask tap JSON: 19 Feb 02:54 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 11
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.7 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: 11-core 64-bit arm_lobos
Clang: 16.0.0 build 1600
Git: 2.39.5 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.3.1-arm64
CLT: 16.2.0.0.1.1733547573
Xcode: 16.2
Rosetta 2: false

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed kegs have no formulae!
This means they were either deleted or installed manually.
You should find replacements for the following formulae:
  statesmith

Warning: You have uncommitted modifications to Homebrew/homebrew-core.
If this is a surprise to you, then you should stash these modifications.
Stashing returns Homebrew to a pristine state but can be undone
should you later need to do so for some reason.
  cd /opt/homebrew/Library/Taps/homebrew/homebrew-core && git stash -u && git clean -d -f

Uncommitted files:
   M Formula/s/statesmith.rb
  ?? Formula/s/.statesmith.rb.swp

Warning: Some taps are not on the default git origin branch and may not receive
updates. If this is a surprise to you, check out the default branch with:
  git -C $(brew --repo homebrew/core) checkout master

Description of issue

I am developing a formula to build and install a mac command line tool. On startup, the tool writes settings to the user's Application Support directory using FileManager.SearchPathDirectory.applicationSupportDirectory per mac best practices.

This works fine in real life, but it fails in the brew sandbox. The brew sandbox remaps $HOME to a directory in /tmp, but it does not remap FileManager.SearchPathDirectory.applicationSupportDirectory which still points to /Users/brew/Library/Application Support when running brew test. The sandbox rejects any attempt to write to Application Support with

Unhandled exception. System.UnauthorizedAccessException: Access to the path '/Users/brew/Library/Application Support/StateSmith.Cli' is denied.
 ---> System.IO.IOException: Operation not permitted
   --- End of inner exception stack trace ---
   at System.IO.FileSystem.CreateDirectory(String fullPath, UnixFileMode unixCreateMode)
   at System.IO.Directory.CreateDirectory(String path)
   at StateSmith.Cli.Data.DataPaths.GetOrMakeDataDirPath()
   at StateSmith.Cli.Data.DataPaths..ctor(IAnsiConsole console)
   at StateSmith.Cli.Program..ctor(String currentDirectory)
   at StateSmith.Cli.Program.Main(String[] args)

I tried symlinking ln -s #/tmp/.../StateSmith.Cli /Users/brew/Library/Application Support/StateSmith.Cli but it does not fool the sandbox, the ln fails with Operation not permitted.

What is the best way to test this app, which writes to Application Support on startup?

[p-linnane]

This isn't possible. Formulae are only installed into the Homebrew prefix. You will be better off creating a cask.

[emmby]

Yes, the formula installs into the homebrew prefix. The problem is the test, which breaks when the tool tries to write to the Application Support directory.

That link doesn't explain what a cask is, but my understanding is that casks are generally for pre-built binaries or GUI apps, neither of which apply here. Are you suggesting a cask as a workaround to get around the sandbox limitations for the Application Support directory? Because it doesn't sound like a natural fit otherwise, unless I'm misunderstanding.

[p-linnane]

The cask option is definitely a workaround so that you can do what you're trying to. I agree, that the current state doesn't sound like a good fit for how Homebrew works. Many CLI tools write their settings to ~/.config or ~/.<appname> folders in the user's home directory instead.