new-run #4586

fgaz · 2017-07-04T18:02:37Z

The ci should be green now. Changelog, docs, and more tests incoming.

Can someone review it in the meantime?

This reverts commit aa682f9.

mention-bot · 2017-07-04T18:02:38Z

@fgaz, thanks for your PR! By analyzing the history of the files in this pull request, we identified @dcoutts to be a potential reviewer.

fgaz · 2017-07-04T18:03:16Z

ping @dmwit

ezyang

Looks OK to me. Would definitely good to expand test coverage.

ezyang · 2017-07-04T18:16:13Z

cabal-install/Distribution/Client/CmdRun.hs

@@ -90,7 +121,8 @@ runAction (configFlags, configExFlags, installFlags, haddockFlags)
    baseCtx <- establishProjectBaseContext verbosity cliConfig

    targetSelectors <- either (reportTargetSelectorProblems verbosity) return
-                   =<< readTargetSelectors (localPackages baseCtx) targetStrings
+                   =<< readTargetSelectors (localPackages baseCtx)
+                         (take 1 targetStrings) -- we drop the exe's args


Urk. Wouldn't it be better to split targetStrings first before doing anything with it? See (A).

ezyang · 2017-07-04T18:16:53Z

cabal-install/Distribution/Client/CmdRun.hs

+                                  pkg
+                                  exe
+               </> exe
+    let args = drop 1 targetStrings


(A) Then it wouldn't be necessary to subsequently drop it here!

I did this because (see B1 and B2) we don't know anything about the targets at the beginning of the function, and an empty targetStrings could have a meaning. If i wrote a match for (command:args) and [] then I'd have to duplicate most of the code or put everything in an helper function.

the take/drop approach handles well the absence of a target component.

But now that i think of it, what if we do
new-run <no explicit exe here, there is only one in the package> args? the first arg will be interpreted as a target and the command will fail.
But I don't think there is a way to handle this without ambiguities.

How about new-run -- args for "run whatever executable is the default, with arguments args" and new-run args for "search for an executable named args and run it"? Separating arguments for a subprocess with -- is a pretty common idiom. (e.g. old-style run supports this.)

Good idea.

Cabal's flags are filtered from the targetStrings already, so it should simply be a matter of checking the first targetString

No it isn't.

The first -- is stripped from the targetStrings as well, probably because it marks the end of cabal flags and the beginning of the targets.

But if we document it appropriately we could use two of them, the first separating flags and targets, the second targets and targets' args

(actually its only use will be as a "placeholder" for the target when args are also needed, like @dmwit says)

Ok, -- -- is ugly and prevents future modification of the syntax, so I'll leave this as it is for now (args only when the exe/package is specified)

btw, #3638 may be the origin of -- --

Apparently even new-run target -exearg --exearg doesn't work, as all --prefixed args are interpreted as cabal args, while new-run target -- --exearg works.

Fixing this requires some modifications to the args parsing, but I don't think this is needed, as this was also the behaviour of old-run (except #4600).

edit: also run <empty> args isn't supported in old-run, so I'll leave new-run as is for now

Keep in mind: modeling our behavior after old-style run is good to a point (because it lets us leverage folks' familiarity with the old UI); but as we're making lots of breaking UI changes here, this is also a good time to fix warts. So I think any behavior we choose should either be the right behavior or have a clear forward-compatibility story with the right behavior; backward-compatibility with the old, wrong behavior is less of a concern.

ezyang · 2017-07-04T18:20:02Z

cabal-testsuite/PackageTests/NewBuild/T4477/cabal.out

+ - T4477-1.0 (exe:foo) (first run)
+Configuring executable 'foo' for T4477-1.0..
+Preprocessing executable 'foo' for T4477-1.0..
+Building executable 'foo' for T4477-1.0..


Maybe not for this PR, but we should think carefully about whether or new-run should output this sort of information. In favor: it's good to be able to see when your executable is being built. Against: you won't be able to use new-run to run executables that you want to run as part of a pipeline, since there will be extra Cabal build progress goo in your stderr. Perhaps this is related to the desire for a mode, "please run this as quickly as possible, don't rebuild at all!"

The question of whether new-run should output its own information is a good one. My stance on "build or not" is that new-run should always rebuild the appropriate files, and new-exec should never rebuild anything (but should ensure that all the project's executables would be on the PATH if they were built). This makes the divide nice and clean, and supports both needs.

We also have -v0

ezyang · 2017-07-04T18:22:06Z

cabal-install/Distribution/Client/CmdRun.hs

+  -> FilePath
+binDirectoryFor layout config package exe = case elabBuildStyle package of
+  BuildAndInstall -> installedBinDirectory package
+  BuildInplaceOnly -> inplaceBinRoot layout config package </> exe


This logic is duplicated with inplace_bin_dir in Distribution/Client/ProjectPlanning.hs. It would be good to have this in only one place.

Will merge.

ezyang · 2017-07-04T18:25:39Z

cabal-install/Distribution/Client/CmdRun.hs

+    matchingExecutable p = atMostOne
+                         $ filter (\x -> Just x == componentString
+                                      || isNothing componentString)
+                         $ executablesOfPackage p


You don't have to change this, but my preference here would have been to case on p here for the package versus component case, rather than put everything through the "get the list of components it defines, and now check if anything in that list matches". My primary reasoning here is that the component case is more reflective of what the "good" code looks like, but the way things are wired up now it makes it seem like the package case (where multiple components may exist) is more important.

ezyang · 2017-07-04T18:28:09Z

cabal-install/Distribution/Client/CmdRun.hs

+  -> ElaboratedInstallPlan -- ^ a plan in with to search for matching exes
+  -> [(ElaboratedConfiguredPackage, UnqualComponentName)] -- ^ the matching package and the exe name
+extractMatchingElaboratedConfiguredPackages
+  pkgId component = nubBy equalPackageIdAndExe


Why is a nub necessary here? If the same package and component show up multiple times, isn't that a problem?

If the there are multiple exes, then for each exe there will be an ElaboratedConfiguredPackage in the ElaboratedInstallPlan.

For each ElaboratedConfiguredPackage then, when elabPkgOrComp (see executablesOfPackage) is ElabComponent then there will be one exe, the matching one (it's a Maybe), but if it's an ElabPackage (which happens in custom builds), then that field is Nothing and we have to look at the executables field, which contains every exe in the package, hence the duplication.

I can't dedup before that point because i need both the package and the component name to do so (what if two packages have the seme exe name?).

Maybe I should add a comment about this. Or maybe it's a bug? Or should I add another field to ElaboratedConfiguredPackage/ElaboratedInstallPlan?

ezyang · 2017-07-04T18:33:07Z

cabal-install/Distribution/Client/CmdRun.hs

@@ -90,7 +121,8 @@ runAction (configFlags, configExFlags, installFlags, haddockFlags)
    baseCtx <- establishProjectBaseContext verbosity cliConfig

    targetSelectors <- either (reportTargetSelectorProblems verbosity) return
-                   =<< readTargetSelectors (localPackages baseCtx) targetStrings
+                   =<< readTargetSelectors (localPackages baseCtx)


It is too bad there is no singular version of readTargetSelectors, since it seems like that is what you want.

ezyang · 2017-07-04T18:34:14Z

cabal-install/Distribution/Client/CmdRun.hs

+         of [x] -> return x
+            [ ] -> die'
+                     verbosity
+                     "No targets given"


Won't this error message occur when I type cabal new-run? I think you can give a more informative message here.

(B2) No arguments means "treat the package in the cwd/project as a target, if there is one" (look at the comment above extractMatchingElaboratedConfiguredPackages). But this is also handled before, so same as (B1).
This will work after #4583 is merged.
I'll add a test for it too.

ezyang · 2017-07-04T18:35:11Z

cabal-install/Distribution/Client/CmdRun.hs

+                     "No targets given"
+            _   -> die'
+                     verbosity
+                     "Multiple targets given"


This cannot happen, by virtue of the take 1, correct? In that case, better to mark it as an internal error.

(B1) Not by virtue of the take 1. For example if we specify a package with 2 exes in the targetStrings there will be two targets. But this is handled by lines 138-149 already, so you are correct. Should iI use an assert?

Wait, did i just duplicate resolveTargets' functionality? >.<
edit: Nvm, it doesn't return enough information. I think I already checked it some time ago.

I think "multiple targets" is a bit misleading. It's more like a single, ambiguous target. Perhaps the error could be improved by mentioning longer target strings the user might want to try to disambiguate between the possibilities, e.g. "Ambiguous target my-package; could match my-package:my-exe1 or my-package:my-exe2" or "Ambiguous target my-exe; could match my-package1:my-exe or my-package2:my-exe". (Or am I misunderstanding when we would hit this branch?)

@dmwit: as @ezyang says we shouldn't hit that branch. It's allalready handled with better error reporting before.

ezyang · 2017-07-04T18:39:31Z

cabal-install/Distribution/Client/CmdRun.hs

+    let args = drop 1 targetStrings
+    runProgramInvocation
+      verbosity
+      (simpleProgramInvocation exePath args)


I don't think you want runProgramInvocation here; it doesn't propagate the exit code. Use rawSystemExitWithEnv instead, see Distribution/Client/Run.hs for how it is previously done. You may also want to consider adding LD_LIBRARY_PATH if it is necessary.

Will do.
Isn't LD_LIBRARY_PATH automated by the os or something?

I checked and it does propagate it, as it uses rawSystemExit under the hood.

Do we have a test case that checks that new-run propagates the exit code?

Now we have one

23Skidoo · 2017-07-05T10:50:46Z

Revert "(,) is not Traversable on older base versions"

Note that you can also do git rebase -i and just remove that commit from your branch. This is not super important, but makes history cleaner and PRs a bit easier to review.

23Skidoo · 2017-07-05T11:09:13Z

And by the way, can you please format your commit messages in the standard way? See this article: https://chris.beams.io/posts/git-commit/ for an explanation of what that means. No need to reword commit messages in this PR, but something to keep in mind for the future.

fgaz · 2017-07-05T12:39:59Z

@23Skidoo
Sorry, I don't know why but I assumed the standard merging procedure here was to squash the commits, so I didn't pay much attention to the messages. Will do next time, thanks for the heads up.

We test both for single and multiple exe/package. `new-run` must halt on ambiguities or no compatible exes, and must run the exe if one can be uniquely selected between packages and components.

fgaz · 2017-07-05T15:48:14Z

Tests ready!
If they fail it's probably because of #4583

fgaz · 2017-07-05T15:51:48Z

cabal-testsuite/PackageTests/NewBuild/CmdRun/MultipleExes/cabal.out

+# cabal new-run
+Up to date
+# cabal new-run
+cabal: The run command is for running a single executable at once. The target '' refers to the package MultipleExes-1.0 which includes the executables foo and bar.


And here's the need for #4583

dmwit · 2017-07-05T21:13:34Z

cabal-install/Distribution/Client/CmdRun.hs

+             -> Bool
+matchPackage pkgId pkg =
+  pkgId == Just (elabPkgSourceId pkg)
+  || isNothing pkgId --if the package is unspecified (Nothing), all packages match


Slight bikeshed: matchPackage pkgId pkg = maybe True (elabPkgSourceId pkg ==) pkgId seems to say what you want more concisely. Might be worth pulling out

maybeMatches :: Eq a => a -> Maybe a -> Bool maybeMatches x = maybe True (x==)

since matchComponent has very similar code.

Looks ok, but I'd prefer to flip the arguments (so the "thing to search/filter for" is first as usual).

I'm also tempted to change Maybe with something like
data Match a = Anything | Exactly a
but losing all the functions in Data.Maybe probably isn't worth it.

dmwit · 2017-07-05T21:23:29Z

cabal-install/Distribution/Client/CmdRun.hs

+                         $ executablesOfPackage p
+    componentString = componentNameString =<< component
+    atMostOne [x] = Just x
+    atMostOne _ = Nothing


exactlyOne?

or safeHead, or headMaybe...

yeah, that probably wasn't the best name :-P

23Skidoo

Left some minor comments.

23Skidoo · 2017-07-14T16:47:58Z

Cabal/doc/nix-local-build.rst

+target, which can be a component, a package or can be left blank, as
+long as it can uniquely identify an executable within the project.
+
+See new-build for the target syntax.


s/new-build/new-build section/
Can you also add a hyperlink here?

23Skidoo · 2017-07-14T16:48:34Z

cabal-install/changelog

@@ -5,6 +5,9 @@
 	roll back the index to an earlier state.
 	* 'cabal new-configure' now backs up the old 'cabal.project.local'
 	file if it exists (#4460).
+	* Completeded the 'new-run' command (#4477). The functionality is the


Typo: completeded.

23Skidoo · 2017-07-14T16:57:13Z

cabal-install/Distribution/Client/CmdRun.hs

+            selectedComponent
+            elaboratedPlan
+
+    -- the names to match. used only for user feedback, as


Please use proper punctuation in comments (i.e. start sentences with capital letters, end them with full stops, and so on).

23Skidoo · 2017-07-14T16:59:28Z

cabal-install/Distribution/Client/CmdRun.hs

+    let args = drop 1 targetStrings
+    runProgramInvocation
+      verbosity
+      (simpleProgramInvocation exePath args)


Do we have a test case that checks that new-run propagates the exit code?

23Skidoo · 2017-07-14T17:02:32Z

Would be nice if @dcoutts could take a look too.

fgaz · 2017-07-14T17:46:24Z

Once I fix those minor problems there are two issues I'd like to address before merging:

The args issue.

I thought about it, and i think the old-run command solved the problem well. I often add a cabal flag after the exe name, and the -- syntax is standard. This conflicts with the <empty-target> -- args syntax which I don't think is that useful, so in my opinion we should keep it as it is.

Returning the target from runProjectPreBuildPhase
There is duplication between the logic in the function passed to runProjectPreBuildPhase and around line 170. We check for multiple/no targets two times (the second one assuming everything will go well) when we could simply return the single target from runProjectPreBuildPhase.

But in the only function we pass to it we can only alter the ElaboratedInstallPlan, so i think we should change its type to one of the following:

.... -> (ElabInstPlan -> (ElabInstPlan, arbitraryData) -> ... -> IO (ProjectBuildContext, arbitraryData)
which is probably bad because that type variable only calls for misuse
.... -> (ElaboratedInstallPlan -> (ElaboratedInstallPlan, Targets) -> ... -> IO ProjectBuildContext
.... ((ElaboratedInstallPlan, ProjectBuildContext )-> (ElaboratedInstallPlan, ProjectBuildContext) -> ... -> IO ProjectBuildContext
and put the target in a new field in ProjectBuildContext.
This way future modifications are also easy.

I think @dcoutts has some more info about this.

This last issue is only an implementation problem and does not affect the behaviour of new-run, so if you are all ok with this I'll open another issue for this and finally merge this one (once I've fixed those typos and tests >.<) so we can have a working new-run and solve that problem when we know more.

fgaz · 2017-07-15T21:22:15Z

All done here, I'm merging this by tomorrow. I'll also open another pr for the duplication issue.

fgaz added 15 commits July 2, 2017 19:23

second attempt at new-run

4a1753e

removed trace import

e0734f3

inline justIfCondition

6d2c200

removed debug statement

e3a87d2

removed unused function

17958db

removed old comment

33d4362

fixed new-run for custom setups

041ffa3

cleaned imports

41db836

added compat prelude import

a0727b2

(,) is not Traversable on older base versions

aa682f9

fixed haskell#4477

b27fb01

Revert "(,) is not Traversable on older base versions"

65a25de

This reverts commit aa682f9.

specialized sequenceA

f9cb38d

removed redundant imports

6a969fc

added test output

1657537

ezyang reviewed Jul 4, 2017

View reviewed changes

23Skidoo requested a review from dcoutts July 5, 2017 11:03

Add tests for new-run

22497b2

We test both for single and multiple exe/package. `new-run` must halt on ambiguities or no compatible exes, and must run the exe if one can be uniquely selected between packages and components.

fgaz commented Jul 5, 2017

View reviewed changes

dmwit reviewed Jul 5, 2017

View reviewed changes

fgaz added 2 commits July 10, 2017 10:11

Rename atMostOne to a more correct exactlyOne

28dbf7f

Add new-run docs

6dfc20f

fgaz added 2 commits July 13, 2017 16:23

Add changelog entry for new-run

c0a50a3

Deduplicate build path construction

9c5b475

23Skidoo reviewed Jul 14, 2017

View reviewed changes

fgaz added 5 commits July 15, 2017 21:36

Fix typos and punctuation

2fd4c18

Output more info about "impossible" failures

fa851f2

Add link to new-build section

7f7cb93

Add assert for exit code

8fd5225

Add test for exit code propagation in new-run

b2e7523

Merge branch 'master' into new-run-2

81b5f4c

fgaz merged commit 7327c12 into haskell:master Jul 16, 2017

fgaz mentioned this pull request Jul 17, 2017

Reduce duplication in CmdRun #4607

Merged

fgaz added the cabal-install: nix-local-build label Jul 28, 2017

fgaz deleted the new-run-2 branch December 15, 2020 14:58

new-run #4586

new-run #4586

Conversation

fgaz commented Jul 4, 2017

mention-bot commented Jul 4, 2017

fgaz commented Jul 4, 2017

ezyang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmwit Jul 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fgaz Jul 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fgaz Jul 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fgaz Jul 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

23Skidoo commented Jul 5, 2017

23Skidoo commented Jul 5, 2017

fgaz commented Jul 5, 2017

fgaz commented Jul 5, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

23Skidoo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

23Skidoo commented Jul 14, 2017

fgaz commented Jul 14, 2017

fgaz commented Jul 15, 2017

dmwit Jul 5, 2017 •

edited

Loading

fgaz Jul 10, 2017 •

edited

Loading

fgaz Jul 5, 2017 •

edited

Loading

fgaz Jul 13, 2017 •

edited

Loading