add var/std gradient wrt kw mean

[chengchingwen]

fix problem mentioned in #446

[CarloLucibello]

looks fine and tests are exhaustive, merge?

[CarloLucibello]

@chengchingwen is this giving warnings as you seem to mention in #446 (comment)?

[chengchingwen]

@CarloLucibello yes, the warnings are still there.

[chengchingwen]

if we want to avoid those warnings, we need to avoid using @adjoint for var and std by manually define the _pullback functions.

[DhairyaLGandhi]

Defining internal methods like so seems sketchy. The adjoint definition from ChainRules as well as within zygote can handle it. The adjoint can accept kwargs

[chengchingwen]

I don't really get what you mean. The current @adjoint return nothing for all the keyword arguments.

[DhairyaLGandhi]

Let me rephrase that, we don't want to differentiate kwargs here rather let those be passed to the functions appropriately.

[chengchingwen]

But the default std/var behavior only accept mean as keyword argument. If we want to make mean be differentiable as position argument, then we will result in a std/var api that only work when Zygote is imported. Otherwise, we might need to make a PR to Statistics for the std/var that accept mean as position argument.

[racinmat]

Hi, is there any update on this? Is there a plan for that to be supported?

[mcabbott]

What's an example of when this would matter, and would give correct results? On simple things, and the example from here: #446 (comment) it gives zero. With @show inserted:

julia> gradient([1,2,3]) do x
         m = mean(x)
         std(x; mean=m)
       end
backmean = -0.0
([-0.5, 0.0, 0.5],)

julia> function normalise(x::AbstractArray; dims=1)
         μ′ = mean(x, dims = dims)
         σ′ = std(x, dims = dims, mean = μ′, corrected=false)
         return (x .- μ′) ./ σ′
       end
normalise (generic function with 1 method)

julia> gradient(x -> sum(sin, normalise(x)), [1,2,3.0])
backmean = [-0.0]
([-0.2697761903106471, 0.5395523806212941, -0.2697761903106471],)

Stumbled on this while thinking about JuliaDiff/ChainRules.jl#567

[chengchingwen]

@mcabbott Yes, normalise was not an good example for this as the gradient of std(x, mean=μ) wrt μ is 0 if μ == mean(x). It would only matter when std(x, mean=μ) where μ != mean(x) and it's probably arguable whether you would ever do that.