Skip to content

Autofocus improvements #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: next
Choose a base branch
from
Open

Autofocus improvements #270

wants to merge 8 commits into from

Conversation

njhollinghurst
Copy link
Collaborator

The goal is to overhaul the RPI AF algorithm to:

  • Treat low-confidence PDAF values more linearly, which may improve reliability and reduce wobble a little.
  • Check for IR lighting and suppress PDAF when it's detected (only when a new flag is set in the tuning file).
  • Allow CDAF-based scan to be re-triggered multiple times in Continuous AF mode (when there's no PDAF).
  • Allow subsequent scans to start from the current lens position, rather than always from 0 (infinity).
  • Skip the "fine" search altogether (when the tuning file has "step_fine": 0).
  • Read the default lens position from the tuning file, rather than hard-coding it to 1.0 in source code as now.
  • Not send the default position to the lens driver, when it would be immediately overridden by a user-specified manual setting.

without significant regression or making the code too messy.

This was referenced Jun 2, 2025
Open
Open
@pinefeat
Copy link

pinefeat commented Jun 2, 2025

It works great!
I haven’t tried the PDAF-related changes yet, but Continuous AF mode works very well even with CDAF-based scanning on the HQ camera. My setup uses an external motorised lens. If the default lens position - read from the tuning file - is close enough to the actual focus point, the scan doesn't deviate too far from it and quickly determines the sharpest focus.

@njhollinghurst njhollinghurst force-pushed the af_202506 branch 5 times, most recently from 5a7fceb to 06d67e8 Compare June 4, 2025 16:14
@njhollinghurst njhollinghurst marked this pull request as ready for review June 4, 2025 16:15
@njhollinghurst njhollinghurst requested a review from naushir June 4, 2025 16:15
@njhollinghurst njhollinghurst changed the title Autofocus improvements (WIP) Autofocus improvements Jun 4, 2025
@njhollinghurst
Copy link
Collaborator Author

Any review comments so far?

I might fiddle again with the CDAF->PDAF transition (earlyTerminationByPhase()) which is a bit sluggish.

@naushir
Copy link
Collaborator

naushir commented Jun 11, 2025 •
edited
Loading

Nothing stands out as needing attention IMO. I've not been able to follow the AF algorithm changes fully, but expect they are all good from your testing.

If at all possible, can we split the functionality from commit 310bf3f into multiple patches? Only if it's not much work to do this!

@njhollinghurst
Copy link
Collaborator Author

Now split, which makes the intent of each change clearer, although they are somewhat related.

Can be squashed again if required.

@njhollinghurst njhollinghurst force-pushed the af_202506 branch 2 times, most recently from 89e67d3 to a1ba263 Compare June 16, 2025 12:52
@njhollinghurst
Copy link
Collaborator Author

This was based on main, I'd better try to re-base it off next...

@njhollinghurst
ipa: rpi: Defer initialising AF LensPosition ControlInfo and value
49d845c 
This fixes two small bugs:

We previously populated LensPosition's ControlInfo with hard-coded
values, ignoring the tuning file. Now we query the AfAlgorithm to
get limits (over all AF ranges) and default (for AfRangeNormal).

We previosuly sent a default position to the lens driver, even when
a user-specified starting position would follow. Defer doing this,
to reduce unnecessary lens movement at startup (for some drivers).

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: Improve findPeak() function in AF algorithm
6519a61 
Improve quadratic peak fitting in findPeak(). The old approximation
was good but only valid when points were equally spaced and the
MAX was not at one end of the series.

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: AutoFocus weighting tweak
0f37cb5 
In getPhase(), stop using different weights for sumWc and sumWcp.
This should improve linearity e.g. in earlyTerminationByPhase().
Phases are slightly larger but confidence values slightly reduced.

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: Autofocus CAF/PDAF stability tweak
c233e40 
When in Continuous AF mode using PDAF, only move the lens when
phase has had the same sign for at least 4 frames. This reduces
lens wobble in e.g. noisy conditions.

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: AutoFocus tweak earlyTerminationByPhase()
e03aba4 
Increase threshold for ETBP, from "confEpsilon" to "confThresh".
Correct sign test to take account of pdafGain sign (typically -ve).
Reduce allowed extrapolation range, but relax the check in the
case of Continuous AF, when we go back into the PDAF closed loop.

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: Autofocus to use AWB statistics; re-trigger
4a90536 
Analyse AWB statistics: used both for scene change detection
and to detect IR lighting (when a flag is set in the tuning file).

Option to suppress PDAF altogether when IR lighting is detected.

Rather than being based solely on PDAF "dropout", allow a scan to
be (re-)triggered whenever the scene changes and then stabilizes,
based on contrast and average RGB statistics within the AF window.

[XXX Until the following patch, this may result in significant
extra lens movement when PDAF is unavailable, since every scan
will start from the minimum lens position.]

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: controller: AutoFocus bidirectional scanning
e434f1c 
To reduce unnecessary lens movements, allow the CDAF-based
search procedure to start from either end of the range;
or if not near an end, from the current lens position.

This sometimes requires a second coarse scan, if the first
one started in the middle and did not find peak contrast.

Shorten the fine scan from 5 steps to 3 steps; allow fine scan
to be omitted altogether when "step_fine": 0 in the tuning file.

Move updateLensPosition() out of startProgrammedScan() to avoid
calling it more than once per iteration.

Signed-off-by: Nick Hollinghurst <[email protected]>
@njhollinghurst
ipa: rpi: Update IMX708 camera tuning files for AutoFocus changes
37348b2 
Explicitly add new parameters: "retrigger_ratio", "retrigger_delay",
"check_for_ir". Tweak other parameters to suit algorithm changes.
(Though existing tuning files should still work acceptably.)

Add AfSpeedFast parameters for the Raspberry Pi V3 standard lens.

Signed-off-by: Nick Hollinghurst <[email protected]>
@naushir
Copy link
Collaborator

naushir commented Jun 16, 2025

Can I suggest rebasing to https://git.libcamera.org/libcamera/libcamera.git/ (master). We can then post it upstream. Should not have any merge issues...

@njhollinghurst njhollinghurst changed the base branch from main to next June 16, 2025 13:02
@njhollinghurst
Copy link
Collaborator Author

Conflicts in libcamera/pipeline/rpi. I may need to make a new branch for that.

@naushir
Copy link
Collaborator

naushir commented Jun 16, 2025

Hmmm, I will be rebasing next on top of master very soon. Perhaps wait till I've done that then we can look at master with this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@naushir naushir Awaiting requested review from naushir

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@njhollinghurst @pinefeat @naushir