Changed ISTA and FISTA so that g can be None

[MargaretDuff]

Describe your changes

Changed ISTA and FISTA so that g can be:
" g : Function or None
Convex function with simple proximal operator. If None is passed, the algorithm will use the ZeroFunction."

Describe any testing you have performed

Please add any demo scripts to CIL-Demos/misc/

Link relevant issues

Closes #1567

Checklist when you are ready to request a review

• I have performed a self-review of my code
• I have added docstrings in line with the guidance in the developer guide
• I have implemented unit tests that cover any new or modified functionality
• CHANGELOG.md has been updated with any functionality change
• Request review from all relevant developers
• Change pull request label to 'Waiting for review'

Contribution Notes

Please read and adhere to the developer guide and local patterns and conventions.

• The content of this Pull Request (the Contribution) is intentionally submitted for inclusion in CIL (the Work) under the terms and conditions of the Apache-2.0 License.
• I confirm that the contribution does not violate any intellectual property rights of third parties

[jakobsj]

LGTM. Maybe for a separate issue, but would it make sense to allow the function f to be none (maybe not both at the same time)?

[epapoutsellis]

Maybe in the docs we can write that ISTA is GD when g=None and FISTA is Accelerated Gradient Descent by Nesterov ( INTRODUCTORY LECTURES ON CONVEX OPTIMIZATION algorithm 2.2.9), when g=None.

[MargaretDuff]

Thanks @epapoutsellis - will add that!

[MargaretDuff]

@jakobsj - what's the thought around f being None? What would it default to?

[jakobsj]

ZeroFunction. Just minimize g. I wonder if there are cases that is a sensible thing to do. For example, if it is a simple 1- or 2-norm I suppose the minimizer is simply the zero vector. Could be relevant for completeness, for testing purposes and possibly for cases we haven't thought of yet.

…

On Fri, 24 Nov 2023 at 10:42, Margaret Duff ***@***.***> wrote: @jakobsj <https://github.com/jakobsj> - what's the thought around f being None? What would it default to? — Reply to this email directly, view it on GitHub <#1580 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACMDSCC3LAATG4GIHOROYGTYGBTYVAVCNFSM6AAAAAA7UZBOGCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRVGM4TONJRGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[epapoutsellis]

If f=None and g!=None then this is called Proximal Point Algorithm which I am not sure acceleration can be applied in Nesterov way.

[MargaretDuff]

@jakobsj @epapoutsellis - I added options for f to be None too. Wasn't sure what to do with the step-size here, so set it to be 1 if f=None and step_size=None is passed

[epapoutsellis]

@MargaretDuff just to let you know that this is related to #1586
These changes could live in PGA and inherited by ISTA/FISTA.

[jakobsj]

LGTM! Nice extension to either f or g being None. Also nice tests, making direct use of the proximal. I didn't check, but what happens if both f and g are set to None? Should this throw an error or might it make sense to return a zero vector in the shape/geometry of the initial guess?

[MargaretDuff]

Thanks, @jakobsj. I have added a warning if f and g are None. It should return the initial value.

[paskino]

I propose to raise an error in case both f and g are None, and a couple of formatting changes.

[MargaretDuff]

Jenkins is happy

[gfardell]

Changes requested in individual comments. It looks like a good change to FISTA/ISTA though.

Changes have been implemented - Thanks for reviewing!